Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Applies to: ✅ Azure Data Explorer
The .ingest-from-storage-queued
command is used with the .list blobs command to queue blobs for ingestion into a table. This command supports bulk ingestion of an entire storage container, a specific folder within a container, or all blobs that match a given prefix and suffix.
Note
This ingestion method is intended for exploration and prototyping. Don't use it in production or high-volume scenarios. For more information about ingestion options, see Data ingestion overview.
Note
Queued ingestion commands are run on the data ingestion URI endpoint https://ingest-<YourClusterName><Region>.kusto.chinacloudapi.cn
.
Permissions
You must have at least Table Ingestor permissions to run this command.
Syntax
.ingest-from-storage-queued
into
table
[database(DatabaseName).]TableName[EnableTracking
=EnableTracking] [SkipBatching
=SkipBatching] [CompressionFactor
=CompressionFactor] [with (IngestionPropertyName = IngestionPropertyValue [, ...])] <|
IngestionSource
Learn more about syntax conventions.
Parameters
Name | Type | Required | Description |
---|---|---|---|
DatabaseName | string |
The name of the database into which to ingest data. If no database name is provided, the request's context database is used. | |
TableName | string |
✔️ | The name of the table into which to ingest data. |
EnableTracking | boolean |
Determines whether to track the blob ingestion. For more information, see .show queued ingestion operations command. The default is false . |
|
SkipBatching | boolean |
If set to true , the blobs are ingested individually rather than batched together with other blobs. The default value is false . |
|
CompressionFactor | real |
The compression factor (ratio) between the original size and the compressed size of blobs. Compression factor is used to estimate the original size of the data for batching purposes, when blobs are provided in a compressed format. | |
IngestionPropertyName, IngestionPropertyValue | string |
Optional ingestion properties. For more information about ingestion properties, see Data ingestion properties. | |
IngestionSource | table | ✔️ | The ingestion source. The source is a list of blobs returned using the .list blobs command. |
Note
The .list blobs
command can be used with the .ingest-from-storage-queued
command to return the blobs you want to ingest. For detailed information about the command and a full list of its parameters, see .list blobs command.
Returns
The result of the command is a table with one row and one column.
Name | Type | Description |
---|---|---|
IngestionOperationId | string |
A unique ID used to track the set of blobs, whether or not tracking is enabled. |
ClientRequestId | string |
The client request ID of the command. |
OperationInfo | string |
Displays the command to run to retrieve the current status of the operation. |
Note
This command doesn't modify the schema of the target table. If necessary, the data is converted to fit the table's schema during ingestion. Extra columns are ignored and missing columns are treated as null values.
Example
The example in this section shows how to use the syntax to help you get started.
Ingest all blobs in a folder
The following example queues all blobs within a folder for ingestion using the cluster's system managed identity.
.ingest-from-storage-queued into table database('LogDb').RawLogs
EnableTracking=true
with (
format='csv',
ingestionMappingReference='LogMapping',
ignoreFirstRecord=false
)
<|
.list blobs (
"https://mystorageaccount.blob.core.chinacloudapi.cn/datasets/myfolder;managed_identity=system"
)