Azure 数据资源管理器的数据引入属性Azure Data Explorer data ingestion properties

数据引入是指在 Azure 数据资源管理器中将数据添加到表中供查询这一过程。Data ingestion is the process by which data is added to a table and is made available for query in Azure Data Explorer. 可以将属性添加到 with 关键字后面的引入命令。You add properties to the ingestion command after the with keyword.

引入属性Ingestion properties

下表列出了 Azure 数据资源管理器支持的属性,对其进行了说明,并提供了示例:The following table lists the properties supported by Azure Data Explorer, describes them, and provides examples:

propertiesProperty 说明Description 示例Example
ingestionMapping 一个字符串值,表示如何将数据从源文件映射到表中的实际列。A string value that indicates how to map data from the source file to the actual columns in the table. 使用相关的映射类型定义 format 值。Define the format value with the relevant mapping type. 请参阅数据映射See data mappings. with (format="json", ingestionMapping = "[{\"column\":\"rownumber\", \"Properties\":{\"Path\":\"$.RowNumber\"}}, {\"column\":\"rowguid\", \"Properties\":{\"Path\":\"$.RowGuid\"}}]")
(已弃用:avroMappingcsvMappingjsonMapping(deprecated: avroMapping, csvMapping, jsonMapping)
ingestionMappingReference 一个字符串值,表示如何使用命名的映射策略对象将数据从源文件映射到表中的实际列。A string value that indicates how to map data from the source file to the actual columns in the table using a named mapping policy object. 使用相关的映射类型定义 format 值。Define the format value with the relevant mapping type. 请参阅数据映射See data mappings. with (format="csv", ingestionMappingReference = "Mapping1")
(已弃用:avroMappingReferencecsvMappingReferencejsonMappingReference(deprecated: avroMappingReference, csvMappingReference, jsonMappingReference)
creationTime 日期/时间值(格式化为 ISO8601 字符串),在创建引入的数据盘区时使用。The datetime value (formatted as an ISO8601 string) to use at the creation time of the ingested data extents. 在未指定的情况下会使用当前值 (now())。If unspecified, the current value (now()) will be used. 在引入的数据较旧时可以重写默认值,以便正确应用保留策略。Overriding the default is useful when ingesting older data, so that the retention policy will be applied correctly. with (creationTime="2017-02-13T11:09:36.7992775Z")
extend_schema 一个布尔值,在指定的情况下会指示命令扩展表的架构(默认为 false)。A Boolean value that, if specified, instructs the command to extend the schema of the table (defaults to false). 此选项仅适用于 .append.set-or-append 命令。This option applies only to .append and .set-or-append commands. 进行架构扩展时,只能在表的末尾添加其他列。The only allowed schema extensions have additional columns added to the table at the end. 如果原始表架构为 (a:string, b:int),则 (a:string, b:int, c:datetime, d:string) 是有效的架构扩展,而 (a:string, c:datetime) 则不是。If the original table schema is (a:string, b:int), a valid schema extension would be (a:string, b:int, c:datetime, d:string), but (a:string, c:datetime) wouldn't be valid
folder 使用 ingest-from-query 命令时分配给表的文件夹。For ingest-from-query commands, the folder to assign to the table. 如果表已存在,则此属性会重写表的文件夹If the table already exists, this property will override the table's folder. with (folder="Tables/Temporary")
format 数据格式(请参阅支持的数据格式)。The data format (see supported data formats). with (format="csv")
ingestIfNotExists 一个字符串值,如果已指定,则在表的数据已通过一个具有相同值的 ingest-by: 标记进行了标记时,引入操作会失败。A string value that, if specified, prevents ingestion from succeeding if the table already has data tagged with an ingest-by: tag with the same value. 这确保了幂等数据引入。This ensures idempotent data ingestion. 有关详细信息,请参阅 ingest-by: tagsFor more information, see ingest-by: tags. 属性 with (ingestIfNotExists='["Part0001"]', tags='["ingest-by:Part0001"]') 表示,如果已存在带 ingest-by:Part0001 标记的数据,请勿完成当前的引入操作。The properties with (ingestIfNotExists='["Part0001"]', tags='["ingest-by:Part0001"]') indicate that if data with the tag ingest-by:Part0001 already exists, then don't complete the current ingestion. 如果该数据不存在,则在执行这个新的引入操作时,应该设置此标记(防止将来有人尝试再次引入相同的数据)。If it doesn't already exist, this new ingestion should have this tag set (in case a future ingestion attempts to ingest the same data again.)
ignoreFirstRecord 一个布尔值,在设置为 true 的情况下表示引入操作应该忽略每个文件的第一条记录。A Boolean value that, if set to true, indicates that ingestion should ignore the first record of every file. 如果文件中的第一条记录是列名称,则此属性适用于 CSV 和类似格式的文件。This property is useful for files in CSVand similar formats, if the first record in the file are the column names. 默认为 falseBy default, false is assumed. with (ignoreFirstRecord=false)
persistDetails 一个布尔值,在指定的情况下表示命令应该保留详细的结果(即使命令已成功执行),这样 .show operation details 命令就能够检索它们。A Boolean value that, if specified, indicates that the command should persist the detailed results (even if successful) so that the .show operation details command could retrieve them. 默认为 falseDefaults to false. with (persistDetails=true)
policy_ingestiontime 一个布尔值,在指定的情况下,说明是否在通过此命令创建的表上启用引入时间策略A Boolean value that, if specified, describes whether to enable the Ingestion Time Policy on a table that is created by this command. 默认为 trueThe default is true. with (policy_ingestiontime=false)
recreate_schema 一个布尔值,在指定的情况下,说明此命令是否会重新创建表的架构。A Boolean value that, if specified, describes whether the command may recreate the schema of the table. 此属性只适用于 .set-or-replace 命令。This property applies only to the .set-or-replace command. 在同时设置的情况下,此属性优先于 extend_schema 属性。This property takes precedence over the extend_schema property if both are set. with (recreate_schema=true)
tags 与引入的数据(格式化为 JSON 字符串)相关联的标记的列表A list of tags to associate with the ingested data, formatted as a JSON string with (tags="['Tag1', 'Tag2']")
validationPolicy 一个 JSON 字符串,表示在引入期间要运行哪些验证。A JSON string that indicates which validations to run during ingestion. 请参阅数据引入,了解不同选项的说明。See Data ingestion for an explanation of the different options. with (validationPolicy='{"ValidationOptions":1, "ValidationImplications":1}')(实际上,这是默认策略)。with (validationPolicy='{"ValidationOptions":1, "ValidationImplications":1}') (this is actually the default policy)
zipPattern 从包含 ZIP 存档的存储中引入数据时,请使用此属性。Use this property when ingesting data from storage that has a ZIP archive. 这是一个字符串值,表示在要引入的 ZIP 存档中选择具体文件时需使用的正则表达式。This is a string value indicating the regular expression to use when selecting which files in the ZIP archive to ingest. 存档中的所有其他文件会被忽略。All other files in the archive will be ignored. with (zipPattern="*.csv")

后续步骤Next steps