Kusto.Ingest 错误和异常Kusto.Ingest errors and exceptions

10/29/2020

在客户端上处理引入时发生的任何错误都通过 C# 异常来指示。Any error during the ingestion handling on the client side is indicated by a C# exception.

失败数Failures

KustoDirectIngestClient 异常KustoDirectIngestClient exceptions

尝试从多个源进行引入时，在引入过程中可能会发生错误。While attempting to ingest from multiple sources, errors might occur during the ingestion process. 如果某个源的引入失败，则会记录该失败，客户端会继续引入剩余的源。If an ingestion fails for one of the sources, it's logged and the client continues to ingest the remaining sources. 完成用于引入的所有源后，会引发 IngestClientAggregateException，其中包含 IList<IngestClientException> IngestionErrors 成员。After going over all sources for ingestion, an IngestClientAggregateException is thrown, containing the IList<IngestClientException> IngestionErrors member.

IngestClientException 及其派生类包含 IngestionSource 字段和 Error 字段。IngestClientException and its derived classes contain a field IngestionSource and an Error field. 这两个字段共同创建了一个映射，从引入失败的源映射到尝试引入时发生的错误。The two fields together create a mapping, from the source that failed ingestion, to the error that occurred while attempting the ingestion. 此信息可用于 IngestionErrors 列表，用以调查哪些源引入失败以及原因是什么。The information can be used in the IngestionErrors list to investigate which sources failed ingestion and why. IngestClientAggregateException 异常还包含布尔属性 GlobalError，用于指示是否所有源都发生了某个错误。The IngestClientAggregateException exception also contains a boolean property GlobalError, that indicates whether an error occurred for all sources.

从文件或 blob 引入时的失败Failures ingesting from files or blobs

如果尝试从 blob 或文件引入时发生引入失败，则即使 deleteSourceOnSuccess 标志设置为 true，也不会删除引入源。If an ingestion failure occurs while attempting to ingest from a blob or file, the ingestion sources won't be deleted, even if the deleteSourceOnSuccess flag is set to true. 将保留源供进一步分析。The sources are preserved for further analysis. 在了解错误来源后，如果该错误不是源自引入源本身，则客户端的用户可以尝试重新引入它。Once the origin of the error is understood, and if the error didn't originate from the ingestion source itself, then the user of the client may attempt to reingest it.

从 IDataReader 引入时的失败Failures ingesting from IDataReader

从 DataReader 引入时，要引入的数据将保存到默认位置为 <Temp Path>\Ingestions_<current date and time> 的一个临时文件夹中。While ingesting from DataReader, the data to ingest is saved to a temporary folder whose default location is <Temp Path>\Ingestions_<current date and time>. 成功引入后，始终会删除此默认文件夹。This default folder is always deleted after a successful ingestion.

在 IngestFromDataReader 和 IngestFromDataReaderAsync 方法中，retainCsvOnFailure 标志（其默认值为 false）决定了引入失败后是否应保留文件。In the IngestFromDataReader and IngestFromDataReaderAsync methods, the retainCsvOnFailure flag, whose default value is false, determines whether the files should be kept after a failed ingestion. 如果此标志设置为 false，则不会持久保存引入失败的数据，因此很难了解发生了什么问题。If this flag is set to false, data that fails the ingestion won't be persisted, making it hard to understand what went wrong.

KustoQueuedIngestClient 异常KustoQueuedIngestClient exceptions

KustoQueuedIngestClient 通过将消息上传到 Azure 队列来引入数据。KustoQueuedIngestClient ingests data by uploading messages to an Azure queue. 如果在排队过程之前或排队过程中发生错误，则在该过程结束时会引发 IngestClientAggregateException。If an error occurs before or during the queueing process, an IngestClientAggregateException is thrown at the end of the process. 引发的异常包括一个 IngestClientException 集合，该集合包含每个失败的源且尚未发布到队列。The thrown exception includes a collection of IngestClientException, that contains the source of each failure, and hadn't been posted to the queue. 还会引发尝试发布消息时发生的错误。The error that occurred while attempting to post the message is also thrown.

以文件或 blob 作为源发布到队列时的失败Posting to queue failures with a file or blob as a source

如果使用 KustoQueuedIngestClient 的 IngestFromFile/IngestFromBlob 方法时发生错误，则即使 deleteSourceOnSuccess 标志设置为 true，也不会删除源。If an error occurs while using the KustoQueuedIngestClient's IngestFromFile/IngestFromBlob methods, the sources aren't deleted, even if the deleteSourceOnSuccess flag is set to true. 相反，将保留源供进一步分析。Instead, the sources are preserved for further analysis.

在了解错误来源后，如果该错误不是源自引入源本身，则客户端的用户可以通过对失败的源使用相关的 IngestFromFile/IngestFromBlob 方法来尝试将数据重新排队。Once the origin of the error is understood, and if the error didn't originate from the ingestion source itself, the user of the client may attempt to requeue the data by using the relevant IngestFromFile/IngestFromBlob methods with the failed source.

使用 IDataReader 作为源发布到队列时的失败Posting to queue failures with IDataReader as a source

使用 DataReader 源时，发布到队列的数据将保存到默认位置为 <Temp Path>\Ingestions_<current date and time> 的一个临时文件夹中。While using a DataReader source, the data to post to the queue is saved to a temporary folder whose default location is <Temp Path>\Ingestions_<current date and time>. 将数据成功发布到队列后，始终会删除此文件夹。This folder is always deleted after the data has been successfully posted to the queue. 在 IngestFromDataReader 和 IngestFromDataReaderAsync 方法中，retainCsvOnFailure 标志（其默认值为 false）决定了引入失败后是否应保留文件。In the IngestFromDataReader and IngestFromDataReaderAsync methods, the retainCsvOnFailure flag, whose default value is false, determines whether the files should be kept after a failed ingestion. 如果此标志设置为 false，则不会持久保存引入失败的数据，因此很难了解发生了什么问题。If this flag is set to false, data that fails the ingestion won't be persisted, making it hard to understand what went wrong.

常见失败Common Failures

错误Error	ReasonReason	缓解措施Mitigation
数据库名称不存在Database name doesn't exist	数据库不存在The database doesn't exist	在 `kustoIngestionProperties` 中检查数据库名称/创建数据库Check the database name at `kustoIngestionProperties`/Create the database
找不到类型为 "Table" 的实体“不存在的表名”。Entity 'table name that doesn't exist' of kind 'Table' wasn't found.	该表不存在，也没有 CSV 映射。The table doesn't exist and there's no CSV mapping.	添加 CSV 映射/创建所需的表Add CSV mapping / create the required table
Blob 被排除，原因如下：必须通过 jsonMapping 参数引入 JSON 模式Blob excluded for reason: JSON pattern must be ingested with jsonMapping parameter	进行 JSON 引入时未提供 JSON 映射。JSON ingestion when no JSON mapping is provided.	提供 JSON 映射Provide a JSON mapping
下载 blob 失败：“远程服务器返回了错误: (404)未找到。”Failed to download blob: 'The remote server returned an error: (404) Not Found.'	Blob 不存在。The blob doesn't exist.	请验证该 blob 是否存在。Verify that the blob exists. 如果它存在，请重试并联系 Kusto 团队If it exists, retry and contact the Kusto team
JSON 列映射无效：两个或多个映射元素指向同一列。JSON column mapping isn't valid: Two or more mapping elements point to the same column.	JSON 映射有 2 个不同路径的列JSON mapping has 2 columns with different paths	纠正 JSON 映射Fix JSON mapping
EngineError - [UtilsException] `IngestionDownloader.Download`：一个或多个文件下载失败（请在 KustoLogs 中搜索 ActivityID: 和 RootActivityId:）EngineError - [UtilsException] `IngestionDownloader.Download`: One or more files failed to download (search KustoLogs for ActivityID:, RootActivityId:)	一个或多个文件下载失败。One or more files failed to download.	重试Retry
无法分析:ID 为“”的流的 CSV 格式错误，无法通过 ValidationOptions 策略Failed to parse: Stream with ID '' has a malformed CSV format, failing per ValidationOptions policy	CSV 文件格式不正确（例如，每行的列数不相同）。Malformed CSV file (such as, not having the same number of columns on every line). 仅当验证策略设置为 `ValidationOptions` 时才会失败。Fails only when validation policy is set to `ValidationOptions`. ValidateCsvInputConstantColumnsValidateCsvInputConstantColumns	检查你的 CSV 文件。Check your CSV files. 此消息仅适用于 CSV/TSV 文件This message applies only to CSV/TSV files
`IngestClientAggregateException`，错误消息为“缺少有效共享访问签名的必需参数”`IngestClientAggregateException` with error message 'Missing mandatory parameters for valid Shared Access Signature'	使用的 SAS 是服务的，而不是存储帐户的The SAS being used is of the service, and not of the storage account	使用存储帐户的 SASUse the SAS of the storage account

引入错误代码Ingestion error codes

为了便于以编程方式处理引入失败，我们对失败消息进行了扩充，在其中包含了数值错误代码 (IngestionErrorCode enumeration)。To help handle ingestion failures programmatically, failure information is enriched with a numeric error code (IngestionErrorCode enumeration).

ErrorCodeErrorCode	ReasonReason
未知Unknown	发生未知错误Unknown error occurred
Stream_LowMemoryConditionStream_LowMemoryCondition	操作用尽了内存Operation ran out of memory
Stream_WrongNumberOfFieldsStream_WrongNumberOfFields	CSV 文档包含不一致的字段数CSV document has inconsistent number of fields
Stream_InputStreamTooLargeStream_InputStreamTooLarge	提交的用于引入的文档超出了允许的大小The document submitted for ingestion has exceeded the allowed size
Stream_NoDataToIngestStream_NoDataToIngest	找不到要引入的数据流Found no data streams to ingest
Stream_DynamicPropertyBagTooLargeStream_DynamicPropertyBagTooLarge	引入的数据中的某个动态列包含太多的唯一属性One of the dynamic columns in the ingested data contains too many unique properties
Download_SourceNotFoundDownload_SourceNotFound	从 Azure 存储下载源失败 - 未找到源Failed to download source from Azure storage - source not found
Download_AccessConditionNotSatisfiedDownload_AccessConditionNotSatisfied	从 Azure 存储下载源失败 - 访问被拒绝Failed to download source from Azure storage - Access denied
Download_ForbiddenDownload_Forbidden	从 Azure 存储下载源失败 - 不允许访问Failed to download source from Azure storage - Access not permitted
Download_AccountNotFoundDownload_AccountNotFound	从 Azure 存储下载源失败 - 未找到帐户Failed to download source from Azure storage - Account not found
Download_BadRequestDownload_BadRequest	从 Azure 存储下载源失败 - 请求不正确Failed to download source from Azure storage - Bad request
Download_NotTransientDownload_NotTransient	从 Azure 存储下载源失败 - 非暂时性错误Failed to download source from Azure storage - Not transient error
Download_UnknownErrorDownload_UnknownError	从 Azure 存储下载源失败 - 未知错误Failed to download source from Azure storage - Unknown error
UpdatePolicy_QuerySchemaDoesNotMatchTableSchemaUpdatePolicy_QuerySchemaDoesNotMatchTableSchema	调用更新策略失败。Failed to invoke update policy. 查询架构与表架构不匹配Query schema doesn't match table schema
UpdatePolicy_FailedDescendantTransactionUpdatePolicy_FailedDescendantTransaction	调用更新策略失败。Failed to invoke update policy. 后代事务性更新策略失败Failed descendant transactional update policy
UpdatePolicy_IngestionErrorUpdatePolicy_IngestionError	调用更新策略失败。Failed to invoke update policy. 发生引入错误Ingestion error occurred
UpdatePolicy_UnknownErrorUpdatePolicy_UnknownError	调用更新策略失败。Failed to invoke update policy. 发生未知错误Unknown error occurred
BadRequest_MissingJsonMappingtFailureBadRequest_MissingJsonMappingtFailure	没有使用 jsonMapping 参数引入 JSON 模式JSON pattern didn't ingest with the jsonMapping parameter
BadRequest_InvalidBlobBadRequest_InvalidBlob	引擎无法打开和读取非 zip blobEngine failed to open and read non-zip blob
BadRequest_EmptyBlobBadRequest_EmptyBlob	空 blobEmpty blob
BadRequest_EmptyArchiveBadRequest_EmptyArchive	zip 文件不包含任何已存档的元素The zip file doesn’t contain any archived elements
BadRequest_EmptyBlobUriBadRequest_EmptyBlobUri	指定的 blob URI 为空The specified blob URI is empty
BadRequest_DatabaseNotExistBadRequest_DatabaseNotExist	数据库不存在Database doesn't exist
BadRequest_TableNotExistBadRequest_TableNotExist	表不存在Table doesn't exist
BadRequest_InvalidKustoIdentityTokenBadRequest_InvalidKustoIdentityToken	Kusto 标识令牌无效Invalid Kusto identity token
BadRequest_UriMissingSasBadRequest_UriMissingSas	源自未知 Blob 存储的 Blob 路径未包含 SASBlob path without SAS from unknown blob storage
BadRequest_FileTooLargeBadRequest_FileTooLarge	尝试引入的文件太大Trying to ingest too large file
BadRequest_NoValidResponseFromEngineBadRequest_NoValidResponseFromEngine	未收到引入命令的有效回复No valid reply from ingest command
BadRequest_TableAccessDeniedBadRequest_TableAccessDenied	被拒绝访问表Access to table is denied
BadRequest_MessageExhaustedBadRequest_MessageExhausted	消息已用尽Message is exhausted
General_BadRequestGeneral_BadRequest	常规错误请求。General bad request. 在引入到不存在的数据库/表时可能会提示May hint at ingestion to non-existent Database/Table
General_InternalServerErrorGeneral_InternalServerError	发生了内部服务器错误Internal server error occurred

详细的异常参考信息Detailed exceptions reference

CloudQueuesNotFoundExceptionCloudQueuesNotFoundException

当数据管理群集未返回任何队列时引发Raised when no queues were returned from the Data Management cluster

基类：异常Base Class: Exception

字段名称Field Name	类型Type	含义Meaning
错误Error	StringString	尝试从 DM 检索队列时发生的错误The error that occurred while attempting to retrieve queues from the DM

仅当使用 Kusto 排队引入客户端时才适用。Relevant only when using the Kusto Queued Ingest Client. 在引入过程中，会多次尝试检索链接到 DM 的 Azure 队列。During the ingestion process, several attempts are made to retrieve the Azure Queues linked to the DM. 当这些尝试失败时，将在“错误”字段中引发包含失败原因的异常。When these attempts fail, the exception containing the reason for failure, is raised in the 'Error' field. 还可能会在“InnerException”字段中引发内部异常。Possibly an inner exception in the 'InnerException' field is also raised.

CloudBlobContainersNotFoundExceptionCloudBlobContainersNotFoundException

当数据管理群集未返回任何 blob 容器时引发Raised when no blob containers were returned from the Data Management cluster

基类：异常Base Class: Exception

字段名称Field Name	类型Type	含义Meaning
KustoEndpointKustoEndpoint	StringString	相关 DM 的终结点The endpoint of the relevant DM

仅当使用 Kusto 排队引入客户端时才适用。Relevant only when using the Kusto Queued Ingest Client.
当引入尚未存在于 Azure 容器中的源（例如文件、DataReader 或流）时，数据将上传到临时 blob 进行引入。When ingesting sources that aren't already in an Azure container, such as files, DataReader, or Stream, then the data uploads to a temporary blob for ingestion. 如果未找到要将数据上传到其中的容器，则会引发异常。The exception is raised when there are no containers found to upload the data to.

DuplicateIngestionPropertyExceptionDuplicateIngestionPropertyException

在多次配置了某个引入属性时引发Raised when an ingestion property is configured more than once

基类：异常Base Class: Exception

字段名称Field Name	类型Type	含义Meaning
PropertyNamePropertyName	StringString	重复属性的名称The name of the duplicate property

PostMessageToQueueFailedExceptionPostMessageToQueueFailedException

将消息发布到队列失败时引发Raised when posting a message to the queue fails

基类：异常Base Class: Exception

字段名称Field Name	类型Type	含义Meaning
QueueUriQueueUri	StringString	队列的 URIThe URI of the queue
错误Error	StringString	尝试发布到队列时生成的错误消息The error message that was generated while attempting to post to the queue

仅当使用 Kusto 排队引入客户端时才适用。Relevant only when using the Kusto Queued Ingest Client.
排队引入客户端通过将消息上传到相关的 Azure 队列来引入数据。The queued ingest client ingests data by uploading a message to the relevant Azure queue. 如果出现发布失败，则会引发异常。If there's a post failure, the exception is raised. 它将包含队列 URI，在“错误”字段中包含失败原因，并可能在“InnerException”字段中包含内部异常。It will contain the queue URI, the reason for the failure in the 'Error' field, and possibly an inner exception in the 'InnerException' field.

DataFormatNotSpecifiedExceptionDataFormatNotSpecifiedException

当需要指定数据格式但未在 IngestionProperties 中指定时引发Raised when a data format is required but not specified in IngestionProperties