使用诊断日志监视 Azure 数据资源管理器的引入、命令和查询Monitor Azure Data Explorer ingestion, commands, and queries using diagnostic logs
Azure 数据资源管理器是一项快速、完全托管的数据分析服务,用于实时分析从应用程序、网站和 IoT 设备等资源流式传输的海量数据。Azure Data Explorer is a fast, fully managed data analytics service for real-time analysis on large volumes of data streaming from applications, websites, IoT devices, and more. Azure Monitor 诊断日志提供有关 Azure 资源操作的数据。Azure Monitor diagnostic logs provide data about the operation of Azure resources. Azure 数据资源管理器使用诊断日志获取有关引入成功、引入失败、命令和查询操作的见解。Azure Data Explorer uses diagnostic logs for insights on ingestion successes, ingestion failures, commands, and query operations. 可将操作日志导出到 Azure 存储、事件中心或 Log Analytics 以监视引入、命令和查询状态。You can export operation logs to Azure Storage, Event Hub, or Log Analytics to monitor ingestion, commands, and query status. 可将 Azure 存储和 Azure 事件中心的日志路由到 Azure 数据资源管理器群集中的某个表,以进一步分析。Logs from Azure Storage and Azure Event Hub can be routed to a table in your Azure Data Explorer cluster for further analysis.
重要
诊断日志数据可能包含敏感数据。Diagnostic log data may contain sensitive data. 请根据监视需求限制日志目标的权限。Restrict permissions of the logs destination according to your monitoring needs.
先决条件Prerequisites
- 如果没有 Azure 订阅,请创建一个试用版订阅。If you don't have an Azure subscription, create a Trial Subscription.
- 登录到 Azure 门户。Sign in to the Azure portal.
- 创建群集和数据库。Create a cluster and database.
设置 Azure 数据资源管理器群集的诊断日志Set up diagnostic logs for an Azure Data Explorer cluster
诊断日志可用于配置以下日志数据的收集:Diagnostic logs can be used to configure the collection of the following log data:
- 成功的引入操作:这些日志包含有关已成功完成的引入操作的信息。Successful ingestion operations: These logs have information about successfully completed ingestion operations.
- 失败的引入操作:这些日志包含有关失败的引入操作的详细信息,包括错误详细信息。Failed ingestion operations: These logs have detailed information about failed ingestion operations including error details.
然后可根据规范将数据存档到存储帐户、流式传输到事件中心,或发送到 Log Analytics。The data is then archived into a Storage account, streamed to an Event Hub, or sent to Log Analytics, as per your specifications.
启用诊断日志Enable diagnostic logs
诊断日志默认已禁用。Diagnostic logs are disabled by default. 若要启用诊断日志,请执行以下步骤:To enable diagnostic logs, do the following steps:
在 Azure 门户中,选择要监视的 Azure 数据资源管理器群集资源。In the Azure portal, select the Azure Data Explorer cluster resource that you want to monitor.
在“监视”下,选择“诊断设置” 。Under Monitoring, select Diagnostic settings.
选择“添加诊断设置”。 Select Add diagnostic setting.
在“诊断设置”窗口中: In the Diagnostics settings window:
- 选择诊断设置的 名称。Select Name for your diagnostic setting.
- 选择一个或多个目标:存储帐户、事件中心或 Log Analytics。Select one or more targets: a Storage account, Event Hub, or Log Analytics.
- 选择要收集的日志:
SucceededIngestion
、FailedIngestion
、Command
或Query
。Select logs to be collected:SucceededIngestion
,FailedIngestion
,Command
, orQuery
. - 选择要收集的指标(可选)。Select metrics to be collected (optional).
- 选择“保存”以保存新的诊断日志设置和指标。 Select Save to save the new diagnostic logs settings and metrics.
在几分钟内即会完成新的设置。New settings will be set in a few minutes. 日志随后会显示在配置的存档目标(存储帐户、事件中心或 Log Analytics)中。Logs then appear in the configured archival target (Storage account, Event Hub, or Log Analytics).
备注
如果将日志发送到 Log Analytics,则 SucceededIngestion
、FailedIngestion
、Command
和 Query
日志将分别存储在名为 SucceededIngestion
、FailedIngestion
、ADXCommand
和 ADXQuery
的 Log Analytics 表中。If you send logs to Log Analytics, the SucceededIngestion
, FailedIngestion
, Command
, and Query
logs will be stored in Log Analytics tables named: SucceededIngestion
, FailedIngestion
, ADXCommand
, ADXQuery
, respectively.
诊断日志架构Diagnostic logs schema
所有 Azure Monitor 诊断日志共享一个通用的顶级架构。All Azure Monitor diagnostic logs share a common top-level schema. Azure 数据资源管理器对其自身的事件使用唯一属性。Azure Data Explorer has unique properties for their own events. 所有日志均以 JSON 格式存储。All logs are stored in a JSON format.
引入日志架构Ingestion logs schema
日志 JSON 字符串包含下表中列出的元素:Log JSON strings include elements listed in the following table:
名称Name | 说明Description |
---|---|
timetime | 报告时间Time of the report |
ResourceIdresourceId | Azure Resource Manager 资源 IDAzure Resource Manager resource ID |
operationNameoperationName | 操作名称:'MICROSOFT.KUSTO/CLUSTERS/INGEST/ACTION'Name of the operation: 'MICROSOFT.KUSTO/CLUSTERS/INGEST/ACTION' |
operationVersionoperationVersion | 架构版本:'1.0'Schema version: '1.0' |
categorycategory | 操作类别。Category of the operation. SucceededIngestion 或 FailedIngestion 。SucceededIngestion or FailedIngestion . 成功的操作或失败的操作的属性不同。Properties differ for successful operation or failed operation. |
propertiesproperties | 操作的详细信息。Detailed information of the operation. |
成功引入操作日志Successful ingestion operation log
示例:Example:
{
"time": "",
"resourceId": "",
"operationName": "MICROSOFT.KUSTO/CLUSTERS/INGEST/ACTION",
"operationVersion": "1.0",
"category": "SucceededIngestion",
"properties":
{
"succeededOn": "2019-05-27 07:55:05.3693628",
"operationId": "b446c48f-6e2f-4884-b723-92eb6dc99cc9",
"database": "Samples",
"table": "StormEvents",
"ingestionSourceId": "66a2959e-80de-4952-975d-b65072fc571d",
"ingestionSourcePath": "https://kustoingestionlogs.blob.core.chinacloudapi.cn/sampledata/events8347293.json",
"rootActivityId": "d0bd5dd3-c564-4647-953e-05670e22a81d"
}
}
成功操作诊断日志的属性Properties of a successful operation diagnostic log
名称Name | 说明Description |
---|---|
succeededOnsucceededOn | 引入完成时间Time of ingestion completion |
operationIdoperationId | Azure 数据资源管理器引入操作 IDAzure Data Explorer ingestion operation ID |
databasedatabase | 目标数据库的名称Name of the target database |
表table | 目标表的名称Name of the target table |
ingestionSourceIdingestionSourceId | 引入数据源的 IDID of the ingestion data source |
ingestionSourcePathingestionSourcePath | 引入数据源或 Blob URI 的路径Path of the ingestion data source or blob URI |
rootActivityIdrootActivityId | 活动 IDActivity ID |
失败引入操作日志Failed ingestion operation log
示例:Example:
{
"time": "",
"resourceId": "",
"operationName": "MICROSOFT.KUSTO/CLUSTERS/INGEST/ACTION",
"operationVersion": "1.0",
"category": "FailedIngestion",
"properties":
{
"failedOn": "2019-05-27 08:57:05.4273524",
"operationId": "5956515d-9a48-4544-a514-cf4656fe7f95",
"database": "Samples",
"table": "StormEvents",
"ingestionSourceId": "eee56f8c-2211-4ea4-93a6-be556e853e5f",
"ingestionSourcePath": "https://kustoingestionlogs.blob.core.chinacloudapi.cn/sampledata/events5725592.json",
"rootActivityId": "52134905-947a-4231-afaf-13d9b7b184d5",
"details": "Permanent failure downloading blob. URI: ..., permanentReason: Download_SourceNotFound, DownloadFailedException: 'Could not find file ...'",
"errorCode": "Download_SourceNotFound",
"failureStatus": "Permanent",
"originatesFromUpdatePolicy": false,
"shouldRetry": false
}
}
失败操作诊断日志的属性Properties of a failed operation diagnostic log
名称Name | 说明Description |
---|---|
failedOnfailedOn | 引入完成时间Time of ingestion completion |
operationIdoperationId | Azure 数据资源管理器引入操作 IDAzure Data Explorer ingestion operation ID |
databasedatabase | 目标数据库的名称Name of the target database |
表table | 目标表的名称Name of the target table |
ingestionSourceIdingestionSourceId | 引入数据源的 IDID of the ingestion data source |
ingestionSourcePathingestionSourcePath | 引入数据源或 Blob URI 的路径Path of the ingestion data source or blob URI |
rootActivityIdrootActivityId | 活动 IDActivity ID |
详细信息details | 失败和错误消息的详细说明Detailed description of the failure and error message |
errorCodeerrorCode | 错误代码Error code |
failureStatusfailureStatus | Permanent 或 Transient 。Permanent or Transient . 重试暂时性故障可能会成功。Retry of a transient failure may succeed. |
originatesFromUpdatePolicyoriginatesFromUpdatePolicy | 如果故障源自更新策略,则为 TrueTrue if failure originates from an update policy |
shouldRetryshouldRetry | 如果重试可以成功,则为 TrueTrue if retry may succeed |
后续步骤Next steps
- 使用指标来监视群集运行状况Use metrics to monitor cluster health
- 教程:在 Azure 数据资源管理器中引入和查询监视数据,可帮助获取引入诊断日志Tutorial: Ingest and query monitoring data in Azure Data Explorer for ingestion diagnostic logs