在 Azure Monitor 的 Log Analytics 工作区中收集 Azure 平台日志Collect Azure platform logs in Log Analytics workspace in Azure Monitor

Azure 中的平台日志(包括 Azure 活动日志和资源日志)提供 Azure 资源及其所依赖的 Azure 平台的详细诊断和审核信息。Platform logs in Azure, including Azure Activity log and resource logs, provide detailed diagnostic and auditing information for Azure resources and the Azure platform they depend on. 本文介绍如何收集 Log Analytics 工作区中的资源日志,以便可以结合通过强大的日志查询在 Azure Monitor 日志中收集的其他监视数据对其进行分析,并利用其他 Azure Monitor 功能,例如警报和可视化。This article describes collecting resource logs in a Log Analytics workspace which allows you to analyze it with other monitoring data collected in Azure Monitor Logs using powerful log queries and also to leverage other Azure Monitor features such as alerts and visualizations.

如何处理工作区中的平台日志What you can do with platform logs in a workspace

将平台日志收集到 Log Analytics 工作区中可以统一分析所有 Azure 资源的日志,并利用适用于 Azure Monitor 日志的所有功能,包括:Collecting platform logs into a Log Analytics workspace allows you to analyze the logs of all your Azure resources together and to take advantage of all the features available to Azure Monitor Logs which includes the following:

  • 日志查询 - 使用强大的查询语言创建日志查询,以快速分析和洞察诊断数据,并结合从 Azure Monitor 中的其他源收集的数据对其进行分析。Log queries - Create log queries using a powerful query language to quickly analyze and gain insights into your diagnostic data, and to analyze it with data collected from other sources in Azure Monitor.
  • 警报 - 使用 Azure Monitor 中的日志警报获取在资源日志中识别到的关键状况和模式的主动通知。Alerting - Get proactive notification of critical conditions and patterns identified in your resource logs using log alerts in Azure Monitor.
  • 可视化 - 将日志查询的结果固定到 Azure 仪表板,或将其作为交互式报告的一部分包含在工作簿中。Visualizations - Pin the results of a log query to an Azure dashboard or include it in a workbook as part of an interactive report.

先决条件Prerequisites

如果没有工作区,需要创建新的工作区You need to create a new workspace if you don't already have one. 只要配置设置的用户同时拥有两个订阅的相应 RBAC 访问权限,工作区就不必位于发送日志的资源所在的订阅中。The workspace does not have to be in the same subscription as the resource sending logs as long as the user who configures the setting has appropriate RBAC access to both subscriptions.

创建诊断设置Create a diagnostic setting

通过创建 Azure 资源的诊断设置,将平台日志发送到 Log Analytics 工作区和其他目标。Send platform logs to a Log Analytics workspace and other destinations by creating a diagnostic setting for an Azure resource. 有关详细信息,请参阅创建诊断设置以收集 Azure 中的日志和指标See Create diagnostic setting to collect logs and metrics in Azure for details.

活动日志集合Activity log collection

可以将任一订阅中的活动日志发送到最多五个 Log Analytics 工作区。You can send the Activity log from any single subscription to up to five Log Analytics workspaces. Log Analytics 工作区中收集的资源日志数据存储在 AzureActivity 表中 。Resource log data collected in a Log Analytics workspace is stored in the AzureActivity table.

资源日志收集模式Resource log collection mode

Azure Monitor 日志的结构中所述,在 Log Analytics 工作区中收集的资源日志存储在表中。Resource log data collected in a Log Analytics workspace is stored in tables as described in Structure of Azure Monitor Logs. 资源日志使用的表取决于资源使用的收集类型:The tables used by resource logs depend on what type of collection the resource is using:

  • Azure 诊断 - 所有数据将写入到 AzureDiagnostics 表中。Azure diagnostics - All data written is to the AzureDiagnostics table.
  • 特定于资源 - 每个类别的资源的数据将写入到单独的表中。Resource-specific - Data is written to individual table for each category of the resource.

Azure 诊断模式Azure Diagnostics mode

在此模式下,任何诊断设置中的所有数据将收集到 AzureDiagnostics 表中。In this mode, all data from any diagnostic setting will be collected in the AzureDiagnostics table. 这是当今大多数 Azure 服务使用的传统方法。This is the legacy method used today by most Azure services.

由于多个资源类型会将数据发送到同一个表,因此其架构是所收集的所有不同数据类型的架构的超集。Since multiple resource types send data to the same table, its schema is the superset of the schemas of all the different data types being collected.

在以下示例中,以下数据类型的诊断设置将收集到同一个工作区中:Consider the following example where diagnostic settings are being collected in the same workspace for the following data types:

  • 服务 1 的审核日志(架构由列 A、B 和 C 组成)Audit logs of service 1 (having a schema consisting of columns A, B, and C)
  • 服务 1 的错误日志(架构由列 D、E 和 F 组成)Error logs of service 1 (having a schema consisting of columns D, E, and F)
  • 服务 2 的审核日志(架构由列 G、H 和 I 组成)Audit logs of service 2 (having a schema consisting of columns G, H, and I)

AzureDiagnostics 表的外观如下所示:The AzureDiagnostics table will look as follows:

ResourceProviderResourceProvider CategoryCategory AA BB CC DD EE FF GG HH II
Microsoft.Service1Microsoft.Service1 AuditLogsAuditLogs x1x1 y1y1 z1z1
Microsoft.Service1Microsoft.Service1 ErrorLogsErrorLogs q1q1 w1w1 e1e1
Microsoft.Service2Microsoft.Service2 AuditLogsAuditLogs j1j1 k1k1 l1l1
Microsoft.Service1Microsoft.Service1 ErrorLogsErrorLogs q2q2 w2w2 e2e2
Microsoft.Service2Microsoft.Service2 AuditLogsAuditLogs j3j3 k3k3 l3l3
Microsoft.Service1Microsoft.Service1 AuditLogsAuditLogs x5x5 y5y5 z5z5
......

特定于资源Resource-Specific

在此模式下,将会根据在诊断设置中选择的每个类别,在所选工作区中创建各个表。In this mode, individual tables in the selected workspace are created for each category selected in the diagnostic setting. 建议使用此方法,因为它可以大幅简化日志查询中数据的处理、更好地发现架构及其结构、改善引入延迟和查询时间方面的性能、可以授予针对特定表的 RBAC 权限,等等。This method is recommended since it makes it much easier to work with the data in log queries, provides better discoverability of schemas and their structure, improves performance across both ingestion latency and query times, and the ability to grant RBAC rights on a specific table. 所有 Azure 服务最终都会迁移到特定于资源的模式。All Azure services will eventually migrate to the Resource-Specific mode.

以上示例会创建三个表:The example above would result in three tables being created:

  • 如下所示的表 Service1AuditLogsTable Service1AuditLogs as follows:

    资源提供程序Resource Provider CategoryCategory AA BB CC
    Service1Service1 AuditLogsAuditLogs x1x1 y1y1 z1z1
    Service1Service1 AuditLogsAuditLogs x5x5 y5y5 z5z5
    ......
  • 如下所示的表 Service1ErrorLogsTable Service1ErrorLogs as follows:

    资源提供程序Resource Provider CategoryCategory DD EE FF
    Service1Service1 ErrorLogsErrorLogs q1q1 w1w1 e1e1
    Service1Service1 ErrorLogsErrorLogs q2q2 w2w2 e2e2
    ......
  • 如下所示的表 Service2AuditLogsTable Service2AuditLogs as follows:

    资源提供程序Resource Provider CategoryCategory GG HH II
    Service2Service2 AuditLogsAuditLogs j1j1 k1k1 l1l1
    Service2Service2 AuditLogsAuditLogs j3j3 k3k3 l3l3
    ......

选择收集模式Select the collection mode

大多数 Azure 资源在“Azure 诊断”或“特定于资源”模式下将数据写入工作区,而不允许用户选择模式。 Most Azure resources will write data to the workspace in either Azure Diagnostic or Resource-Specific mode without giving you a choice. 有关使用哪种模式的详细信息,请参阅每个服务的文档See the documentation for each service for details on which mode it uses. 所有 Azure 服务最终都会使用特定于资源的模式。All Azure services will eventually use Resource-Specific mode. 在进行这种过渡期间,某些资源允许在诊断设置中选择模式。As part of this transition, some resources will allow you to select a mode in the diagnostic setting. 应该为任何新的诊断设置指定特定于资源的模式,因为这可以更轻松地管理数据,并可帮助你在今后避免复杂的迁移。You should specify resource-specific mode for any new diagnostic settings since this makes the data easier to manage and may help you to avoid complex migrations at a later date.

诊断设置模式选择器

Note

目前,只能在 Azure 门户中配置诊断设置时选择“Azure 诊断”和“特定于资源”模式。 Currently, Azure diagnostics and Resource-specific mode can only be selected when configuring the diagnostic setting in the Azure portal. 如果使用 CLI、PowerShell 或 REST API 配置设置,则模式默认为“Azure 诊断”。 If you configure the setting using CLI, PowerShell, or Rest API, it will default to Azure diagnostic.

可将现有的诊断设置修改为特定于资源的模式。You can modify an existing diagnostic setting to resource-specific mode. 在这种情况下,已收集的数据将保留在 AzureDiagnostics 表中,直到根据工作区的保留设置删除了这些数据。In this case, data that was already collected will remain in the AzureDiagnostics table until it's removed according to your retention setting for the workspace. 新数据将收集到专用表中。New data will be collected in to the dedicated table. 可以使用 union 运算符跨两个表查询数据。Use the union operator to query data across both tables.

有关支持特定于资源模式的 Azure 服务的公告,请继续阅读 Azure 更新博客。Continue to watch Azure Updates blog for announcements about Azure services supporting Resource-Specific mode.

AzureDiagnostics 中的列限制Column limit in AzureDiagnostics

Azure Monitor 日志中的任何一个表限制为 500 个属性。There is a 500 property limit for any table in Azure Monitor Logs. 一旦达到该限制,在引入时,包含不属于前 500 个属性的数据的所有行将被删除。Once this limit is reached, any rows containing data with any property outside of the first 500 will be dropped at ingestion time. AzureDiagnostics 表特别容易超过此限制,因为它包含写入到其中的所有 Azure 服务的属性。The AzureDiagnostics table is in particular susceptible to this limit since it includes properties for all Azure services writing to it.

如果从多个服务收集资源日志, AzureDiagnostics 可能会超过此限制,因此数据将会丢失。If you're collecting resource logs from multiple services, AzureDiagnostics may exceed this limit, and data will be missed. 在所有 Azure 服务都支持特定于资源的模式之前,应将资源配置为写入到多个工作区,以减少达到 500 列限制的可能性。Until all Azure services support resource-specific mode, you should configure resources to write to multiple workspaces to reduce the possibility of reaching the 500 column limit.

Azure 数据工厂Azure Data Factory

Azure 数据工厂包含非常详细的日志集,已知它是一个写入大量列的服务,可能会导致 AzureDiagnostics 超过其限制。Azure Data Factory, because of a very detailed set of logs, is a service that is known to write a large number of columns and potentially cause AzureDiagnostics to exceed its limit. 对于在支持特定于资源的模式之前配置的任何诊断设置,将会针对任一活动,为每个唯一命名的用户参数创建一个新列。For any diagnostic settings configured before the resource-specific mode was enabled there will be a new column created for every uniquely-named user parameter against any activity. 由于活动输入和输出的详细特性,将会创建更多列。More columns will be created because of the verbose nature of activity inputs and outputs.

应尽快迁移日志,以使用特定于资源的模式。You should migrate your logs to use the resource-specific mode as soon as possible. 如果无法立即做到这一点,一种临时的替代方案是将 Azure 数据工厂日志隔离到其自身的工作区,以尽量减少这些日志影响工作区中收集的其他日志类型的可能性。If you are unable to do so immediately, an interim alternative is to isolate Azure Data Factory logs into their own workspace to minimize the chance of these logs impacting other log types being collected in your workspaces.

后续步骤Next steps