Application Insights 中基于日志的指标和预先聚合的指标Log-based and pre-aggregated metrics in Application Insights

本文介绍基于日志的“传统”Application Insights 指标与目前以公共预览版提供的预先聚合指标之间的差别。This article explains the difference between "traditional" Application Insights metrics that are based on logs, and pre-aggregated metrics that are currently in public preview. 这两种类型的指标都可供 Application Insights 用户使用,每种指标在监视应用程序运行状况、诊断和分析方面发挥了独特的作用。Both types of metrics are available to the users of Application Insights, and each brings a unique value in monitoring application health, diagnostics and analytics. 检测应用程序的开发人员可以根据应用程序的大小、预期遥测量以及指标精度和警报方面的业务要求,确定哪种类型的指标最适合特定的方案。The developers who are instrumenting applications can decide which type of metric is best suited to a particular scenario, depending on the size of the application, expected volume of telemetry, and business requirements for metrics precision and alerting.

基于日志的指标Log-based Metrics

最近,Application Insights 中的应用程序监视遥测数据模型只是基于少量的预定义类型的事件,例如请求、异常、依赖项调用、页面视图,等等。开发人员可以使用 SDK 手动发出这些事件(编写显式调用该 SDK 的代码),或者依赖于自动检测产品中的自动事件收集功能。Until recently, the application monitoring telemetry data model in Application Insights was solely based on a small number of predefined types of events, such as requests, exceptions, dependency calls, page views, etc. Developers can use the SDK to either emit these events manually (by writing code that explicitly invokes the SDK) or they can rely on the automatic collection of events from auto-instrumentation. 在任一情况下,Application Insights 后端都会将所有收集的事件存储为日志。可以使用 Azure 门户中的 Application Insights 边栏选项卡作为分析和诊断工具来可视化日志中基于事件的数据。In either case, the Application Insights backend stores all collected events as logs, and the Application Insights blades in the Azure portal act as an analytical and diagnostic tool for visualizing event-based data from logs.

使用日志保留完整事件集能够为分析和诊断带来很大的帮助。Using logs to retain a complete set of events can bring great analytical and diagnostic value. 例如,可以获取对特定 URL 发出的确切请求计数,以及发出这些调用的非重复用户数。For example, you can get an exact count of requests to a particular URL with the number of distinct users who made these calls. 或者,可以获取详细的诊断跟踪,包括任何用户会话的异常和依赖项调用。Or you can get detailed diagnostic traces, including exceptions and dependency calls for any user session. 获取此类信息能够明显提高应用程序运行状况和使用情况的可见性,从而可以缩短诊断应用问题所需的时间。Having this type of information can significantly improve visibility into the application health and usage, allowing to cut down the time necessary to diagnose issues with an app.

同时,对于生成大量遥测数据的应用程序而言,收集完整事件集可能不切实际(甚至不可能)。At the same time, collecting a complete set of events may be impractical (or even impossible) for applications that generate a large volume of telemetry. 如果事件量过高,Application Insights 会实施多种遥测数据量缩减技术(例如采样筛选),以减少收集和存储的事件数量。For situations when the volume of events is too high, Application Insights implements several telemetry volume reduction techniques, such as sampling and filtering that reduce the number of collected and stored events. 遗憾的是,降低存储事件数量也会降低指标的准确性,因此,在幕后必须对日志中存储的事件执行查询时聚合。Unfortunately, lowering the number of stored events also lowers the accuracy of the metrics that, behind the scenes, must perform query-time aggregations of the events stored in logs.

备注

在 Application Insights 中,基于日志中存储的事件和测量值查询时聚合的指标称为基于日志的指标。In Application Insights, the metrics that are based on the query-time aggregation of events and measurements stored in logs are called log-based metrics. 这些指标通常在事件属性中具有许多维度,因此非常适合用于分析,但这些指标的准确性受到采样和筛选的负面影响。These metrics typically have many dimensions from the event properties, which makes them superior for analytics, but the accuracy of these metrics is negatively affected by sampling and filtering.

预先聚合的指标Pre-aggregated metrics

除了基于日志的指标以外,在 2018 年年底,Application Insights 团队交付了存储在专用存储库(已针对时序进行优化)中的指标的公共预览版。In addition to log-based metrics, in late 2018, the Application Insights team shipped a public preview of metrics that are stored in a specialized repository that is optimized for time series. 新指标不再作为包含大量属性的单个事件进行保存。The new metrics are no longer kept as individual events with lots of properties. 它们存储为预先聚合的时序,并且仅包含键维度。Instead, they are stored as pre-aggregated time series, and only with key dimensions. 这使得新指标在查询时间方面非常出色:检索数据的速度要快得多,而且所需的计算能力更低。This makes the new metrics superior at query time: retrieving data happens much faster and requires less compute power. 因此,可以实现针对指标维度发出近实时警报、响应能力更强的仪表板等方案。This consequently enables new scenarios such as near real-time alerting on dimensions of metrics, more responsive dashboards, and more.

重要

基于日志的指标和预先聚合的指标可在 Application Insights 中共存。Both, log-based and pre-aggregated metrics coexist in Application Insights. 为了区分两者,在 Application Insights UX 中,预先聚合的指标现在称为“标准指标(预览版)”,而事件中的传统指标已重命名为“基于日志的指标”。To differentiate the two, in the Application Insights UX the pre-aggregated metrics are now called "Standard metrics (preview)", while the traditional metrics from the events were renamed to "Log-based metrics".

较新的 SDK(适用于 .NET 的 Application Insights 2.7 SDK 或更高版本)会在收集期间预先聚合指标,然后,遥测量缩减技术将会介入。The newer SDKs (Application Insights 2.7 SDK or later for .NET) pre-aggregate metrics during collection before telemetry volume reduction techniques kick in. 这意味着,使用最新 Application Insights SDK 时,新指标的准确性不受采样和筛选的影响。This means that the accuracy of the new metrics isn't affected by sampling and filtering when using the latest Application Insights SDKs.

对于不实施预先聚合的 SDK(即,早期版本的 Application Insights SDK 或用于浏览器检测的 SDK),Application Insights 后端仍会通过聚合 Application Insights 事件收集终结点收到的事件来填充新指标。For the SDKs that don't implement pre-aggregation (that is, older versions of Application Insights SDKs or for browser instrumentation) the Application Insights backend still populates the new metrics by aggregating the events received by the Application Insights event collection endpoint. 这意味着,尽管不能减少通过网络传输的数据量,但仍可以使用预先聚合的指标并改善性能,同时,在收集期间,可以使用不预先聚合指标的 SDK 来支持近实时维度警报。This means that while you don't benefit from the reduced volume of data transmitted over the wire, you can still use the pre-aggregated metrics and experience better performance and support of the near real-time dimensional alerting with SDKs that don't pre-aggregate metrics during collection.

值得一提的是,收集终结点会在引入采样之前预先聚合事件,这意味着,引入采样永远不会影响预先聚合的指标,不管对应用程序使用哪个 SDK 版本。It is worth mentioning that the collection endpoint pre-aggregates events before ingestion sampling, which means that ingestion sampling will never impact the accuracy of pre-aggregated metrics, regardless of the SDK version you use with your application.

对 Application Insights 自定义指标使用预先聚合Using pre-aggregation with Application Insights custom metrics

可对自定义指标使用预先聚合。You can use pre-aggregation with custom metrics. 带来的两个主要好处是,能够配置自定义指标的维度并发出其警报,以及减少从 SDK 发送到 Application Insights 收集终结点的数据量。The two main benefits are the ability to configure and alert on a dimension of a custom metric and reducing the volume of data sent from the SDK to the Application Insights collection endpoint.

可通过多种方法从 Application Insights SDK 发送自定义指标There are several ways of sending custom metrics from the Application Insights SDK. 如果 SDK 版本提供 GetMetric 和 TrackValue 方法,则最好是使用这些方法发送自定义指标,因为在这种情况下,预先聚合在 SDK 中发生,这不仅可以减少 Azure 中存储的数据量,而且还能减少从 SDK 传输到 Application Insights 的数据量。If your version of the SDK offers the GetMetric and TrackValue methods, this is the preferred way of sending custom metrics, since in this case pre-aggregation happens inside of the SDK, not only reducing the volume of data stored in Azure, but also the volume of data transmitted from the SDK to Application Insights. 否则请使用 trackMetric 方法。此方法在数据引入期间会预先聚合指标事件。Otherwise, use the trackMetric method, which will pre-aggregate metric events during data ingestion.

自定义指标维度和预先聚合Custom metrics dimensions and pre-aggregation

使用 trackMetricGetMetric 和 TrackValue API 调用发送的所有指标将自动存储在日志和指标存储中。All metrics that you send using trackMetric or GetMetric and TrackValue API calls are automatically stored in both logs and metrics stores. 但是,自定义指标的基于日志的版本始终保留所有维度,而指标的预先聚合版本在存储时默认不包含任何维度。However, while the log-based version of your custom metric always retains all dimensions, the pre-aggregated version of the metric is stored by default with no dimensions. 可以在用量和预估成本选项卡上选中“针对自定义指标维度启用警报”,来启用自定义指标维度的收集:You can turn on collection of dimensions of custom metrics on the usage and estimated cost tab by checking “Enable alerting on custom metric dimensions”:

用量和预估成本

为何默认禁用了自定义指标维度的收集?Why is collection of custom metrics dimensions turned off by default?

之所以默认禁用自定义指标维度的收集,是因为存储包含维度的自定义指标将来会在 Application Insights 中单独计费,而存储无维度自定义指标会保持免费(但不能超过配额限制)。The collection of custom metrics dimensions is turned off by default because in the future storing custom metrics with dimensions will be billed separately from Application Insights, while storing the non-dimensional custom metrics will remain free (up to a quota). 可以在官方定价页中了解我们即将做出的定价模式变化。You can learn about the upcoming pricing model changes on our official pricing page.

创建图表和浏览基于日志的指标与预先聚合的标准指标Creating charts and exploring log-based and standard pre-aggregated metrics

使用 Azure Monitor 指标资源管理器可以绘制预先聚合的指标和基于日志的指标的图表,以及创作包含图表的仪表板。Use Azure Monitor Metrics Explorer to plot charts from pre-aggregated and log-based metrics, and to author dashboards with charts. 选择所需的 Application Insights 资源后,使用命名空间选取器在标准指标(预览版)和基于日志的指标之间切换,或选择自定义指标命名空间:After selecting the desired Application Insights resource, use the namespace picker to switch between standard (preview) and log-based metrics, or select a custom metric namespace:

指标命名空间

Application Insights 指标的定价模型Pricing models for Application Insights metrics

如果将指标引入 Application Insights(无论是基于日志的还是预聚合的),将产生基于引入数据的大小的成本,如此处所述。Ingesting metrics into Application Insights, whether log-based or pre-aggregated, will generate costs based on the size of the ingested data, as described here. 自定义指标(包括其所有维度)始终存储在 Application Insights 日志存储中;此外,默认情况下,会将自定义指标(不包含维度)的预聚合版本转发到指标存储。Your custom metrics, including all its dimensions, are always stored in the Application Insights log-store; additionally, a pre-aggregated version of your custom metrics (with no dimensions) is forwarded to the metrics store by default.

如果为了在指标存储中存储预聚合指标的所有维度而选择在自定义指标维度上启用警报选项,会产生基于自定义指标定价的额外成本。Selecting the Enable alerting on custom metric dimensions option to store all dimensions of the pre-aggregated metrics in the metric store, can generate additional costs based on Custom Metrics pricing.

后续步骤Next steps