Application Insights SDK 中的筛选和预处理遥测 | Microsoft AzureFiltering and preprocessing telemetry in the Application Insights SDK

可以为 Application Insights SDK 编写和配置插件,在将遥测发送到 Application Insights 服务之前自定义遥测的扩充和处理方式。You can write and configure plug-ins for the Application Insights SDK to customize how telemetry can be enriched and processed before it's sent to the Application Insights service.

  • 采样可在不影响统计信息的情况下减少遥测量。Sampling reduces the volume of telemetry without affecting your statistics. 它还保留相关数据点,以便可以在诊断问题时导航这些点。It keeps together related data points so that you can navigate between them when diagnosing a problem. 在门户中,总计相乘以补偿采样。In the portal, the total counts are multiplied to compensate for the sampling.
  • 使用遥测处理器进行筛选,就可以先在 SDK 中筛选出遥测,然后再将其发送到服务器。Filtering with Telemetry Processors lets you filter out telemetry in the SDK before it is sent to the server. 例如,可以通过排除机器人请求减少遥测量。For example, you could reduce the volume of telemetry by excluding requests from robots. 与采样相比,筛选是更基本的减少流量的方法。Filtering is a more basic approach to reducing traffic than sampling. 它允许更好地控制传输的内容,但你必须知道它会影响统计信息(例如,如果筛选出所有成功请求)。It allows you more control over what is transmitted, but you have to be aware that it affects your statistics - for example, if you filter out all successful requests.
  • 对于从应用发送的任何遥测(包括来自标准模块的遥测),遥测初始值设定项可以添加或修改属性Telemetry Initializers add or modify properties to any telemetry sent from your app, including telemetry from the standard modules. 例如,可以添加计算得出的值;或在门户中筛选数据所依据的版本号。For example, you could add calculated values; or version numbers by which to filter the data in the portal.
  • SDK API 用于发送自定义事件和指标。The SDK API is used to send custom events and metrics.

开始之前:Before you start:

筛选Filtering

使用此技术可以直接控制要在遥测流中包含或排除的内容。This technique gives you direct control over what is included or excluded from the telemetry stream. 可以通过筛选删除要发送到 Application Insights 的遥测项。Filtering can be used to drop telemetry items from being sent to Application Insights. 可以将其与采样结合使用,也可以单独使用。You can use it in conjunction with Sampling, or separately.

若要筛选遥测,请编写遥测处理器并通过 TelemetryConfiguration 注册它。To filter telemetry, you write a telemetry processor and register it with the TelemetryConfiguration. 所有遥测都通过处理器,可以选择从流中删除它或将它提供给链中的下一处理器。All telemetry goes through your processor, and you can choose to drop it from the stream or give it to the next processor in the chain. 这包括来自标准模块的遥测(例如 HTTP 请求收集器和依赖项收集器)以及自行跟踪的遥测。This includes telemetry from the standard modules such as the HTTP request collector and the dependency collector, and telemetry you have tracked yourself. 例如,可以筛选出有关机器人请求或成功依赖项调用的遥测。You can, for example, filter out telemetry about requests from robots, or successful dependency calls.

警告

使用处理器筛选从 SDK 发送的遥测会使门户中看到的统计信息出现偏差,并使它难以跟进相关项目。Filtering the telemetry sent from the SDK using processors can skew the statistics that you see in the portal, and make it difficult to follow related items.

此时,考虑使用采样Instead, consider using sampling.

创建遥测处理器 (C#)Create a telemetry processor (C#)

  1. 若要创建筛选器,请实现 ITelemetryProcessorTo create a filter, implement ITelemetryProcessor.

    请注意,遥测处理器构建一个处理链。Notice that Telemetry Processors construct a chain of processing. 当你实例化遥测处理器时,系统会为你提供链中下一个处理器的引用。When you instantiate a telemetry processor, you are given a reference to the next processor in the chain. 将遥测数据点传递到处理方法时,该方法会生效,然后调用(或不调用)链中的下一个遥测处理器。When a telemetry data point is passed to the Process method, it does its work and then calls (or not calls) the next Telemetry Processor in the chain.

    using Microsoft.ApplicationInsights.Channel;
    using Microsoft.ApplicationInsights.Extensibility;
    
    public class SuccessfulDependencyFilter : ITelemetryProcessor
    {
        private ITelemetryProcessor Next { get; set; }
    
        // next will point to the next TelemetryProcessor in the chain.
        public SuccessfulDependencyFilter(ITelemetryProcessor next)
        {
            this.Next = next;
        }
    
        public void Process(ITelemetry item)
        {
            // To filter out an item, return without calling the next processor.
            if (!OKtoSend(item)) { return; }
    
            this.Next.Process(item);
        }
    
        // Example: replace with your own criteria.
        private bool OKtoSend (ITelemetry item)
        {
            var dependency = item as DependencyTelemetry;
            if (dependency == null) return true;
    
            return dependency.Success != true;
        }
    }
    
  2. 添加处理器。Add your processor.

ASP.NET 应用 在 ApplicationInsights.config 中插入此代码片段:ASP.NET apps Insert this snippet in ApplicationInsights.config:

<TelemetryProcessors>
  <Add Type="WebApplication9.SuccessfulDependencyFilter, WebApplication9">
     <!-- Set public property -->
     <MyParamFromConfigFile>2-beta</MyParamFromConfigFile>
  </Add>
</TelemetryProcessors>

可通过在类中提供公共命名属性,传递 .config 文件中的字符串值。You can pass string values from the .config file by providing public named properties in your class.

警告

注意将 .config 文件中的类型名称和任何属性名称匹配到代码中的类和属性名称。Take care to match the type name and any property names in the .config file to the class and property names in the code. 如果 .config 文件引用不存在的类型或属性,该 SDK 在发送任何遥测时可能静默失败。If the .config file references a non-existent type or property, the SDK may silently fail to send any telemetry.

或者, 可以在代码中初始化筛选器。Alternatively, you can initialize the filter in code. 在合适的初始化类(例如,Global.asax.cs 中的 AppStart)中,将处理器插入链:In a suitable initialization class - for example AppStart in Global.asax.cs - insert your processor into the chain:

var builder = TelemetryConfiguration.Active.DefaultTelemetrySink.TelemetryProcessorChainBuilder;
builder.Use((next) => new SuccessfulDependencyFilter(next));

// If you have more processors:
builder.Use((next) => new AnotherProcessor(next));

builder.Build();

在此点后创建的 TelemetryClients 将使用处理器。TelemetryClients created after this point will use your processors.

ASP.NET Core/辅助角色服务应用ASP.NET Core/ Worker Service apps

备注

使用 ApplicationInsights.configTelemetryConfiguration.Active 添加处理器对于 ASP.NET Core 应用程序无效,或者你在使用 Microsoft.ApplicationInsights.WorkerService SDK。Adding processor using ApplicationInsights.config or using TelemetryConfiguration.Active is not valid for ASP.NET Core applications or if you are using Microsoft.ApplicationInsights.WorkerService SDK.

对于使用 ASP.NET CoreWorkerService 编写的应用,若要添加新的 TelemetryProcessor,请在 IServiceCollection 上使用 AddApplicationInsightsTelemetryProcessor 扩展方法,如下所示。For apps written using ASP.NET Core or WorkerService, adding a new TelemetryProcessor is done by using AddApplicationInsightsTelemetryProcessor extension method on IServiceCollection, as shown below. 此方法在 Startup.cs 类的 ConfigureServices 方法中调用。This method is called in ConfigureServices method of your Startup.cs class.

    public void ConfigureServices(IServiceCollection services)
    {
        // ...
        services.AddApplicationInsightsTelemetry();
        services.AddApplicationInsightsTelemetryProcessor<SuccessfulDependencyFilter>();

        // If you have more processors:
        services.AddApplicationInsightsTelemetryProcessor<AnotherProcessor>();
    }

示例筛选器Example filters

综合请求Synthetic requests

筛选出机器人和 Web 测试。Filter out bots and web tests. 尽管指标资源管理器提供筛选出综合源的选项,但此选项可通过在 SDK 自身中筛选它们减少流量和引入大小。Although Metrics Explorer gives you the option to filter out synthetic sources, this option reduces traffic and ingestion size by filtering them at the SDK itself.

public void Process(ITelemetry item)
{
  if (!string.IsNullOrEmpty(item.Context.Operation.SyntheticSource)) {return;}

  // Send everything else:
  this.Next.Process(item);
}

身份验证失败Failed authentication

筛选出带有“401”响应的请求。Filter out requests with a "401" response.

public void Process(ITelemetry item)
{
    var request = item as RequestTelemetry;

    if (request != null &&
    request.ResponseCode.Equals("401", StringComparison.OrdinalIgnoreCase))
    {
        // To filter out an item, return without calling the next processor.
        return;
    }

    // Send everything else
    this.Next.Process(item);
}

筛选出快速远程依赖项调用Filter out fast remote dependency calls

如果只想诊断速度较慢的调用,则筛选出快速调用。If you only want to diagnose calls that are slow, filter out the fast ones.

备注

这会使门户上看到的统计信息出现偏差。This will skew the statistics you see on the portal.

public void Process(ITelemetry item)
{
    var request = item as DependencyTelemetry;

    if (request != null && request.Duration.TotalMilliseconds < 100)
    {
        return;
    }
    this.Next.Process(item);
}

诊断依赖项问题Diagnose dependency issues

本博客介绍了通过自动将常规 Ping 发送到依赖项诊断依赖项问题的项目。This blog describes a project to diagnose dependency issues by automatically sending regular pings to dependencies.

JavaScript Web 应用程序JavaScript Web applications

使用 ITelemetryInitializer 进行筛选Filtering using ITelemetryInitializer

  1. 创建遥测初始化程序回调函数。Create a telemetry initializer callback function. 回调函数将 ITelemetryItem 作为参数,该项是正在处理的事件。The callback function takes ITelemetryItem as a parameter, which is the event that is being processed. 从此回调返回 false 将导致筛选掉遥测项。Returning false from this callback results in the telemetry item to be filtered out.

    var filteringFunction = (envelope) => {
      if (envelope.data.someField === 'tobefilteredout') {
          return false;
      }
    
      return true;
    };
    
  2. 添加遥测初始化程序回调:Add your telemetry initializer callback:

    appInsights.addTelemetryInitializer(filteringFunction);
    

添加/修改属性:ITelemetryInitializerAdd/modify properties: ITelemetryInitializer

通过遥测初始值设定项使用其他信息来扩充遥测,以及/或者重写通过标准遥测模块设置的遥测属性。Use telemetry initializers to enrich telemetry with additional information and/or to override telemetry properties set by the standard telemetry modules.

例如,Web 包的 Application Insights 将收集有关 HTTP 请求的遥测。For example, the Application Insights for Web package collect telemetry about HTTP requests. 默认情况下,它标记为所有请求失败,并且响应代码 >= 400。By default, it flags as failed any request with a response code >= 400. 但是,如果希望将 400 视为成功,可以提供一个设置成功属性的遥测初始值设定项。But if you want to treat 400 as a success, you can provide a telemetry initializer that sets the Success property.

如果提供了遥测初始值设定项,只要调用任何 Track*() 方法,就会调用它。If you provide a telemetry initializer, it is called whenever any of the Track*() methods are called. 这包括由标准遥测模块调用的 Track() 方法。This includes Track() methods called by the standard telemetry modules. 按照约定,这些模块不会设置已由初始值设定项设置的任何属性。By convention, these modules do not set any property that has already been set by an initializer. 在调用遥测处理器之前调用遥测初始值设定项。Telemetry initializers are called before calling telemetry processors. 因此,由初始值设定项完成的任何扩充对于处理器来说都是可见的。So any enrichments done by initializers are visible to processors.

定义初始值设定项Define your initializer

C#C#

using System;
using Microsoft.ApplicationInsights.Channel;
using Microsoft.ApplicationInsights.DataContracts;
using Microsoft.ApplicationInsights.Extensibility;

namespace MvcWebRole.Telemetry
{
  /*
   * Custom TelemetryInitializer that overrides the default SDK
   * behavior of treating response codes >= 400 as failed requests
   *
   */
  public class MyTelemetryInitializer : ITelemetryInitializer
  {
    public void Initialize(ITelemetry telemetry)
    {
        var requestTelemetry = telemetry as RequestTelemetry;
        // Is this a TrackRequest() ?
        if (requestTelemetry == null) return;
        int code;
        bool parsed = Int32.TryParse(requestTelemetry.ResponseCode, out code);
        if (!parsed) return;
        if (code >= 400 && code < 500)
        {
            // If we set the Success property, the SDK won't change it:
            requestTelemetry.Success = true;

            // Allow us to filter these requests in the portal:
            requestTelemetry.Properties["Overridden400s"] = "true";
        }
        // else leave the SDK to set the Success property
    }
  }
}

ASP.NET 应用:加载初始值设定项ASP.NET apps: Load your initializer

在 ApplicationInsights.config 中:In ApplicationInsights.config:

<ApplicationInsights>
  <TelemetryInitializers>
    <!-- Fully qualified type name, assembly name: -->
    <Add Type="MvcWebRole.Telemetry.MyTelemetryInitializer, MvcWebRole"/>
    ...
  </TelemetryInitializers>
</ApplicationInsights>

或者, 可以在代码中实例化初始值设定项,例如在 Global.aspx.cs 中:Alternatively, you can instantiate the initializer in code, for example in Global.aspx.cs:

protected void Application_Start()
{
    // ...
    TelemetryConfiguration.Active.TelemetryInitializers.Add(new MyTelemetryInitializer());
}

查看此示例的详细信息。See more of this sample.

ASP.NET Core/辅助角色服务应用:加载初始值设定项ASP.NET Core/ Worker Service apps: Load your initializer

备注

使用 ApplicationInsights.configTelemetryConfiguration.Active 添加初始值设定项对于 ASP.NET Core 应用程序无效,或者你在使用 Microsoft.ApplicationInsights.WorkerService SDK。Adding initializer using ApplicationInsights.config or using TelemetryConfiguration.Active is not valid for ASP.NET Core applications or if you are using Microsoft.ApplicationInsights.WorkerService SDK.

对于使用 ASP.NET CoreWorkerService 编写的应用,添加新的 TelemetryInitializer 是通过将其添加到依赖项注入容器来完成的,如下所示。For apps written using ASP.NET Core or WorkerService, adding a new TelemetryInitializer is done by adding it to the Dependency Injection container, as shown below. 这可以在 Startup.ConfigureServices 方法中完成。This is done in Startup.ConfigureServices method.

 using Microsoft.ApplicationInsights.Extensibility;
 using CustomInitializer.Telemetry;
 public void ConfigureServices(IServiceCollection services)
{
    services.AddSingleton<ITelemetryInitializer, MyTelemetryInitializer>();
}

JavaScript 遥测初始值设定项JavaScript telemetry initializers

JavaScriptJavaScript

在从门户获取的初始化代码后立即插入遥测初始值设定项:Insert a telemetry initializer immediately after the initialization code that you got from the portal:

<script type="text/javascript">
    // ... initialization code
    ...({
        instrumentationKey: "your instrumentation key"
    });
    window.appInsights = appInsights;


    // Adding telemetry initializer.
    // This is called whenever a new telemetry item
    // is created.

    appInsights.queue.push(function () {
        appInsights.context.addTelemetryInitializer(function (envelope) {
            var telemetryItem = envelope.data.baseData;

            // To check the telemetry items type - for example PageView:
            if (envelope.name == Microsoft.ApplicationInsights.Telemetry.PageView.envelopeType) {
                // this statement removes url from all page view documents
                telemetryItem.url = "URL CENSORED";
            }

            // To set custom properties:
            telemetryItem.properties = telemetryItem.properties || {};
            telemetryItem.properties["globalProperty"] = "boo";

            // To set custom metrics:
            telemetryItem.measurements = telemetryItem.measurements || {};
            telemetryItem.measurements["globalMetric"] = 100;
        });
    });

    // End of inserted code.

    appInsights.trackPageView();
</script>

有关 telemetryItem 上可用的非自定义属性摘要,请参阅Application Insights 导出数据模型For a summary of the non-custom properties available on the telemetryItem, see Application Insights Export Data Model.

可添加任意数量的初始值设定项,并按添加顺序调用它们。You can add as many initializers as you like, and they are called in the order they are added.

OpenCensus Python 遥测处理器OpenCensus Python telemetry processors

OpenCensus Python 中的遥测处理器是简单的回调函数,在导出遥测之前调用这些函数来处理遥测。Telemetry processors in OpenCensus Python are simply callback functions called to process telemetry before they are exported. 回调函数必须接受信封数据类型作为其参数。The callback function must accept an envelope data type as its parameter. 若要筛选掉导出的遥测,请确保回调函数返回 FalseTo filter out telemetry from being exported,make sure the callback function returns False. 可在此处查看信封中 Azure Monitor 数据类型的架构。You can see the schema for Azure Monitor data types in the envelopes here.

备注

可以通过更改 tags 字段中的 ai.cloud.role 属性来修改 cloud_RoleNameYou can modify the cloud_RoleName by changing the ai.cloud.role attribute in the tags field.

def callback_function(envelope):
    envelope.tags['ai.cloud.role'] = 'new_role_name'
# Example for log exporter
import logging

from opencensus.ext.azure.log_exporter import AzureLogHandler

logger = logging.getLogger(__name__)

# Callback function to append '_hello' to each log message telemetry
def callback_function(envelope):
    envelope.data.baseData.message += '_hello'
    return True

handler = AzureLogHandler(connection_string='InstrumentationKey=<your-instrumentation_key-here>')
handler.add_telemetry_processor(callback_function)
logger.addHandler(handler)
logger.warning('Hello, World!')
# Example for trace exporter
import requests

from opencensus.ext.azure.trace_exporter import AzureExporter
from opencensus.trace import config_integration
from opencensus.trace.samplers import ProbabilitySampler
from opencensus.trace.tracer import Tracer

config_integration.trace_integrations(['requests'])

# Callback function to add os_type: linux to span properties
def callback_function(envelope):
    envelope.data.baseData.properties['os_type'] = 'linux'
    return True

exporter = AzureExporter(
    connection_string='InstrumentationKey=<your-instrumentation-key-here>'
)
exporter.add_telemetry_processor(callback_function)
tracer = Tracer(exporter=exporter, sampler=ProbabilitySampler(1.0))
with tracer.span(name='parent'):
response = requests.get(url='https://www.wikipedia.org/wiki/Rabbit')
# Example for metrics exporter
import time

from opencensus.ext.azure import metrics_exporter
from opencensus.stats import aggregation as aggregation_module
from opencensus.stats import measure as measure_module
from opencensus.stats import stats as stats_module
from opencensus.stats import view as view_module
from opencensus.tags import tag_map as tag_map_module

stats = stats_module.stats
view_manager = stats.view_manager
stats_recorder = stats.stats_recorder

CARROTS_MEASURE = measure_module.MeasureInt("carrots",
                                            "number of carrots",
                                            "carrots")
CARROTS_VIEW = view_module.View("carrots_view",
                                "number of carrots",
                                [],
                                CARROTS_MEASURE,
                                aggregation_module.CountAggregation())

# Callback function to only export the metric if value is greater than 0
def callback_function(envelope):
    return envelope.data.baseData.metrics[0].value > 0

def main():
    # Enable metrics
    # Set the interval in seconds in which you want to send metrics
    exporter = metrics_exporter.new_metrics_exporter(connection_string='InstrumentationKey=<your-instrumentation-key-here>')
    exporter.add_telemetry_processor(callback_function)
    view_manager.register_exporter(exporter)

    view_manager.register_view(CARROTS_VIEW)
    mmap = stats_recorder.new_measurement_map()
    tmap = tag_map_module.TagMap()

    mmap.measure_int_put(CARROTS_MEASURE, 1000)
    mmap.record(tmap)
    # Default export interval is every 15.0s
    # Your application should run for at least this amount
    # of time so the exporter will meet this interval
    # Sleep can fulfill this
    time.sleep(60)

    print("Done recording metrics")

if __name__ == "__main__":
    main()

可以根据需要添加任意数量的处理器,然后按添加顺序对其进行调用。You can add as many processors as you like, and they are called in the order they are added. 如果一个处理器应引发异常,则它不会影响后面的处理器。If one processor should throw an exception, it does not impact the following processors.

示例 TelemetryInitializerExample TelemetryInitializers

添加自定义属性Add custom property

以下初始值设定项示例将自定义属性添加到每个被跟踪的遥测。The following sample initializer adds a custom property to every tracked telemetry.

public void Initialize(ITelemetry item)
{
  var itemProperties = item as ISupportProperties;
  if(itemProperties != null && !itemProperties.Properties.ContainsKey("customProp"))
    {
        itemProperties.Properties["customProp"] = "customValue";
    }
}

添加云角色名称Add cloud role name

以下初始值设定项示例将云角色名称设置为每个被跟踪的遥测。The following sample initializer sets cloud role name to every tracked telemetry.

public void Initialize(ITelemetry telemetry)
{
    if (string.IsNullOrEmpty(telemetry.Context.Cloud.RoleName))
    {
        telemetry.Context.Cloud.RoleName = "MyCloudRoleName";
    }
}

从 HttpContext 添加信息Add information from HttpContext

下面的示例初始值设定项从 HttpContext 读取数据并将其附加到 RequestTelemetry 实例。The following sample initializer reads data from the HttpContext and appends it to a RequestTelemetry instance. IHttpContextAccessor 是通过构造函数依赖项注入自动提供的。The IHttpContextAccessor is automatically provided through constructor dependency injection.

public class HttpContextRequestTelemetryInitializer : ITelemetryInitializer
{
    private readonly IHttpContextAccessor httpContextAccessor;

    public HttpContextRequestTelemetryInitializer(IHttpContextAccessor httpContextAccessor)
    {
        this.httpContextAccessor =
            httpContextAccessor ??
            throw new ArgumentNullException(nameof(httpContextAccessor));
    }

    public void Initialize(ITelemetry telemetry)
    {
        var requestTelemetry = telemetry as RequestTelemetry;
        if (requestTelemetry == null) return;

        var claims = this.httpContextAccessor.HttpContext.User.Claims;
        Claim oidClaim = claims.FirstOrDefault(claim => claim.Type == "oid");
        requestTelemetry.Properties.Add("UserOid", oidClaim?.Value);
    }
}

ITelemetryProcessor 和 ITelemetryInitializerITelemetryProcessor and ITelemetryInitializer

遥测处理器和遥测初始值设定项之间的区别是什么?What's the difference between telemetry processors and telemetry initializers?

  • 它们的用途有一些重叠之处:二者均可用于添加或修改遥测属性,不过建议将初始值设定项用于该目的。There are some overlaps in what you can do with them: both can be used to add or modify properties of telemetry, though it is recommended to use initializers for that purpose.
  • TelemetryInitializers 始终在 TelemetryProcessors 之前运行。TelemetryInitializers always run before TelemetryProcessors.
  • TelemetryInitializer 可以多次调用。TelemetryInitializers may be called more than once. 按照约定,它们不会设置已设置的任何属性。By convention, they do not set any property that has already been set.
  • TelemetryProcessors 允许完全替换或丢弃遥测项。TelemetryProcessors allow you to completely replace or discard a telemetry item.
  • 可保证为所有遥测项调用所有注册的 TelemetryInitializer。All registered TelemetryInitializers are guaranteed to be called for every telemetry item. 对于遥测处理器,SDK 保证调用刚好第一个遥测处理器。For Telemetry processors, SDK guarantees calling the very first telemetry processor. 是否调用余下的处理器取决于前面的遥测处理器。Whether the rest of the processors are called or not, is decided by the preceding telemetry processors.
  • 通过 TelemetryInitializer 使用其他属性来扩充遥测,或者重写现有的。Use TelemetryInitializers to enrich telemetry with additional properties, or override existing one. 使用 TelemetryProcessor 筛选出遥测。Use TelemetryProcessor to filter out telemetry.

ApplicationInsights.config 故障排除Troubleshooting ApplicationInsights.config

  • 确认完全限定的类型名称和程序集名称是正确的。Confirm that the fully qualified type name and assembly name are correct.
  • 确认 applicationinsights.config 文件在输出目录中并且包含所有最新更改。Confirm that the applicationinsights.config file is in your output directory and contains any recent changes.

参考文档Reference docs

SDK 代码SDK Code

后续步骤Next steps