使用 EventFlow 进行事件聚合和收集Event aggregation and collection using EventFlow

Microsoft 诊断 EventFlow 可将事件从某个节点路由到一个或多个监视目标。Microsoft Diagnostics EventFlow can route events from a node to one or more monitoring destinations. 由于它以 NuGet 包的形式包含在服务项目中,EventFlow 代码和配置可以连同服务一起传播,消除了前面所述的需要在 Azure 诊断中为每个节点完成配置的问题。Because it is included as a NuGet package in your service project, EventFlow code and configuration travel with the service, eliminating the per-node configuration issue mentioned earlier about Azure Diagnostics. EventFlow 在服务进程中运行,直接连接到配置的输出。EventFlow runs within your service process, and connects directly to the configured outputs. 由于这种直接连接,EventFlow 适用于 Azure、容器和本地服务部署。Because of the direct connection, EventFlow works for Azure, container, and on-premises service deployments. 在容器等高密度方案中运行 EventFlow 时请保持谨慎,因为每个 EventFlow 管道会建立外部连接。Be careful if you run EventFlow in high-density scenarios, such as in a container, because each EventFlow pipeline makes an external connection. 因此如若托管多个进程,会获得多个出站连接!So, if you host several processes, you get several outbound connections! 这对于 Service Fabric 应用程序而言并不是一个大问题,因为 ServiceType 的所有副本在同一个进程中运行,限制了出站连接数。This isn't as much a concern for Service Fabric applications, because all replicas of a ServiceType run in the same process, and this limits the number of outbound connections. EventFlow 还提供事件筛选,以便只发送与指定的筛选器匹配的事件。EventFlow also offers event filtering, so that only the events that match the specified filter are sent.

设置 EventFlowSet up EventFlow

EventFlow 二进制文件都可用作一组 NuGet 包。EventFlow binaries are available as a set of NuGet packages. 要将 EventFlow 添加到 Service Fabric 服务项目中,右键单击解决方案资源管理器中的项目,并选择“管理 NuGet 包”。To add EventFlow to a Service Fabric service project, right-click the project in the Solution Explorer and choose "Manage NuGet packages." 切换到“浏览”选项卡,搜索“Diagnostics.EventFlow”:Switch to the "Browse" tab and search for "Diagnostics.EventFlow":

Visual Studio NuGet 包管理器 UI 中的 EventFlow NuGet 包

将显示一个不同包的列表,带有“输入”和“输出”标记。You will see a list of various packages show up, labeled with "Inputs" and "Outputs". EventFlow 支持不同日志提供程序和分析器。EventFlow supports various different logging providers and analyzers. 托管 EventFlow 的服务应包括相应的包,具体取决于应用程序日志的源和目标。The service hosting EventFlow should include appropriate packages depending on the source and destination for the application logs. 除核心 ServiceFabric 包外,至少还需配置一个输入和输出。In addition to the core ServiceFabric package, you also need at least one Input and Output configured. 例如,可添加下列包将 EventSource 事件发送到 Application Insights:For example, you can add the following packages to send EventSource events to Application Insights:

  • Microsoft.Diagnostics.EventFlow.Inputs.EventSource(从该服务的 EventSource 类和标准 EventSource 捕获数据,例如 Microsoft-ServiceFabric-Services 和 Microsoft-ServiceFabric-Actors )Microsoft.Diagnostics.EventFlow.Inputs.EventSource to capture data from the service's EventSource class, and from standard EventSources such as Microsoft-ServiceFabric-Services and Microsoft-ServiceFabric-Actors)
  • Microsoft.Diagnostics.EventFlow.Outputs.ApplicationInsights(我们会将日志发送到 Azure Application Insights 资源)Microsoft.Diagnostics.EventFlow.Outputs.ApplicationInsights (we are going to send the logs to an Azure Application Insights resource)
  • Microsoft.Diagnostics.EventFlow.ServiceFabric(允许初始化来自 Service Fabric 服务配置的 EventFlow 管道,并以 Service Fabric 运行状况报表的形式报告发送诊断数据的任何相关问题)Microsoft.Diagnostics.EventFlow.ServiceFabric(enables initialization of the EventFlow pipeline from Service Fabric service configuration and reports any problems with sending diagnostic data as Service Fabric health reports)

备注

Microsoft.Diagnostics.EventFlow.Inputs.EventSource 包要求服务项目面向.NET Framework 4.6 或更高版本。Microsoft.Diagnostics.EventFlow.Inputs.EventSource package requires the service project to target .NET Framework 4.6 or newer. 请确保在项目属性中设置相应的目标框架,再安装此包。Make sure you set the appropriate target framework in project properties before installing this package.

安装所有的包后,下一步是在服务中配置和启用 EventFlow。After all the packages are installed, the next step is to configure and enable EventFlow in the service.

配置和启用日志收集Configure and enable log collection

EventFlow 管道(负责发送日志)根据配置文件中存储的规范创建。The EventFlow pipeline responsible for sending the logs is created from a specification stored in a configuration file. Microsoft.Diagnostics.EventFlow.ServiceFabric 包在 PackageRoot\Config 解决方案文件夹下安装名为 eventFlowConfig.json 的起始 EventFlow 配置文件。The Microsoft.Diagnostics.EventFlow.ServiceFabric package installs a starting EventFlow configuration file under PackageRoot\Config solution folder, named eventFlowConfig.json. 需要对此配置文件进行修改,以从默认服务 EventSource 类和其他任何希望配置的输入中捕获数据,并将数据发送到适当位置。This configuration file needs to be modified to capture data from the default service EventSource class, and any other inputs you want to configure, and send data to the appropriate place.

备注

如果项目文件具有 VisualStudio 2017 格式,则不会自动添加 eventFlowConfig.json 文件。If your project file has VisualStudio 2017 format the eventFlowConfig.json file will not be automatically added. 要修复此问题,请在 Config 文件夹中创建该文件,并将生成操作设置为Copy if newerTo fix this create the file in the Config folder and set the build action to Copy if newer.

以下是基于上文提到的 NuGet 包的 eventFlowConfig.json 示例:Here is a sample eventFlowConfig.json based on the NuGet packages mentioned above:

{
  "inputs": [
    {
      "type": "EventSource",
      "sources": [
        { "providerName": "Microsoft-ServiceFabric-Services" },
        { "providerName": "Microsoft-ServiceFabric-Actors" },
        // (replace the following value with your service's ServiceEventSource name)
        { "providerName": "your-service-EventSource-name" }
      ]
    }
  ],
  "filters": [
    {
      "type": "drop",
      "include": "Level == Verbose"
    }
  ],
  "outputs": [
    {
      "type": "ApplicationInsights",
      // (replace the following value with your AI resource's instrumentation key)
      "instrumentationKey": "00000000-0000-0000-0000-000000000000"
    }
  ],
  "schemaVersion": "2016-08-11"
}

服务的 ServiceEventSource 名称是应用于 ServiceEventSource 类的 EventSourceAttribute 的 Name 属性值。The name of service's ServiceEventSource is the value of the Name property of the EventSourceAttribute applied to the ServiceEventSource class. 这在 ServiceEventSource.cs 文件中指定,该文件是服务代码的一部分。It is all specified in the ServiceEventSource.cs file, which is part of the service code. 例如,在以下代码片段中,ServiceEventSource 的名称是 MyCompany-Application1-Stateless1 :For example, in the following code snippet the name of the ServiceEventSource is MyCompany-Application1-Stateless1:

[EventSource(Name = "MyCompany-Application1-Stateless1")]
internal sealed class ServiceEventSource : EventSource
{
    // (rest of ServiceEventSource implementation)
}

请注意,eventFlowConfig.json 文件属于服务配置包。Note that eventFlowConfig.json file is part of service configuration package. 对此文件的更改可以包含在服务的完全升级或仅配置的升级中,并且会接受 Service Fabric 升级运行状况检查和自动回退(如果升级失败)。Changes to this file can be included in full- or configuration-only upgrades of the service, subject to Service Fabric upgrade health checks and automatic rollback if there is upgrade failure. 有关详细信息,请参阅 Service Fabric 应用程序升级For more information, see Service Fabric application upgrade.

通过配置的筛选器 部分可进一步自定义将通过 EventFlow 管道到达输出的信息,使你能够删除或包含某些信息,或更改事件数据的结构。The filters section of the config allows you to further customize the information that is going to go through the EventFlow pipeline to the outputs, allowing you to drop or include certain information, or change the structure of the event data. 有关筛选的详细信息,请参阅 EventFlow 筛选器For more information on filtering, see EventFlow filters.

最后一步是在服务的启动代码中(位于 Program.cs 文件)实例化 EventFlow 管道:The final step is to instantiate EventFlow pipeline in your service's startup code, located in Program.cs file:

using System;
using System.Diagnostics;
using System.Threading;
using Microsoft.ServiceFabric;
using Microsoft.ServiceFabric.Services.Runtime;

// **** EventFlow namespace
using Microsoft.Diagnostics.EventFlow.ServiceFabric;

namespace Stateless1
{
    internal static class Program
    {
        /// <summary>
        /// This is the entry point of the service host process.
        /// </summary>
        private static void Main()
        {
            try
            {
                // **** Instantiate log collection via EventFlow
                using (var diagnosticsPipeline = ServiceFabricDiagnosticPipelineFactory.CreatePipeline("MyApplication-MyService-DiagnosticsPipeline"))
                {

                    ServiceRuntime.RegisterServiceAsync("Stateless1Type",
                    context => new Stateless1(context)).GetAwaiter().GetResult();

                    ServiceEventSource.Current.ServiceTypeRegistered(Process.GetCurrentProcess().Id, typeof(Stateless1).Name);

                    Thread.Sleep(Timeout.Infinite);
                }
            }
            catch (Exception e)
            {
                ServiceEventSource.Current.ServiceHostInitializationFailed(e.ToString());
                throw;
            }
        }
    }
}

作为 ServiceFabricDiagnosticsPipelineFactoryCreatePipeline 方法的参数传递的名称是表示 EventFlow 日志收集管道的运行状况实体的名称。The name passed as the parameter of the CreatePipeline method of the ServiceFabricDiagnosticsPipelineFactory is the name of the health entity representing the EventFlow log collection pipeline. 如果 EventFlow 发生错误,并通过 Service Fabric 运行状况子系统进行报告,则使用此名称。This name is used if EventFlow encounters and error and reports it through the Service Fabric health subsystem.

在 eventFlowConfig 中使用 Service Fabric 设置和应用程序参数Use Service Fabric settings and application parameters in eventFlowConfig

EventFlow 支持使用 Service Fabric 设置和应用程序参数来配置 EventFlow 设置。EventFlow supports using Service Fabric settings and application parameters to configure EventFlow settings. 可使用此特殊语法来引用 Service Fabric 设置参数的值:You can refer to Service Fabric settings parameters using this special syntax for values:

servicefabric:/<section-name>/<setting-name>

<section-name> 是 Service Fabric 配置节的名称;而 <setting-name> 是配置设置,提供用于配置 EventFlow 设置的值。<section-name> is the name of the Service Fabric configuration section, and <setting-name> is the configuration setting providing the value that will be used to configure an EventFlow setting. 若要深入了解如何执行此操作,请转到 Service Fabric 和应用程序参数支持To read more about how to do this, go to Support for Service Fabric settings and application parameters.

验证Verification

在 Visual Studio 中启动服务,并观察调试输出窗口。Start your service and observe the Debug output window in Visual Studio. 该服务启动后,应开始看到服务正向已配置输出发送记录的证据。After the service is started, you should start seeing evidence that your service is sending records to the output that you have configured. 导航到事件分析和可视化平台,并确认日志已开始显示(可能需要几分钟时间)。Navigate to your event analysis and visualization platform and confirm that logs have started to show up (could take a few minutes).

后续步骤Next steps