收集遥测数据以用于分析搜索流量Collect telemetry data for search traffic analytics

搜索流量分析是一种用于收集遥测数据的模式,它收集有关用户与 Azure 认知搜索应用程序之间的交互(例如用户发起的单击事件和键盘输入)的遥测数据。Search traffic analytics is a pattern for collecting telemetry about user interactions with your Azure Cognitive Search application, such as user-initiated click events and keyboard inputs. 使用此信息,你可以确定搜索解决方案的有效性,包括热门搜索词、点击率以及哪些查询输入产生了零个结果。Using this information, you can determine the effectiveness of your search solution, including popular search terms, clickthrough rate, and which query inputs yield zero results.

此模式依赖于 Application InsightsAzure Monitor 的一项功能)来收集用户数据。This pattern takes a dependency on Application Insights (a feature of Azure Monitor) to collect user data. 你还需要向客户端代码中添加检测机制,如本文中所述。It requires that you add instrumentation to your client code, as described in this article. 最后,你将需要一个报告机制来分析数据。Finally, you will need a reporting mechanism to analyze the data. 建议使用 Power BI,但你可以使用应用程序仪表板或可连接到 Application Insights 的任何工具。We recommend Power BI but you can use the Application Dashboard or any tool that connects to Application Insights.

备注

本文中所述的模式适用于你添加到客户端的代码生成的高级方案和点击流数据。The pattern described in this article is for advanced scenarios and clickstream data generated by code you add to your client. 相比之下,服务日志易于设置,提供各种指标,且无需编写任何代码即可在门户中操作。In contrast, service logs are easy to set up, provide a range of metrics, and can be done in the portal with no code required. 建议对所有方案启用日志记录。Enabling logging is recommended for all scenarios. 有关详细信息,请参阅收集和分析日志数据For more information, see Collect and analyze log data.

标识相关的搜索数据Identify relevant search data

若要为搜索流量分析提供有用的指标,必须记录搜索应用程序用户发出的一些信号。To have useful metrics for search traffic analytics, it's necessary to log some signals from the users of your search application. 这些信号表示用户感兴趣的内容以及他们认为相关的内容。These signals signify content that users are interested in and that they consider relevant. 对于搜索流量分析,这些信号包括:For search traffic analytics, these include:

  • 用户生成的搜索事件:只有用户发起的搜索查询才是需要关注的。User-generated search events: Only search queries initiated by a user are interesting. 用于填充 Facet、附加内容或任何内部信息的搜索查询都不重要,它们会扭曲结果并造成结果有偏差。Search requests used to populate facets, additional content or any internal information, are not important and they skew and bias your results.

  • 用户生成的单击事件:在搜索结果页上,单击事件通常意味着某个文档是特定搜索查询的相关结果。User-generated click events: On a search results page, a click event generally means that a document is a relevant result for a specific search query.

通过将搜索和单击事件链接到相关 ID,可以更深入地了解应用程序搜索功能的性能表现。By linking search and click events with a correlation ID, you'll gain a deeper understanding of how well your application's search functionality is performing.

添加搜索流量分析Add search traffic analytics

在 Azure 认知搜索服务的门户页中,“搜索流量分析”页面包含一个用于遵循此遥测模式的速查表。In the portal page for your Azure Cognitive Search service, the Search Traffic Analytics page contains a cheat sheet for following this telemetry pattern. 在此页中,可以选择或创建一个 Application Insights 资源,获取检测密钥,复制可以根据你的解决方案改编的代码片段,并下载基于模式中反映的架构生成的 Power BI 报表。From this page, you can select or create an Application Insights resource, get the instrumentation key, copy snippets that you can adapt for your solution, and download a Power BI report that's built over the schema reflected in the pattern.

门户中的“搜索流量分析”页Search Traffic Analytics page in the portal

1 - 设置 Application Insights1 - Set up Application Insights

选择现有的 Application Insights 资源,如果没有 Application Insights 资源,则创建一个资源Select an existing Application Insights resource or create one if you don't have one already. 如果使用“搜索流量分析”页,则可以复制应用程序在连接到 Application Insights 时所需的检测密钥。If you use the Search Traffic Analytics page, you can copy the instrumentation key your application needs to connect to Application Insights.

有了 Application Insights 资源后,可以按照适用于受支持语言和平台的说明来注册应用。Once you have an Application Insights resource, you can follow instructions for supported languages and platforms to register your app. 注册只是将 Application Insights 中的检测密钥添加到代码,以设置关联。Registration is simply adding the instrumentation key from Application Insights to your code, which sets up the association. 选择现有的资源时,可以在门户或“搜索流量分析”页中找到该密钥。You can find the key in the portal, or from the Search Traffic Analytics page when you select an existing resource.

以下步骤反映了适用于某些 Visual Studio 项目类型的快捷方式。A shortcut that works for some Visual Studio project types is reflected in the following steps. 只需单击几下鼠标,即可创建资源并注册应用。It creates a resource and registers your app in just a few clicks.

  1. 对于 Visual Studio 和 ASP.NET 开发,请打开你的解决方案,并选择“项目” > “添加 Application Insights 遥测” 。For Visual Studio and ASP.NET development, open your solution and select Project > Add Application Insights Telemetry.

  2. 单击“开始”。Click Get Started.

  3. 通过提供 Microsoft 帐户、Azure 订阅和 Application Insights 资源(某个新资源是默认资源)来注册你的应用。Register your app by providing a Microsoft account, Azure subscription, and an Application Insights resource (a new resource is the default). 单击“注册”。Click Register.

此时,你已经为应用程序设置了应用程序监视,这意味着,将使用默认指标跟踪所有页面加载。At this point, your application is set up for application monitoring, which means all page loads are tracked with default metrics. 有关上述步骤的详细信息,请参阅启用 Application Insights 服务器端遥测For more information about the previous steps, see Enable Application Insights server-side telemetry.

2 - 添加检测2 - Add instrumentation

在此步骤中,将使用以上步骤中创建的 Application Insights 资源检测自己的搜索应用程序。This step is where you instrument your own search application, using the Application Insights resource your created in the step above. 此过程包括四个步骤,第一个步骤是创建遥测客户端。There are four steps to this process, starting with creating a telemetry client.

步骤 1:创建遥测客户端Step 1: Create a telemetry client

创建一个用于将事件发送到 Application Insights 的对象。Create an object that sends events to Application Insights. 可以将检测机制添加到服务器端应用程序代码或在浏览器中运行的客户端代码中,此处的代码以 C# 和 JavaScript 变体表示(对于其他语言,请参阅支持的平台和框架的完整列表)。You can add instrumentation to your server-side application code or client-side code running in a browser, expressed here as C# and JavaScript variants (for other languages, see the complete list of supported platforms and frameworks. 选择可提供所需信息深度的方法。Choose the approach that gives you the desired depth of information.

服务器端遥测将捕获应用程序层(例如,在云中作为 Web 服务运行的应用程序,或者在企业网络中作为本地应用运行的应用程序)的指标。Server-side telemetry captures metrics at the application layer, for example in applications running as a web service in the cloud, or as an on-premises app on a corporate network. 服务器端遥测捕获搜索和单击事件、结果中文档的位置以及查询信息,但数据收集范围将限定为该层上可用的任何信息。Server-side telemetry captures search and click events, the position of a document in results, and query information, but your data collection will be scoped to whatever information is available at that layer.

在客户端上,可以使用附加的代码来操作查询输入、添加导航或包含上下文(例如,从主页或产品页发起的查询)。On the client, you might have additional code that manipulates query inputs, adds navigation, or includes context (for example, queries initiated from a home page versus a product page). 如果此描述符合你的解决方案,则你可以选择启用客户端检测机制,使遥测数据反映附加的详细信息。If this describes your solution, you might opt for client-side instrumentation so that your telemetry reflects the additional detail. 如何收集这些附加详细信息超出了此模式的范围,但你可以查看适用于网页的 Application Insights 来获取更多指导。How this additional detail is collected goes beyond the scope of this pattern, but you can review Application Insights for web pages for more direction.

使用 C#Use C#

对于 C#,如果项目是 ASP.NET,则可以在应用程序配置(例如 appsettings.json)中找到 InstrumentationKey。For C#, the InstrumentationKey is found in your application configuration, such as appsettings.json if your project is ASP.NET. 如果你不确定密钥的位置,请再次参考注册说明。Refer back to the registration instructions if you are unsure of the key location.

private static TelemetryClient _telemetryClient;

// Add a constructor that accepts a telemetry client:
public HomeController(TelemetryClient telemetry)
{
    _telemetryClient = telemetry;
}

使用 JavaScriptUse JavaScript

<script type="text/javascript">var appInsights=window.appInsights||function(config){function r(config){t[config]=function(){var i=arguments;t.queue.push(function(){t[config].apply(t,i)})}}var t={config:config},u=document,e=window,o="script",s=u.createElement(o),i,f;s.src=config.url||"//az416426.vo.msecnd.net/scripts/a/ai.0.js";u.getElementsByTagName(o)[0].parentNode.appendChild(s);try{t.cookie=u.cookie}catch(h){}for(t.queue=[],i=["Event","Exception","Metric","PageView","Trace","Dependency"];i.length;)r("track"+i.pop());return r("setAuthenticatedUserContext"),r("clearAuthenticatedUserContext"),config.disableExceptionTracking||(i="onerror",r("_"+i),f=e[i],e[i]=function(config,r,u,e,o){var s=f&&f(config,r,u,e,o);return s!==!0&&t["_"+i](config,r,u,e,o),s}),t}
({
instrumentationKey: "<YOUR INSTRUMENTATION KEY>"
});
window.appInsights=appInsights;
</script>

步骤 2:请求用于关联的搜索 IDStep 2: Request a Search ID for correlation

为了将搜索请求与单击相关联,必须具有一个将这两个不同事件关联起来的相关性 ID。To correlate search requests with clicks, it's necessary to have a correlation ID that relates these two distinct events. 使用 HTTP 标头请求搜索 ID 时,Azure 认知搜索将提供该 ID。Azure Cognitive Search provides you with a search ID when you request it with an HTTP header.

使用搜索 ID 可将 Azure 认知搜索为请求本身发出的指标关联到在 Application Insights 中记录的自定义指标。Having the search ID allows correlation of the metrics emitted by Azure Cognitive Search for the request itself, with the custom metrics you are logging in Application Insights.

使用 C#Use C#

// This sample uses the .NET SDK https://www.nuget.org/packages/Microsoft.Azure.Search

var client = new SearchIndexClient(<SearchServiceName>, <IndexName>, new SearchCredentials(<QueryKey>)

// Use HTTP headers so that you can get the search ID from the response
var headers = new Dictionary<string, List<string>>() { { "x-ms-azs-return-searchid", new List<string>() { "true" } } };
var response = await client.Documents.SearchWithHttpMessagesAsync(searchText: searchText, searchParameters: parameters, customHeaders: headers);
string searchId = string.Empty;
if (response.Response.Headers.TryGetValues("x-ms-azs-searchid", out IEnumerable<string> headerValues))
{
    searchId = headerValues.FirstOrDefault();
}

使用 JavaScript(调用 REST API)Use JavaScript (calling REST APIs)

request.setRequestHeader("x-ms-azs-return-searchid", "true");
request.setRequestHeader("Access-Control-Expose-Headers", "x-ms-azs-searchid");
var searchId = request.getResponseHeader('x-ms-azs-searchid');

步骤 3:记录搜索事件Step 3: Log Search events

每当用户发出搜索请求时,应在 Application Insights 自定义事件上使用以下架构,将该请求记录为一个搜索事件。Every time that a search request is issued by a user, you should log that as a search event with the following schema on an Application Insights custom event. 请记得仅记录用户生成的搜索查询。Remember to log only user-generated search queries.

  • SearchServiceName:(字符串)搜索服务名称SearchServiceName: (string) search service name
  • SearchId:(GUID) 搜索查询的唯一标识符(位于搜索响应中)SearchId: (guid) unique identifier of the search query (comes in the search response)
  • IndexName:(字符串)要查询的搜索服务索引IndexName: (string) search service index to be queried
  • QueryTerms:(字符串)用户输入的搜索字词QueryTerms: (string) search terms entered by the user
  • ResultCount:(整数)返回的文档数(位于搜索响应中)ResultCount: (int) number of documents that were returned (comes in the search response)
  • ScoringProfile:(字符串)使用的评分配置文件的名称(如果有)ScoringProfile: (string) name of the scoring profile used, if any

备注

通过向搜索查询添加 $count=true 来请求用户生成的查询的计数。Request the count of user generated queries by adding $count=true to your search query. 有关详细信息,请参阅搜索文档 (REST)For more information, see Search Documents (REST).

使用 C#Use C#

var properties = new Dictionary <string, string> 
{
    {"SearchServiceName", <service name>},
    {"SearchId", <search Id>},
    {"IndexName", <index name>},
    {"QueryTerms", <search terms>},
    {"ResultCount", <results count>},
    {"ScoringProfile", <scoring profile used>}
};
_telemetryClient.TrackEvent("Search", properties);

使用 JavaScriptUse JavaScript

appInsights.trackEvent("Search", {
  SearchServiceName: <service name>,
  SearchId: <search id>,
  IndexName: <index name>,
  QueryTerms: <search terms>,
  ResultCount: <results count>,
  ScoringProfile: <scoring profile used>
});

步骤 4:记录单击事件Step 4: Log Click events

每次用户单击文档,都是一个必须记录以用于搜索分析的信号。Every time that a user clicks on a document, that's a signal that must be logged for search analysis purposes. 使用 Application Insights 自定义事件可利用下面的架构来记录这些事件:Use Application Insights custom events to log these events with the following schema:

  • ServiceName:(字符串)搜索服务名称ServiceName: (string) search service name
  • SearchId:(GUID) 相关搜索查询的唯一标识符SearchId: (guid) unique identifier of the related search query
  • DocId:(字符串)文档标识符DocId: (string) document identifier
  • Position:(整数)文档在搜索结果页中的排名Position: (int) rank of the document in the search results page

备注

Position 指的是应用程序中的基数顺序。Position refers to the cardinal order in your application. 可以随意设置此数字以用于比较,只要它始终相同。You are free to set this number, as long as it's always the same, to allow for comparison.

使用 C#Use C#

var properties = new Dictionary <string, string> 
{
    {"SearchServiceName", <service name>},
    {"SearchId", <search id>},
    {"ClickedDocId", <clicked document id>},
    {"Rank", <clicked document position>}
};
_telemetryClient.TrackEvent("Click", properties);

使用 JavaScriptUse JavaScript

appInsights.trackEvent("Click", {
    SearchServiceName: <service name>,
    SearchId: <search id>,
    ClickedDocId: <clicked document id>,
    Rank: <clicked document position>
});

3 - 在 Power BI 中进行分析3 - Analyze in Power BI

检测到应用并确认应用程序已正确连接到 Application Insights 后,下载一个预定义的报表模板以在 Power BI Desktop 中分析数据。After you have instrumented your app and verified your application is correctly connected to Application Insights, you download a predefined report template to analyze data in Power BI desktop. 该报告包含预定义的图表和表,它们可用于分析为搜索流量分析捕获的其他数据。The report contains predefined charts and tables useful for analyzing the additional data captured for search traffic analytics.

  1. 在 Azure 认知搜索仪表板左侧导航窗格中,在“设置”下,单击“搜索流量分析”。In the Azure Cognitive Search dashboard left-navigation pane, under Settings, click Search traffic analytics.

  2. 在“搜索流量分析”页面上,在步骤 3 中,单击“获取 Power BI Desktop”以安装 Power BI。On the Search traffic analytics page, in step 3, click Get Power BI Desktop to install Power BI.

    获取 Power BI 报表Get Power BI reports

  3. 在同一页面上,单击“下载 Power BI 报表”。On the same page, click Download Power BI report.

  4. 该报表将在 Power BI Desktop 中打开,并且会提示你连接到 Application Insights 并提供凭据。The report opens in Power BI Desktop, and you are prompted to connect to Application Insights and provide credentials. 可以在你的 Application Insights 资源的 Azure 门户页面中找到连接信息。You can find connection information in the Azure portal pages for your Application Insights resource. 对于凭据,请提供用于门户登录的相同用户名和密码。For credentials, provide the same user name and password that you use for portal sign-in.

    连接到 Application InsightsConnect to Application Insights

  5. 单击“加载”。Click Load.

该报表包含图表和表,可帮助你做出更明智的决策来提高搜索性能和相关性。The report contains charts and tables that help you make more informed decisions to improve your search performance and relevance.

指标包括以下各项:Metrics included the following items:

  • 搜索量和最常用术语-文档对:导致同一文档被单击的词,按单击次数排序。Search volume and most popular term-document pairs: terms that result in the same document clicked, ordered by clicks.
  • 无单击的搜索:查询次数最多但未记录任何单击的词Searches without clicks: terms for top queries that register no clicks

以下屏幕截图显示了使用了所有架构元素的内置报表的外观。The following screenshot shows what a built-in report might look like if you have used all of the schema elements.

用于 Azure 认知搜索的 Power BI 仪表板Power BI dashboard for Azure Cognitive Search

后续步骤Next steps

检测搜索应用程序,以获取提供深入见解的有关搜索服务的强大数据。Instrument your search application to get powerful and insightful data about your search service.

你可以查找有关 Application Insights 的更多信息并访问定价页面来详细了解其各种服务层级。You can find more information on Application Insights and visit the pricing page to learn more about their different service tiers.

了解有关创建出色报告的详细信息。Learn more about creating amazing reports. 有关详细信息,请参阅 Power BI Desktop 入门See Getting started with Power BI Desktop for details.