Collect telemetry data for search traffic analytics

Search traffic analytics is a pattern for collecting telemetry about user interactions with your Azure AI Search application, such as user-initiated clickstream events and keyboard inputs. Using this information, you can determine the effectiveness of your search solution, including clickthrough rate and which query inputs yield zero results.

Instrumentation has the following parts:

• Add a telemetry client
• Modify a search request to include a correlation Id that maps search results to user actions
• Create and send a custom event to Application Insights and use the visualization and reporting tools to view event data

This pattern takes a dependency on Application Insights (a feature of Azure Monitor) to collect user data. It requires that you add instrumentation to your application code, as described in this article. Finally, you need a reporting mechanism to analyze the data. You can use any visualization tool that connects to Application Insights.

Prerequisites

• Azure AI Search, any region, basic tier and above.

• Application Insights.

• A rich client application providing an interactive search experience that includes clickstream events or other user actions that you want to correlate to search result selections.

Identify relevant search data

To collect useful metrics for search traffic analytics, it's necessary to log some signals from the users of your search application. These signals signify content that users are interested in and that they consider relevant. For search traffic analytics, these include:

• User-generated search events: Only search queries initiated by a user are interesting. Other search requests, such as those used to populate facets or retrieve internal information, aren't important. Be sure to only instrument user-initiated events to avoid skew or bias in your results.

• User-generated clickstream events: On a search results page, a clickstream event generally means that a document is a relevant result for a specific search query.

In your application code, you should correlate these events with the search results returned from a given query. By linking search and clickstream events with a correlation ID, you can gain a deeper understanding of how well your application's search functionality is performing.

Add search traffic analytics

For Azure AI Search, the Azure portal provides a Search Traffic Analytics page that has C# and JavaScript code snippets for adding a telemetry client, request headers, and properties necessary for custom log events.

Step 1: Set up Application Insights

Create an object that sends events to Application Insights. You can add instrumentation to your server-side application code or client-side code running in a browser, expressed here as C# and JavaScript variants. For other languages, see supported platforms and frameworks.

Server-side telemetry captures metrics at the application layer, for example in applications running as a web service on Azure, or as an on-premises app on a corporate network. Server-side telemetry captures search and clickstream events, the position of a document in results, and query information, but your data collection will be scoped to whatever information is available at that layer.

On the client, you might have other code that manipulates query inputs, adds navigation, or includes context (for example, queries initiated from a home page versus a product page). If this describes your solution, you might opt for client-side instrumentation so that your telemetry reflects the extra detail. How this extra detail is collected goes beyond the scope of this pattern, but you can review Application Insights for web pages for help with that decision.

In this step, provide a connection string to Application Insights.

var telemetryConfiguration = new TelemetryConfiguration
{
    ConnectionString = "<PUT YOUR CONNECTION STRING HERE>"
};
var telemetryClient = new TelemetryClient(telemetryConfiguration);

const appInsights = new ApplicationInsights({ config: {
  connectionString: '<PUT YOUR CONNECTION STRING HERE>'
  /* ...Other Configuration Options... */
} });
appInsights.loadAppInsights();

Step 2: Add instrumentation

Add instrumentation code to your client application.

Correlate clickstream events with search results

To correlate search requests with clicks, it's necessary to have a correlation ID that relates these two distinct events. Azure AI Search provides you with a search ID when you request it with an HTTP header.

Having the search ID allows correlation of the metrics emitted by Azure AI Search for the request itself, with the custom metrics you're logging in Application Insights.

var client = new SearchClient(new Uri("https://contoso.search.azure.cn"), "hotels-sample-index", new DefaultAzureCredential());

// Generate a new correlation id for logs
string searchId = Guid.NewGuid().ToString();
string searchText = "*";
SearchResults<SearchDocument> searchResults;

// Set correlation id for search request
using (HttpPipeline.CreateClientRequestIdScope(clientRequestId: searchId))
{
    searchResults = client.Search<SearchDocument>(searchText, options: new SearchOptions { IncludeTotalCount = true } );
}

const searchId = uuidv4();
const searchText = "*";
const searchResults = await searchClient.search(searchText, { includeTotalCount: true, customHeaders: { "x-ms-client-request-id": searchId }});
const properties = {
    searchId: searchId,
    serviceName: "<PUT YOUR SEARCH SERVICE NAME HERE, example contoso-search>",
    indexName: "<PUT YOUR INDEX NAME HERE>",
    searchText: searchText,
    resultsCount: searchResults.count
};
appInsights.trackEvent({ name: "search" }, properties);

Log custom events

Every time that a search request is issued by a user, you should log that as a search event with the following schema on an Application Insights custom event. Remember to log only user-generated search queries.

• SearchId: (guid) unique identifier of the search query (built into the search response)
• SearchServiceName: (string) search service name
• IndexName: (string) search service index to be queried
• SearchText: (string) search terms entered by the user
• ResultCount: (int) number of documents that were returned (built into the search response)

// Create properties for telemetry
var properties = new Dictionary<string, string>
{
    ["searchId"] = searchId,
    ["serviceName"] = "<PUT YOUR SEARCH SERVICE NAME HERE, example: contoso-search>",
    ["indexName"] = "<PUT YOUR INDEX NAME HERE>",
    ["searchText"] = searchText,
    ["resultsCount"] = searchResults.TotalCount?.ToString()
};

const properties = {
    searchId: searchId,
    serviceName: "<PUT YOUR SEARCH SERVICE NAME HERE, example contoso-search>",
    indexName: "<PUT YOUR INDEX NAME HERE>",
    searchText: searchText,
    resultsCount: searchResults.count
};

Send the custom event to Application Insights

Add the custom even to the custom events table in Application Insights. For more information, see Application Insights API for custom events and metrics.

telemetryClient.TrackEvent("search", properties);
telemetryClient.Flush();

appInsights.trackEvent({ name: "search" }, properties);

Step 3: Review logs

Use any of the approaches supported by Application Insights for viewing custom events.

• Create or edit an Azure Workbook
• Create and share dashboards of Log Analytics data
• Integrate Log Analytics with Power BI

Next steps

You can find more information on Application Insights and visit the pricing page to learn more about their different service tiers.

Learn more about creating reports. See Getting started with Power BI Desktop for details.