Visualize Azure AI Search Logs and Metrics with Power BI

  • Article

Azure AI Search can send operation logs and service metrics to an Azure Storage account, which you can then visualize in Power BI. This article explains the steps and how to use a Power BI Template App to visualize the data. The template can help you gain detailed insights about your search service, including information about queries, indexing, operations, and service metrics.

You can find the Power BI Template App Azure AI Search: Analyze Logs and Metrics in the Power BI Apps marketplace.

Set up the app

  1. Enable metric and resource logging for your search service:

    1. Create or identify an existing Azure Storage account where you can archive the logs.

    2. Navigate to your Azure AI Search service in the Azure portal.

    3. Under the Monitoring section on the left column, select Diagnostic settings.

      Screenshot showing how to select Diagnostic settings in the Monitoring section of the Azure AI Search service.

    4. Select + Add diagnostic setting.

    5. Check Archive to a storage account, provide your Storage account information, and check OperationLogs and AllMetrics.

      Screenshot showing how to make selections for metrics and resource logging in the diagnostic settings page.

    6. Select Save.

  2. After logging has been enabled, use your search service to start generating logs and metrics. It takes up to an hour before the containers will appear in Blob storage with these logs. You will see a insights-logs-operationlogs container for search traffic logs and a insights-metrics-pt1m container for metrics.

  3. Find the Azure AI Search Power BI App in the Power BI Apps marketplace and install it into a new workspace or an existing workspace. The app is called Azure AI Search: Analyze Logs and Metrics.

  4. After installing the app, select the app from your list of apps in Power BI.

    Screenshot showing the Azure AI Search app to select from the list of apps.

  5. Select Connect to connect your data

    Screenshot showing how to connect to your data in the Azure AI Search app.

  6. Input the name of the storage account that contains your logs and metrics. By default the app will look at the last 10 days of data but this value can be changed with the Days parameter.

    Screenshot showing how to input the storage account name and the number of days to query in the Connect to Azure AI Search page.

  7. Select Key as the authentication method and provide your storage account key. Select Private as the privacy level. Click Sign In and to begin the loading process.

    Screenshot showing how to input the authentication method, account key, and privacy level in the Connect to Azure AI Search page.

  8. Wait for the data to refresh. This may take some time depending on how much data you have. You can see if the data is still being refreshed based on the below indicator.

    Screenshot showing how to read the information on the data refresh page.

  9. Once the data refresh has completed, select Azure AI Search Report to view the report.

    Screenshot showing how to select the Azure AI Search Report on the data refresh page.

  10. Make sure to refresh the page after opening the report so that it opens with your data.

    Screenshot of the Azure AI Search Power BI report.

Modify app parameters

If you would like to visualize data from a different storage account or change the number of days of data to query, follow the below steps to change the Days and StorageAccount parameters.

  1. Navigate to your Power BI apps, find your Azure AI Search app and select the Edit app button to view the workspace.

    Screenshot showing how to select the Edit app button for the Azure AI Search app.

  2. Select Settings from the Dataset options.

    Screenshot showing how to select Settings from the Azure AI Search Dataset options.

  3. While in the Datasets tab, change the parameter values and select Apply. If there is an issue with the connection, update the data source credentials on the same page.

  4. Navigate back to the workspace and select Refresh now from the Dataset options.

    Screenshot showing how to select Refresh now from the Azure AI Search Dataset options.

  5. Open the report to view the updated data. You might also need to refresh the report to view the latest data.

Troubleshooting report issues

If you find that you cannot see your data follow these troubleshooting steps:

  1. Open the report and refresh the page to make sure you're viewing the latest data. There's an option in the report to refresh the data. Select this to get the latest data.

  2. Ensure the storage account name and access key you provided are correct. The storage account name should correspond to the account configured with your search service logs.

  3. Confirm that your storage account contains the containers insights-logs-operationlogs and insights-metrics-pt1m and each container has data. The logs and metrics will be within a couple layers of folders.

  4. Check to see if the dataset is still refreshing. The refresh status indicator is shown in step 8 above. If it is still refreshing, wait until the refresh is complete to open and refresh the report.

Next steps