在 Azure 认知搜索中监视资源使用情况和查询活动Monitor resource consumption and query activity in Azure Cognitive Search

可以在 Azure 认知搜索服务的“概览”页中查看系统数据,了解资源使用情况、查询指标,以及有多少配额可以用来创建更多的索引、索引器和数据源。In the Overview page of your Azure Cognitive Search service, you can view system data about resource usage, query metrics, and how much quota is available to create more indexes, indexers, and data sources. 也可通过门户来配置 Log Analytics 或其他用于进行持久数据收集的资源。You can also use the portal to configure log analytics or another resource used for persistent data collection.

对于自我诊断和保留操作历史记录而言,设置日志十分有用。Setting up logs is useful for self-diagnostics and preserving operational history. 在内部,日志会短时存在于后端,但足以进行调查和分析(如果你提交了支持票证)。Internally, logs exist on the backend for a short period of time, sufficient for investigation and analysis if you file a support ticket. 若要控制和访问日志信息,则应设置本文所述的某个解决方案。If you want control over and access to log information, you should set up one of the solutions described in this article.

本文介绍监视选项、如何启用日志记录和日志存储,以及如何查看日志内容。In this article, learn about your monitoring options, how to enable logging and log storage, and how to view log contents.

指标概览Metrics at a glance

内置到“概览”页中的“使用情况”和“监视”部分会针对资源使用情况和查询执行指标进行报告。 Usage and Monitoring sections built into the Overview page report out on resource consumption and query execution metrics. 只要你开始使用此服务,就可以使用此信息,不需进行配置。This information becomes available as soon as you start using the service, with no configuration required. 此页每隔几分钟就会刷新一次。This page is refreshed every few minutes. 对于将哪个层用于生产工作负荷或是否要调整活动副本和分区的数目这样的问题,可以根据这些指标进行最终决策,因为这些指标会显示资源的消耗速度,以及当前配置处理现有负载的有效程度。If you are finalizing decisions about which tier to use for production workloads, or whether to adjust the number of active replicas and partitions, these metrics can help you with those decisions by showing you how quickly resources are consumed and how well the current configuration handles the existing load.

“使用情况” 选项卡显示资源可用性(相当于当前限制而言)。The Usage tab shows you resource availability relative to current limits. 下图描述免费服务的情况,该服务的上限是每个类型 3 个对象,最大存储为 50 MB。The following illustration is for the free service, which is capped at 3 objects of each type and 50 MB of storage. “基本”或“标准”服务的限制更高,在增加分区计数的情况下,最大存储会按比例增大。A Basic or Standard service has higher limits, and if you increase the partition counts, maximum storage goes up proportionally.

相对于有效限制的使用状态Usage status relative to effective limits

每秒查询次数 (QPS) 和其他指标Queries per second (QPS) and other metrics

“监视”选项卡 显示每秒搜索查询数 (简称 QPS,每分钟汇总一次)之类的指标的移动平均。The Monitoring tab shows moving averages for metrics like search Queries Per Second (QPS), aggregated per minute. 搜索延迟是搜索服务处理搜索查询所需的时间(每分钟汇总一次)。 Search latency is the amount of time the search service needed to process search queries, aggregated per minute. 限制的搜索查询百分比(未显示)是受限制的搜索查询的百分比(也是每分钟汇总一次)。 Throttled search queries percentage (not shown) is the percentage of search queries that were throttled, also aggregated per minute.

这些数字是大概的数字,只是让你大致了解系统处理请求的情况。These numbers are approximate and are intended to give you a general idea of how well your system is servicing requests. 实际的 QPS 可能高于或低于门户中报告的数字。Actual QPS may be higher or lower than the number reported in the portal.

每秒查询次数活动Queries per second activity

活动日志Activity logs

活动日志从 Azure 资源管理器收集信息。The Activity log collects information from Azure Resource Manager. 例如,在活动日志中发现的信息包括:创建或删除服务、更新资源组、查看名称可用性,或者获取处理请求所需的服务访问密钥。Examples of information found in the Activity log include creating or deleting a service, updating a resource group, checking for name availability, or getting a service access key to handle a request.

可以通过左侧导航窗格、顶部窗口命令栏中的“通知”或者“诊断并解决问题”页访问活动日志You can access the Activity log from the left-navigation pane, or from Notifications in the top window command bar, or from the Diagnose and solve problems page.

对于服务内任务(例如创建索引或删除数据源),你会看到针对每个请求的常规通知(例如“获取管理密钥”),但看不到具体操作本身。For in-service tasks like creating an index or deleting a data source, you'll see generic notifications like "Get Admin Key" for each request, but not the specific action itself. 对于这种级别的信息,必须启用附加监视解决方案。For this level of information, you must enable an add-on monitoring solution.

附加监视解决方案Add-on monitoring solutions

Azure 认知搜索不在其管理的对象之外存储任何数据,这意味着日志数据必须在外部进行存储。Azure Cognitive Search does not store any data beyond the objects it manages, which means log data has to be stored externally. 若要保留日志数据,可以配置下面的任何资源。You can configure any of the resources below if you want to persist log data.

下表比较了各种选项,这些选项用于存储日志以及添加深度监视指标,以便通过 Application Insights 监视服务操作和查询工作负荷。The following table compares options for storing logs and adding in-depth monitoring of service operations and query workloads through Application Insights.

资源Resource 用途Used for
Azure Monitor 日志Azure Monitor logs 记录的事件和查询指标,基于下面的架构。Logged events and query metrics, based on the schemas below. 事件记录到 Log Analytics 工作区。Events are logged to a Log Analytics workspace. 可以针对工作区运行查询,以便从日志返回详细信息。You can run queries against a workspace to return detailed information from the log.
Blob 存储Blob storage 记录的事件和查询指标,基于下面的架构。Logged events and query metrics, based on the schemas below. 事件记录到 Blob 容器并存储在 JSON 文件中。Events are logged to a Blob container and stored in JSON files. 使用 JSON 编辑器来查看文件内容。Use a JSON editor to view file contents.
事件中心Event Hub 记录的事件和查询指标,基于本文中记录的架构。Logged events and query metrics, based on the schemas documented in this article. 对于很大的日志,请选择此项作为备用数据收集服务。Choose this as an alternative data collection service for very large logs.

Azure Monitor 日志和 Blob 存储均以免费服务的形式提供,让你可以在 Azure 订阅的生存期内免费试用它。Both Azure Monitor logs and Blob storage are available as a free service so that you can try it out at no charge for the lifetime of your Azure subscription. Application Insights 可以免费注册和使用,前提是应用程序数据大小不超出特定限制(有关详细信息,请参阅定价页)。Application Insights is free to sign up and use as long as application data size is under certain limits (see the pricing page for details).

下一部分详述如何通过多个步骤来启用和使用 Azure Blob 存储,以便收集和访问 Azure 认知搜索操作创建的日志数据。The next section walks you through the steps of enabling and using Azure Blob storage to collect and access log data created by Azure Cognitive Search operations.

启用日志记录Enable logging

索引编制和查询工作负荷的日志记录默认处于关闭状态,并依赖于日志记录基础结构和长期外部存储的附加解决方案。Logging for indexing and query workloads is off by default and depends on add-on solutions for both logging infrastructure and long-term external storage. Azure 认知搜索本身保留的数据只有其创建和管理的对象,因此日志必须存储在别处。By itself, the only persisted data in Azure Cognitive Search are the objects it creates and manages, so logs must be stored elsewhere.

本部分介绍如何使用 Blob 存储来存储记录的事件和指标数据。In this section, you'll learn how to use Blob storage to store logged events and metrics data.

  1. 创建存储帐户(如果还没有)。Create a storage account if you don't already have one. 可以将它置于 Azure 认知搜索所在的资源组中,这样当你以后需要删除本练习中使用的所有资源时,就可以简化清理过程。You can place it in the same resource group as Azure Cognitive Search to simplify clean up later if you want to delete all resources used in this exercise.

    存储帐户必须存在于 Azure 认知搜索所在的区域。Your storage account must exist in the same region as Azure Cognitive Search.

  2. 打开搜索服务的“概览”页。Open your search service Overview page. 在左侧导航窗格中,向下滚动到“监视”,然后单击“启用监视”。 In the left-navigation pane, scroll down to Monitoring and click Enable Monitoring.

    启用监视Enable monitoring

  3. 选择要导出的数据:日志和/或指标。Choose the data you want to export: Logs, Metrics or both. 可将数据复制到存储帐户,将其发送到事件中心,或将其导出到 Azure Monitor 日志。You can copy it to a storage account, send it to an event hub or export it to Azure Monitor logs.

    若要存档到 Blob 存储,只有存储帐户必须存在。For archival to Blob storage, only the storage account must exist. 容器和 Blob 会在导出日志数据时根据需要创建。Containers and blobs will be created as-needed when log data is exported.

    配置 blob 存储存档Configure blob storage archive

  4. 保存配置文件。Save the profile.

  5. 通过创建或删除对象(创建日志事件)以及通过提交查询(生成指标)来测试日志记录。Test logging by creating or deleting objects (creates log events) and by submitting queries (generates metrics).

保存配置文件后,日志记录就会启用。Logging is enabled once you save the profile. 仅当存在要记录或度量的活动时,才会创建容器。Containers are only created when there is an activity to log or measure. 将数据复制到存储帐户时,数据会被格式化为 JSON 并置于两个容器中:When the data is copied to a storage account, the data is formatted as JSON and placed in two containers:

  • insights-logs-operationlogs:用于搜索流量日志insights-logs-operationlogs: for search traffic logs
  • insights-metrics-pt1m:用于指标insights-metrics-pt1m: for metrics

一个小时后,容器才会出现在 Blob 存储中。每个容器每小时会有一个 Blob。It takes one hour before the containers will appear in Blob storage. There is one blob, per hour, per container.

可以使用 Visual Studio Code 或其他 JSON 编辑器来查看文件。You can use Visual Studio Code or another JSON editor to view the files.

示例路径Example path

resourceId=/subscriptions/<subscriptionID>/resourcegroups/<resourceGroupName>/providers/microsoft.search/searchservices/<searchServiceName>/y=2018/m=12/d=25/h=01/m=00/name=PT1H.json

日志架构Log schema

包含搜索服务流量日志的 Blob 的结构如此部分所述。Blobs containing your search service traffic logs are structured as described in this section. 每个 Blob 都有一个名为 records 的根对象,该对象包含一组日志对象。Each blob has one root object called records containing an array of log objects. 每个 Blob 包含同一小时内发生的所有操作的记录。Each blob contains records for all the operations that took place during the same hour.

名称Name 类型Type 示例Example 注释Notes
timetime datetimedatetime "2018-12-07T00:00:43.6872559Z""2018-12-07T00:00:43.6872559Z" 操作的时间戳Timestamp of the operation
ResourceIdresourceId stringstring “/SUBSCRIPTIONS/11111111-1111-1111-1111-111111111111/"/SUBSCRIPTIONS/11111111-1111-1111-1111-111111111111/
RESOURCEGROUPS/DEFAULT/PROVIDERS/RESOURCEGROUPS/DEFAULT/PROVIDERS/
MICROSOFT.SEARCH/SEARCHSERVICES/SEARCHSERVICE”MICROSOFT.SEARCH/SEARCHSERVICES/SEARCHSERVICE"
ResourceIdYour ResourceId
operationNameoperationName stringstring “Query.Search”"Query.Search" 操作的名称The name of the operation
operationVersionoperationVersion stringstring “2019-05-06”"2019-05-06" 使用的 api-versionThe api-version used
categorycategory stringstring “OperationLogs”"OperationLogs" 常量constant
resultTyperesultType stringstring “Success”"Success" 可能的值:Success 或 FailurePossible values: Success or Failure
resultSignatureresultSignature intint 200200 HTTP 结果代码HTTP result code
durationMSdurationMS intint 5050 操作持续时间,以毫秒为单位Duration of the operation in milliseconds
propertiesproperties objectobject 请参阅下表see the following table 包含特定于操作的数据的对象Object containing operation-specific data

属性架构Properties schema

名称Name 类型Type 示例Example 注释Notes
说明Description stringstring “GET /indexes('content')/docs”"GET /indexes('content')/docs" 操作的终结点The operation's endpoint
查询Query stringstring “?search=AzureSearch&$count=true&api-version=2019-05-06”"?search=AzureSearch&$count=true&api-version=2019-05-06" 查询参数The query parameters
文档Documents intint 4242 处理的文档数目Number of documents processed
IndexNameIndexName stringstring “testindex”"testindex" 与操作关联的索引名称Name of the index associated with the operation

度量值架构Metrics schema

针对查询请求来捕获指标。Metrics are captured for query requests.

名称Name 类型Type 示例Example 注释Notes
ResourceIdresourceId stringstring “/SUBSCRIPTIONS/11111111-1111-1111-1111-111111111111/"/SUBSCRIPTIONS/11111111-1111-1111-1111-111111111111/
RESOURCEGROUPS/DEFAULT/PROVIDERS/RESOURCEGROUPS/DEFAULT/PROVIDERS/
MICROSOFT.SEARCH/SEARCHSERVICES/SEARCHSERVICE”MICROSOFT.SEARCH/SEARCHSERVICES/SEARCHSERVICE"
资源 IDyour resource ID
metricNamemetricName stringstring “Latency”"Latency" 度量值名称the name of the metric
timetime datetimedatetime "2018-12-07T00:00:43.6872559Z""2018-12-07T00:00:43.6872559Z" 操作的时间戳the operation's timestamp
averageaverage intint 6464 度量值时间间隔内原始样本的平均值The average value of the raw samples in the metric time interval
最小值minimum intint 3737 度量值时间间隔内原始样本的最小值The minimum value of the raw samples in the metric time interval
最大值maximum intint 7878 度量值时间间隔内原始样本的最大值The maximum value of the raw samples in the metric time interval
totaltotal intint 258258 度量值时间间隔内原始样本的总计值The total value of the raw samples in the metric time interval
countcount intint 44 用于生成度量值的原始样本数The number of raw samples used to generate the metric
timegraintimegrain stringstring “PT1M”"PT1M" 采用 ISO 8601 的度量值时间粒度The time grain of the metric in ISO 8601

所有度量值会按一分钟的时间间隔报告。All metrics are reported in one-minute intervals. 每个度量值都会显示每分钟的最小、最大和平均值。Every metric exposes minimum, maximum and average values per minute.

对于 SearchQueriesPerSecond 度量值,最小值是该分钟内已注册的每秒搜索查询次数最低值。For the SearchQueriesPerSecond metric, minimum is the lowest value for search queries per second that was registered during that minute. 最大值也是如此。The same applies to the maximum value. 平均值是一分钟内的聚合值。Average, is the aggregate across the entire minute. 假设一分钟内出现以下情形:有 1 秒出现高负载(这是 SearchQueriesPerSecond 的最大值),紧接着有 58 秒的平均负载,最后 1 秒只有 1 个查询(这是最小值)。Think about this scenario during one minute: one second of high load that is the maximum for SearchQueriesPerSecond, followed by 58 seconds of average load, and finally one second with only one query, which is the minimum.

对于 ThrottledSearchQueriesPercentage、minimum、maximum、average 和 total,全都具有相同的值:在一分钟内的搜索查询总数中,已限制搜索查询百分比。For ThrottledSearchQueriesPercentage, minimum, maximum, average and total, all have the same value: the percentage of search queries that were throttled, from the total number of search queries during one minute.

下载文件并在 Visual Studio Code 中打开Download and open in Visual Studio Code

可以使用任何 JSON 编辑器来查看日志文件。You can use any JSON editor to view the log file. 如果没有编辑器,建议使用 Visual Studio CodeIf you don't have one, we recommend Visual Studio Code.

  1. 在 Azure 门户中打开存储帐户。In Azure portal, open your Storage account.

  2. 在左侧导航窗格中,单击“Blob” 。In the left-navigation pane, click Blobs. 此时会看到 insights-logs-operationlogsinsights-metrics-pt1mYou should see insights-logs-operationlogs and insights-metrics-pt1m. 这些容器是在将日志数据导出到 Blob 存储时由 Azure 认知搜索创建的。These containers are created by Azure Cognitive Search when the log data is exported to Blob storage.

  3. 单击文件夹层次结构,直至找到 .json 文件。Click down the folder hierarchy until you reach the .json file. 通过上下文菜单来下载文件。Use the context-menu to download the file.

待文件下载完以后,在 JSON 编辑器中将其打开即可查看内容。Once the file is downloaded, open it in a JSON editor to view the contents.

使用系统 APIUse system APIs

Azure 认知搜索 REST API 和 .NET SDK 支持采用编程方式访问服务指标、索引和索引器信息,以及文档计数。Both the Azure Cognitive Search REST API and the .NET SDK provide programmatic access to service metrics, index and indexer information, and document counts.

若要使用 PowerShell 或 Azure CLI 来启用监视,请参阅此处的文档。To enable using PowerShell or the Azure CLI, see the documentation here.

后续步骤Next steps

管理 Microsoft Azure 上的搜索服务一文提供有关服务管理的详细信息,性能和优化一文提供优化指南。Manage your Search service on Microsoft Azure for more information on service administration and Performance and optimization for tuning guidance.