将来自不同数据源的数据馈送添加到指标顾问Add data feeds from different data sources to Metrics Advisor

使用本文可找到将不同类型的数据源连接到指标顾问的设置和要求。Use this article to find the settings and requirements for connecting different types of data sources to Metrics Advisor. 请务必阅读如何加入数据,了解有关在指标顾问中使用数据的关键概念。Make sure to read how to Onboard your data to learn about the key concepts for using your data with Metrics Advisor.

支持的身份验证类型Supported authentication types

身份验证类型Authentication types 说明Description
基本Basic 需要提供基本参数才能访问数据源。You will need to be able to provide basic parameters for accessing data sources. 例如,连接字符串或密钥。For example a connection string or key. 数据馈送管理员能够查看这些凭据。Data feed admins are able to view these credentials.
AzureManagedIdentityAzureManagedIdentity Azure 资源的托管标识是 Azure Active Directory 的一项功能。Managed identities for Azure resources is a feature of Azure Active Directory. 它为 Azure 服务提供了 Azure AD 中的自动托管标识。It provides Azure services with an automatically managed identity in Azure AD. 可以使用此标识功能向支持 Azure AD 身份验证的任何服务进行身份验证。You can use the identity to authenticate to any service that supports Azure AD authentication.
AzureSQLConnectionStringAzureSQLConnectionString 将 AzureSQL 连接字符串作为“凭据实体”存储在指标顾问中,并在每次加入指标数据时直接使用它。Store your AzureSQL connection string as a credential entity in Metrics Advisor, and use it directly each time when onboarding metrics data. 只有凭据实体的管理员才能查看这些凭据,但允许获得授权的查看者创建数据馈送,无需知道凭据的详细信息。Only admins of the Credential entity are able to view these credentials, but enables authorized viewers to create data feeds without needing to know details for the credentials.
DataLakeGen2SharedKeyDataLakeGen2SharedKey 将数据湖帐户密钥作为“凭据实体”存储在指标顾问中,并在每次加入指标数据时直接使用它。Store your data lake account key as a credential entity in Metrics Advisor and use it directly each time when onboarding metrics data. 只有凭据实体的管理员才能查看这些凭据,但允许获得授权的查看者创建数据馈送,无需知道凭据详细信息。Only admins of the Credential entity are able to view these credentials, but enables authorized viewers to create data feed without needing to know the credential details.
服务主体Service principal 将服务主体作为“凭据实体”存储在指标顾问中,并在每次加入指标数据时直接使用它。Store your service principal as a credential entity in Metrics Advisor and use it directly each time when onboarding metrics data. 只有凭据实体的管理员才能查看这些凭据,但允许获得授权的查看者创建数据馈送,无需知道凭据详细信息。Only admins of Credential entity are able to view the credentials, but enables authorized viewers to create data feed without needing to know the credential details.
密钥保管库中的服务主体Service principal from key vault 将密钥保管库中的服务主体作为“凭据实体”存储在指标顾问中,并在每次加入指标数据时直接使用它。Store your service principal in a key vault as a credential entity in Metrics Advisor and use it directly each time when onboarding metrics data. 只有凭据实体的管理员才能查看这些凭据,但也允许查看者创建数据馈送,无需知道详细的凭据信息。Only admins of a credential entity are able to view the credentials, but also leave viewers able to create data feed without needing to know detailed credentials.

支持的数据源和相应的身份验证类型Data sources supported and corresponding authentication types

数据源Data sources 身份验证类型Authentication Types
Azure Application InsightsAzure Application Insights 基本Basic
Azure Blob 存储 (JSON) Azure Blob Storage (JSON) 基本Basic
ManagedIdentityManagedIdentity
Azure Cosmos DB (SQL) Azure Cosmos DB (SQL) 基本Basic
Azure 数据资源管理器(Kusto) Azure Data Explorer (Kusto) 基本Basic
ManagedIdentityManagedIdentity
Azure Data Lake Storage Gen2Azure Data Lake Storage Gen2 基本Basic
DataLakeGen2SharedKeyDataLakeGen2SharedKey
服务主体Service principal
密钥保管库中的服务主体Service principal from key vault
Azure SQL 数据库 / SQL ServerAzure SQL Database / SQL Server 基本Basic
ManagedIdentityManagedIdentity
服务主体Service principal
密钥保管库中的服务主体Service principal from key vault
AzureSQLConnectionStringAzureSQLConnectionString
Azure 表存储Azure Table Storage 基本Basic
ElasticSearchElasticSearch 基本Basic
Http 请求Http request 基本Basic
InfluxDB (InfluxQL) InfluxDB (InfluxQL) 基本Basic
MongoDBMongoDB 基本Basic
MySQLMySQL 基本Basic
PostgreSQLPostgreSQL 基本Basic

创建“凭据实体”并使用它对数据源进行身份验证。Create an Credential entity and use it for authenticating to your data sources. 以下各节指定基本身份验证所需的参数。The following sections specify the parameters required by for Basic authentication.

Azure Application InsightsAzure Application Insights

  • 应用程序 ID :使用 Application Insights API 时,可以使用应用程序 ID 来标识此应用程序。Application ID : This is used to identify this application when using the Application Insights API. 若要获取应用程序 ID,请执行以下操作:To get the Application ID, do the following:

    1. 在 Application Insights 资源中,单击“API 访问权限” 。From your Application Insights resource, click API Access.

    2. 将生成的应用程序 ID 复制到指标顾问中的“应用程序 ID”字段。Copy the Application ID generated into Application ID field in Metrics Advisor.

  • API 密钥 :浏览器外的应用程序使用 API 密钥访问此资源。API Key : API keys are used by applications outside the browser to access this resource. 若要获取 API 密钥,请执行以下操作:To get the API key, do the following:

    1. 在 Application Insights 资源中,单击“API 访问权限” 。From the Application Insights resource, click API Access.

    2. 单击“创建 API 密钥” 。Click Create API Key.

    3. 输入简短说明,选中“读取遥测”选项,然后单击“生成密钥”按钮。Enter a short description, check the Read telemetry option, and click the Generate key button.

    4. 将 API 密钥复制到指标顾问中的“API 密钥”字段。Copy the API key to the API key field in Metrics Advisor.

  • 查询 :Azure Application Insights 日志在 Azure 数据资源管理器之上构建,Azure Monitor 日志查询使用同一 Kusto 查询语言的某个版本。Query : Azure Application Insights logs are built on Azure Data Explorer, and Azure Monitor log queries use a version of the same Kusto query language. Kusto 查询语言文档提供了该语言的完整详细信息,在针对 Application Insights 编写查询时,应将此文档用作主要参考资源。The Kusto query language documentation has all of the details for the language and should be your primary resource for writing a query against Application Insights.

Azure Blob 存储 (JSON)Azure Blob Storage (JSON)

  • 连接字符串 :有关检索此字符串的信息,请参阅 Azure Blob 存储连接字符串一文。Connection String : See the Azure Blob Storage connection string article for information on retrieving this string.

  • 容器 :指标顾问期望将时序数据作为 Blob 文件存储在单个容器下(每个时间戳一个 Blob)。Container : Metrics Advisor expects time series data stored as Blob files (one Blob per timestamp) under a single container. 这是容器名称字段。This is the container name field.

  • Blob 模板 :这是 Blob 文件名的模板。Blob Template : This is the template of the Blob file names. 例如:/%Y/%m/X_%Y-%m-%d-%h-%M.jsonFor example: /%Y/%m/X_%Y-%m-%d-%h-%M.json. 支持以下参数:The following parameters are supported:

    • %Y 是格式为 yyyy 的年份%Y is the year formatted as yyyy
    • %m 是格式为 MM 的月份%m is the month formatted as MM
    • %d 是格式为 dd 的日期%d is the day formatted as dd
    • %h 是格式为 HH 的小时%h is the hour formatted as HH
    • %M 是格式为 mm 的分钟%M is the minute formatted as mm
  • JSON 格式版本 :定义 JSON 文件中的数据架构。JSON format version : Defines the data schema in the JSON files. 指标顾问目前支持两个版本:Currently Metrics Advisor supports two versions:

    • v1(默认值)v1 (Default value)

      仅接受“名称”和“值”指标 。Only the metrics Name and Value are accepted. 例如: 。For example:

      {"count":11, "revenue":1.23}
      
    • v2v2

      还接受“维度”和“时间戳”指标 。The metrics Dimensions and timestamp are also accepted. 例如: 。For example:

      [
        {"date": "2018-01-01T00:00:00Z", "market":"en-us", "count":11, "revenue":1.23},
        {"date": "2018-01-01T00:00:00Z", "market":"zh-cn", "count":22, "revenue":4.56}
      ]
      

每个 JSON 文件只能有一个时间戳。Only one timestamp is allowed per JSON file.

Azure Cosmos DB (SQL)Azure Cosmos DB (SQL)

  • 连接字符串 :访问 Azure Cosmos DB 的连接字符串。Connection String : The connection string to access your Azure Cosmos DB. 这可在 Cosmos DB 资源的“密钥”中找到。This can be found in the Cosmos DB resource, in Keys .

  • 数据库 :要查询的数据库。Database : The database to query against. 这可在“容器”部分下的“浏览”页中找到 。This can be found in the Browse page under Containers section.

  • 集合 ID :要查询的集合 ID。Collection ID : The collection ID to query against. 这可在“容器”部分下的“浏览”页中找到 。This can be found in the Browse page under Containers section.

  • SQL 查询 :一个 SQL 查询,用于获取数据并将数据构建为多维时序数据。SQL Query : A SQL query to get and formulate data into multi-dimensional time series data. 可在查询中使用 @StartTime@EndTime 变量。You can use the @StartTime and @EndTime variables in your query. 它们应格式化为:yyyy-MM-dd HH:mm:ssThey should be formatted: yyyy-MM-dd HH:mm:ss.

    示例查询:Sample query:

    select StartDate, JobStatusId, COUNT(*) AS JobNumber from IngestionJobs WHERE and StartDate = @StartTime
    

    从 2019/12/12 开始的数据切片的示例查询:Sample query for a data slice from 2019/12/12:

    select StartDate, JobStatusId, COUNT(*) AS JobNumber from IngestionJobs WHERE and StartDate = '2019-12-12 00:00:00'
    

Azure 数据资源管理器(Kusto)Azure Data Explorer (Kusto)

  • 连接字符串 :指标顾问支持使用 Azure AD 应用程序身份验证访问 Azure 数据资源管理器(Kusto)。Connection String : Metrics Advisor supports accessing Azure Data Explorer(Kusto) by using Azure AD application authentication. 你将需要创建并注册一个 Azure AD 应用程序,然后授权其访问 Azure 数据资源管理器数据库。You will need to create and register an Azure AD application and then authorize it to access an Azure Data Explorer database.

  • 查询 :请参阅 Kusto 查询语言,了解如何获取数据并将其表述为多维时序数据。Query : See Kusto Query Language to get and formulate data into multi-dimensional time series data. 可在查询中使用 @StartTime@EndTime 变量。You can use the @StartTime and @EndTime variables in your query. 它们应格式化为:yyyy-MM-dd HH:mm:ssThey should be formatted: yyyy-MM-dd HH:mm:ss.

Azure Data Lake Storage Gen2Azure Data Lake Storage Gen2

  • 帐户名称 :Azure Data Lake Storage Gen2 的帐户名称。Account Name : The account name of your Azure Data Lake Storage Gen2. 这可在 Azure 存储帐户 (Azure Data Lake Storage Gen2) 资源的“访问密钥”中找到。This can be found in your Azure Storage Account (Azure Data Lake Storage Gen2) resource in Access keys .

  • 帐户密钥 :请指定访问 Azure Data Lake Storage Gen2 的帐户名称。Account Key : Please specify the account name to access your Azure Data Lake Storage Gen2. 这可在 Azure 存储帐户 (Azure Data Lake Storage Gen2) 资源的“访问密钥”设置中找到。This could be found in Azure Storage Account (Azure Data Lake Storage Gen2) resource in Access keys setting.

  • 文件系统名称(容器) :指标顾问将期望你的时序数据作为 Blob 文件存储在单个容器下(每个时间戳一个 Blob)。File System Name (Container) : Metrics Advisor will expect your time series data stored as Blob files (one Blob per timestamp) under a single container. 这是容器名称字段。This is the container name field. 这可在 Azure 存储帐户 (Azure Data Lake Storage Gen2) 实例中找到,然后单击“Blob 服务”部分中的“容器”。This can be found in your Azure storage account (Azure Data Lake Storage Gen2) instance, and click 'Containers' in 'Blob Service' section.

  • 目录模板 :这是 Blob 文件的目录模板。Directory Template : This is the directory template of the Blob file. 例如:/%Y/%m/%d。For example: /%Y/%m/%d . 支持以下参数:The following parameters are supported:

    • %Y 是格式为 yyyy 的年份%Y is the year formatted as yyyy
    • %m 是格式为 MM 的月份%m is the month formatted as MM
    • %d 是格式为 dd 的日期%d is the day formatted as dd
    • %h 是格式为 HH 的小时%h is the hour formatted as HH
    • %M 是格式为 mm 的分钟%M is the minute formatted as mm
  • 文件模板 :这是 Blob 文件的文件模板。File Template : This is the file template of the Blob file. 例如:X_%Y-%m-%d-%h-%M.json。For example: X_%Y-%m-%d-%h-%M.json . 支持以下参数:The following parameters are supported:

    • %Y 是格式为 yyyy 的年份%Y is the year formatted as yyyy
    • %m 是格式为 MM 的月份%m is the month formatted as MM
    • %d 是格式为 dd 的日期%d is the day formatted as dd
    • %h 是格式为 HH 的小时%h is the hour formatted as HH
    • %M 是格式为 mm 的分钟%M is the minute formatted as mm

目前,指标顾问支持 JSON 文件中的数据架构,如下所示。Currently Metrics Advisor supports the data schema in the JSON files as follow. 例如: 。For example:

[
  {"date": "2018-01-01T00:00:00Z", "market":"en-us", "count":11, "revenue":1.23},
  {"date": "2018-01-01T00:00:00Z", "market":"zh-cn", "count":22, "revenue":4.56}
]

Azure SQL 数据库 | SQL ServerAzure SQL Database | SQL Server

  • 连接字符串 :指标顾问接受 ADO.NET 样式连接字符串用于 sql server 数据源。Connection String : Metrics Advisor accepts an ADO.NET Style Connection String for sql server data source.

    示例连接字符串:Sample connection string:

    Data Source=db-server.database.chinacloudapi.cn:[port];initial catalog=[database];User ID=[username];Password=[password];Connection Timeout=10ms;
    
  • 查询 :一个 SQL 查询,用于获取数据并将数据构建为多维时序数据。Query : A SQL query to get and formulate data into multi-dimensional time series data. 可在查询中使用 @StartTime 变量来帮助获取预期指标值。You can use a @StartTime variable in your query to help with getting expected metrics value.

    • @StartTime:格式为 yyyy-MM-dd HH:mm:ss 的日期/时间@StartTime: a datetime in the format of yyyy-MM-dd HH:mm:ss

      示例查询:Sample query:

      select StartDate, JobStatusId, COUNT(*) AS JobNumber from IngestionJobs WHERE and StartDate = @StartTime
      

      为 2019/12/12 的数据切片执行的实际查询:Actual query executed for data slice of 2019/12/12:

      select StartDate, JobStatusId, COUNT(*) AS JobNumber from IngestionJobs WHERE and StartDate = '2019-12-12 00:00:00'
      

Azure 表存储Azure Table Storage

  • 连接字符串 :若要了解如何从 Azure 表存储中检索连接字符串,请参阅查看并复制连接字符串Connection String : Please refer to View and copy a connection string for information on how to retrieve the connection string from Azure Table Storage.

  • 表名 :指定要查询的表。Table Name : Specify a table to query against. 这可在 Azure 存储帐户实例中找到。This can be found in your Azure Storage Account instance. 单击“表服务”部分中的“表” 。Click Tables in the Table Service section.

  • 查询 你可在查询中使用 @StartTimeQuery You can use the @StartTime in your query. @StartTime 在脚本中替换为 yyyy-MM-ddTHH:mm:ss 格式字符串。@StartTime is replaced with a yyyy-MM-ddTHH:mm:ss format string in script.

    let StartDateTime = datetime(@StartTime); let EndDateTime = StartDateTime + 1d; 
    SampleTable | where Timestamp >= StartDateTime and Timestamp < EndDateTime | project Timestamp, Market, RPM
    

ElasticsearchElasticsearch

  • 主机 :指定 Elasticsearch 群集的主控主机。Host :Specify the master host of Elasticsearch Cluster.
  • 端口 :指定 Elasticsearch 群集的主端口。Port :Specify the master port of Elasticsearch Cluster.
  • 授权标头 :指定 Elasticsearch 群集的授权标头值。Authorization Header :Specify the authorization header value of Elasticsearch Cluster.
  • 查询 :指定要获取数据的查询。Query :Specify the query to get data. 支持占位符 @StartTime。(例如,引入数据 2020-06-21T00:00:00Z 时,@StartTime = 2020-06-21T00:00:00)Placeholder @StartTime is supported.(e.g. when data of 2020-06-21T00:00:00Z is ingested, @StartTime = 2020-06-21T00:00:00)

HTTP 请求HTTP request

  • 请求 URL :可返回 JSON 的 HTTP url。Request URL : A HTTP url which can return a JSON. 支持占位符 %Y、%m、%d、%h、%M:%Y = yyyy 格式的年份,%m = MM 格式的月份,%d = dd 格式的日期,%h = HH 格式的小时,%M = mm 格式的分钟。The placeholders %Y,%m,%d,%h,%M are supported: %Y=year in format yyyy, %m=month in format MM, %d=day in format dd, %h=hour in format HH, %M=minute in format mm. 例如:http://microsoft.com/ProjectA/%Y/%m/X_%Y-%m-%d-%h-%MFor example: http://microsoft.com/ProjectA/%Y/%m/X_%Y-%m-%d-%h-%M.
  • 请求 HTTP 方法 :使用 GET 或 POST。Request HTTP method : Use GET or POST.
  • 请求标头 :可添加基本身份验证。Request header : Could add basic authentication.
  • 请求有效负载 :仅支持 JSON 有效负载。Request payload : Only JSON payload is supported. 有效负载中支持 @StartTime 占位符。Placeholder @StartTime is supported in the payload. 响应应为以下 JSON 格式:[{"timestamp":"2018-01-01T00:00:00Z", "market":"en-us", "count":11, "revenue":1.23}, {"timestamp":"2018-01-01T00:00:00Z", "market":"zh-cn", "count":22, "revenue":4.56}]。(例如,引入数据 2020-06-21T00:00:00Z 时,@StartTime = 2020-06-21T00:00:00.0000000+00:00)The response should be in the following JSON format: [{"timestamp": "2018-01-01T00:00:00Z", "market":"en-us", "count":11, "revenue":1.23}, {"timestamp": "2018-01-01T00:00:00Z", "market":"zh-cn", "count":22, "revenue":4.56}].(e.g. when data of 2020-06-21T00:00:00Z is ingested, @StartTime = 2020-06-21T00:00:00.0000000+00:00)

InfluxDB (InfluxQL)InfluxDB (InfluxQL)

  • 连接字符串 :访问 InfluxDB 的连接字符串。Connection String : The connection string to access your InfluxDB.
  • 数据库 :要查询的数据库。Database : The database to query against.
  • 查询 :一个查询,用于获取数据并将数据构建为多维时序数据以进行引入。Query : A query to get and formulate data into multi-dimensional time series data for ingestion.
  • 用户名 :这对于身份验证是可选的。User name : This is optional for authentication.
  • 密码 :这对于身份验证是可选的。Password : This is optional for authentication.

MongoDBMongoDB

  • 连接字符串 :访问 MongoDB 的连接字符串。Connection String : The connection string to access your MongoDB.
  • 数据库 :要查询的数据库。Database : The database to query against.
  • 命令 :一个命令,用于获取数据并将数据构建为多维时序数据以进行引入。Command : A command to get and formulate data into multi-dimensional time series data for ingestion.

MySQLMySQL

  • 连接字符串 :访问 MySQL DB 的连接字符串。Connection String : The connection string to access your MySQL DB.
  • 查询 :一个查询,用于获取数据并将数据构建为多维时序数据以进行引入。Query : A query to get and formulate data into multi-dimensional time series data for ingestion.

PostgreSQLPostgreSQL

  • 连接字符串 :访问 PostgreSQL DB 的连接字符串。Connection String : The connection string to access your PostgreSQL DB.
  • 查询 :一个查询,用于获取数据并将数据构建为多维时序数据以进行引入。Query : A query to get and formulate data into multi-dimensional time series data for ingestion.

后续步骤Next steps