使用 C# 为 Azure 数据资源管理器创建事件中心数据连接Create an Event Hub data connection for Azure Data Explorer by using C#
Azure 数据资源管理器是一项快速且高度可缩放的数据探索服务,适用于日志和遥测数据。Azure Data Explorer is a fast and highly scalable data exploration service for log and telemetry data. Azure 数据资源管理器提供了从事件中心、IoT 中心和写入 blob 容器的 blob 引入数据(数据加载)的功能。Azure Data Explorer offers ingestion (data loading) from Event Hubs, IoT Hubs, and blobs written to blob containers.
在本文中,你将使用 C# 为 Azure 数据资源管理器创建事件中心数据连接。In this article, you create an Event Hub data connection for Azure Data Explorer by using C#.
先决条件Prerequisites
- 如果尚未安装 Visual Studio 2019,可以下载并使用 免费的 Visual Studio 2019 Community Edition。If you don't have Visual Studio 2019 installed, you can download and use the free Visual Studio 2019 Community Edition. 在安装 Visual Studio 的过程中,请确保启用“Azure 开发”。Make sure that you enable Azure development during the Visual Studio setup.
- 如果没有 Azure 订阅,请在开始前创建一个试用订阅。If you don't have an Azure subscription, create a Trial Subscription before you begin.
- 创建群集和数据库Create a cluster and database
- 创建表和列映射Create table and column mapping
- 设置数据库和表策略(可选)Set database and table policies (optional)
- 创建包含要引入的数据的事件中心。Create an Event Hub with data for ingestion.
安装 C# NuGetInstall C# NuGet
- 安装 Microsoft.Azure.Management.Kusto NuGet 包。Install the Microsoft.Azure.Management.Kusto NuGet package.
AuthenticationAuthentication
若要运行以下示例,需要可以访问资源的 Azure Active Directory (Azure AD) 应用程序和服务主体。To run the following example, you need an Azure Active Directory (Azure AD) application and service principal that can access resources. 若要创建免费的 Azure AD 应用程序并在订阅级别添加角色分配,请参阅创建 Azure AD 应用程序。To create a free Azure AD application and add role assignment at the subscription level, see Create an Azure AD application. 还需要目录(租户)ID、应用程序 ID 和客户端密码。You also need the directory (tenant) ID, application ID, and client secret.
添加事件中心数据连接Add an Event Hub data connection
以下示例演示如何以编程方式添加事件中心数据连接。The following example shows you how to add an Event Hub data connection programmatically. 请参阅连接到事件中心,了解如何使用 Azure 门户添加事件中心数据连接。See connect to the event hub for adding an Event Hub data connection using the Azure portal.
var tenantId = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx";//Directory (tenant) ID
var clientId = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx";//Application ID
var clientSecret = "xxxxxxxxxxxxxx";//Client Secret
var subscriptionId = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx";
var authenticationContext = new AuthenticationContext($"https://login.chinacloudapi.cn/{tenantId}");
var credential = new ClientCredential(clientId, clientSecret);
var result = await authenticationContext.AcquireTokenAsync(resource: "https://management.core.chinacloudapi.cn/", clientCredential: credential);
var credentials = new TokenCredentials(result.AccessToken, result.AccessTokenType);
var kustoManagementClient = new KustoManagementClient(credentials)
{
SubscriptionId = subscriptionId
};
var resourceGroupName = "testrg";
//The cluster and database that are created as part of the Prerequisites
var clusterName = "mykustocluster";
var databaseName = "mykustodatabase";
var dataConnectionName = "myeventhubconnect";
//The event hub that is created as part of the Prerequisites
var eventHubResourceId = "/subscriptions/xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx/resourceGroups/xxxxxx/providers/Microsoft.EventHub/namespaces/xxxxxx/eventhubs/xxxxxx";
var consumerGroup = "$Default";
var location = "China East 2";
//The table and column mapping are created as part of the Prerequisites
var tableName = "StormEvents";
var mappingRuleName = "StormEvents_CSV_Mapping";
var dataFormat = DataFormat.CSV;
await kustoManagementClient.DataConnections.CreateOrUpdateAsync(resourceGroupName, clusterName, databaseName, dataConnectionName,
new EventHubDataConnection(eventHubResourceId, consumerGroup, location: location, tableName: tableName, mappingRuleName: mappingRuleName, dataFormat: dataFormat));
设置Setting | 建议的值Suggested value | 字段说明Field description |
---|---|---|
tenantIdtenantId | xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxxxxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx | 租户 ID。Your tenant ID. 也称为目录 ID。Also known as directory ID. |
subscriptionIdsubscriptionId | xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxxxxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx | 用于创建资源的订阅 ID。The subscription ID that you use for resource creation. |
clientIdclientId | xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxxxxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx | 可以访问租户中资源的应用程序的客户端 ID。The client ID of the application that can access resources in your tenant. |
clientSecretclientSecret | xxxxxxxxxxxxxxxxxxxxxxxxxxxx | 可以访问租户中资源的应用程序的客户端密码。The client secret of the application that can access resources in your tenant. |
resourceGroupNameresourceGroupName | testrgtestrg | 包含群集的资源组的名称。The name of the resource group containing your cluster. |
clusterNameclusterName | mykustoclustermykustocluster | 群集的名称。The name of your cluster. |
databaseNamedatabaseName | mykustodatabasemykustodatabase | 群集中目标数据库的名称。The name of the target database in your cluster. |
dataConnectionNamedataConnectionName | myeventhubconnectmyeventhubconnect | 所需的数据连接名称。The desired name of your data connection. |
tableNametableName | StormEventsStormEvents | 目标数据库中目标表的名称。The name of the target table in the target database. |
mappingRuleNamemappingRuleName | StormEvents_CSV_MappingStormEvents_CSV_Mapping | 与目标表相关的列映射的名称。The name of your column mapping related to the target table. |
dataFormatdataFormat | csvcsv | 消息的数据格式。The data format of the message. |
eventHubResourceIdeventHubResourceId | 资源 IDResource ID | 包含要引入的数据的事件中心的资源 ID。The resource ID of your Event Hub that holds the data for ingestion. |
consumerGroupconsumerGroup | $Default$Default | 事件中心的使用者组。The consumer group of your Event Hub. |
locationlocation | 中国东部 2China East 2 | 数据连接资源的位置。The location of the data connection resource. |
生成数据Generate data
请参阅可生成数据并将其发送到事件中心的示例应用。See the sample app that generates data and sends it to an event hub.
事件可以包含一个或多个记录(直到达到事件的大小限制)。An event can contain one or more records, up to its size limit. 在下面的示例中,我们将发送两个事件,每个事件有五个追加的记录:In the following sample we send two events, each has five records appended:
var events = new List<EventData>();
var data = string.Empty;
var recordsPerEvent = 5;
var rand = new Random();
var counter = 0;
for (var i = 0; i < 10; i++)
{
// Create the data
var metric = new Metric { Timestamp = DateTime.UtcNow, MetricName = "Temperature", Value = rand.Next(-30, 50) };
var data += JsonConvert.SerializeObject(metric) + Environment.NewLine;
counter++;
// Create the event
if (counter == recordsPerEvent)
{
var eventData = new EventData(Encoding.UTF8.GetBytes(data));
events.Add(eventData);
counter = 0;
data = string.Empty;
}
}
// Send events
eventHubClient.SendAsync(events).Wait();
清理资源Clean up resources
若要删除数据连接,请使用以下命令:To delete the data connection, use the following command:
kustoManagementClient.DataConnections.Delete(resourceGroupName, clusterName, databaseName, dataConnectionName);