使用 Go 创建 Azure 数据资源管理器群集和数据库Create an Azure Data Explorer cluster and database using Go

Azure 数据资源管理器是一项快速、完全托管的数据分析服务,用于实时分析从应用程序、网站和 IoT 设备等资源流式传输的海量数据。Azure Data Explorer is a fast, fully managed data analytics service for real-time analysis on large volumes of data streaming from applications, websites, IoT devices, and more. 若要使用 Azure 数据资源管理器,请先创建群集,再在该群集中创建一个或多个数据库。To use Azure Data Explorer, you first create a cluster, and create one or more databases in that cluster. 然后将数据引入(加载)到数据库,以便对其运行查询。Then you ingest (load) data into a database so that you can run queries against it.

本文将使用 Go 创建 Azure 数据资源管理器群集和数据库。In this article, you create an Azure Data Explorer cluster and database using Go. 然后,你可以列出和删除新的群集和数据库,并对资源执行操作。You can then list and delete your new cluster and database and execute operations on your resources.

先决条件Prerequisites

  • 如果没有 Azure 订阅,请在开始前创建一个试用订阅If you don't have an Azure subscription, create a Trial Subscription before you begin.
  • 安装 GitInstall Git.
  • 安装合适版本的 Go。Install an appropriate version of Go. 有关支持的版本的详细信息,请参阅 Azure Go SDKFor more information regarding supported releases, see the Azure Go SDK.

查看代码Review the code

本部分是可选的。This section is optional. 如果有兴趣了解代码如何工作,可以查看以下代码片段。If you're interested to learn how the code works, you can review the following code snippets. 否则,可以跳到运行应用程序Otherwise, you can skip ahead to Run the application.

身份验证Authentication

在执行任何操作之前,程序都需要向 Azure 数据资源管理器进行身份验证。The program needs to authenticate to Azure Data Explorer before executing any operations. 客户端凭据身份验证类型auth.NewAuthorizerFromEnvironment 使用,后者会查找以下预定义的环境变量:AZURE_CLIENT_IDAZURE_CLIENT_SECRETAZURE_TENANT_IDThe Client credentials authentication type is used by auth.NewAuthorizerFromEnvironment that looks for the following pre-defined environment variables: AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, AZURE_TENANT_ID.

以下示例展示了如何使用此方法创建 kusto.ClustersClientThe following example shows how a kusto.ClustersClient is created using this technique:

func getClustersClient(subscription string) kusto.ClustersClient {
    client := kusto.NewClustersClient(subscription)
    authR, err := auth.NewAuthorizerFromEnvironment()
    if err != nil {
        log.Fatal(err)
    }
    client.Authorizer = authR

    return client
}

提示

如果你为身份验证安装并配置了 Azure CLI,请使用 auth.NewAuthorizerFromCLIWithResource 函数进行本地开发。Use the auth.NewAuthorizerFromCLIWithResource function for local development if you have Azure CLI installed and configured for authentication.

创建群集Create cluster

kusto.ClustersClient 使用 CreateOrUpdate 函数来创建新的 Azure 数据资源管理器群集。Use the CreateOrUpdate function on kusto.ClustersClient to create a new Azure Data Explorer cluster. 等待该过程完成,然后检查结果。Wait for the process to complete before inspecting the results.

func createCluster(sub, name, location, rgName string) {
    ...
    result, err := client.CreateOrUpdate(ctx, rgName, name, kusto.Cluster{Location: &location, Sku: &kusto.AzureSku{Name: kusto.DevNoSLAStandardD11V2, Capacity: &numInstances, Tier: kusto.Basic}})
    ...
    err = result.WaitForCompletionRef(context.Background(), client.Client)
    ...
    r, err := result.Result(client)
}

列出群集List clusters

kusto.ClustersClient 使用 ListByResourceGroup 函数来获取 kusto.ClusterListResult,后者将进行迭代以便以表格式显示输出。Use the ListByResourceGroup function on kusto.ClustersClient to get a kusto.ClusterListResult that is then iterated to show the output in a tabular format.

func listClusters(sub, rgName string) {
    ...
    result, err := getClustersClient(sub).ListByResourceGroup(ctx, rgName)
    ...
    for _, c := range *result.Value {
        // setup tabular representation
    }
    ...
}

创建数据库Create database

kusto.DatabasesClient 使用 CreateOrUpdate 函数,以在现有群集中创建新的 Azure 数据资源管理器数据库。Use CreateOrUpdate function on kusto.DatabasesClient to create a new Azure Data Explorer database in an existing cluster. 等待该过程完成,然后检查结果。Wait for the process to complete before inspecting the results.

func createDatabase(sub, rgName, clusterName, location, dbName string) {
    future, err := client.CreateOrUpdate(ctx, rgName, clusterName, dbName, kusto.ReadWriteDatabase{Kind: kusto.KindReadWrite, Location: &location})
    ...
    err = future.WaitForCompletionRef(context.Background(), client.Client)
    ...
    r, err := future.Result(client)
    ...
}

列出数据库List databases

kusto.DatabasesClient 使用 ListByCluster 函数以获取 kusto.DatabaseListResult,后者将进行迭代以便以表格格式显示输出。Use ListByCluster function on kusto.DatabasesClient to get kusto.DatabaseListResult that is then iterated to show the output in a tabular format.

func listDatabases(sub, rgName, clusterName string) {
    result, err := getDBClient(sub).ListByCluster(ctx, rgName, clusterName)
    ...
    for _, db := range *result.Value {
        // setup tabular representation
    }
    ...
}

删除数据库Delete database

kusto.DatabasesClient 使用 Delete 函数以删除群集中的现有数据库。Use Delete function on a kusto.DatabasesClient to delete an existing database in a cluster. 等待该过程完成,然后检查结果。Wait for the process to complete before inspecting the results.

func deleteDatabase(sub, rgName, clusterName, dbName string) {
    ...
    future, err := getDBClient(sub).Delete(ctx, rgName, clusterName, dbName)
    ...
    err = future.WaitForCompletionRef(context.Background(), client.Client)
    ...
    r, err := future.Result(client)
    if r.StatusCode == 200 {
        // determine success or failure
    }
    ...
}

删除群集Delete cluster

kusto.ClustersClient 使用 Delete 函数以删除群集。Use Delete function on a kusto.ClustersClient to delete a cluster. 等待该过程完成,然后检查结果。Wait for the process to complete before inspecting the results.

func deleteCluster(sub, clusterName, rgName string) {
    result, err := client.Delete(ctx, rgName, clusterName)
    ...
    err = result.WaitForCompletionRef(context.Background(), client.Client)
    ...
    r, err := result.Result(client)
    if r.StatusCode == 200 {
        // determine success or failure
    }
    ...
}

运行此应用程序Run the application

按原样运行示例代码时,将执行以下操作:When you run the sample code as is, the following actions are performed:

  1. 创建一个 Azure 数据资源管理器群集。An Azure Data Explorer cluster is created.

  2. 列出指定资源组中的所有 Azure 数据资源管理器群集。All the Azure Data Explorer clusters in the specified resource group are listed.

  3. 在现有群集中创建一个 Azure 数据资源管理器数据库。An Azure Data Explorer database is created in the existing cluster.

  4. 列出指定群集中的所有数据库。All the databases in the specified cluster are listed.

  5. 删除数据库。The database is deleted.

  6. 删除群集。The cluster is deleted.

    func main() {
        createCluster(subscription, clusterNamePrefix+clusterName, location, rgName)
        listClusters(subscription, rgName)
        createDatabase(subscription, rgName, clusterNamePrefix+clusterName, location, dbNamePrefix+databaseName)
        listDatabases(subscription, rgName, clusterNamePrefix+clusterName)
        deleteDatabase(subscription, rgName, clusterNamePrefix+clusterName, dbNamePrefix+databaseName)
        deleteCluster(subscription, clusterNamePrefix+clusterName, rgName)
    }
    

    提示

    若要尝试不同的操作组合,可以根据需要在 main.go 中取消注释和注释相应的函数。To try different combinations of operations, you can uncomment and comment the respective functions in main.go as needed.

  7. 从 GitHub 克隆示例代码:Clone the sample code from GitHub:

    git clone https://github.com/Azure-Samples/azure-data-explorer-go-cluster-management.git
    cd azure-data-explorer-go-cluster-management
    
  8. 程序使用客户端凭据进行身份验证。The program authenticates using client credentials. 使用 Azure CLI az ad sp create-for-rbac 命令创建服务主体。Use the Azure CLI az ad sp create-for-rbac command to create a service principal. 保存客户端 ID、客户端密码和租户 ID 信息,以便在下一步中使用。Save the client ID, client secret, and tenant ID information for use in the next step.

  9. 导出所需的环境变量,包括服务主体信息。Export required environment variables including service principal information. 输入要在其中创建群集的订阅 ID、资源组和区域。Enter your subscription ID, resource group, and region where you want to create the cluster.

    export AZURE_CLIENT_ID="<enter service principal client ID>"
    export AZURE_CLIENT_SECRET="<enter service principal client secret>"
    export AZURE_TENANT_ID="<enter tenant ID>"
    
    export SUBSCRIPTION="<enter subscription ID>"
    export RESOURCE_GROUP="<enter resource group name>"
    export LOCATION="<enter azure location e.g. Southeast Asia>"
    
    export CLUSTER_NAME_PREFIX="<enter prefix (cluster name will be [prefix]-ADXTestCluster)>"
    export DATABASE_NAME_PREFIX="<enter prefix (database name will be [prefix]-ADXTestDB)>"
    

    提示

    在生产方案中,你可能会使用环境变量向应用程序提供凭据。You're likely to use environment variables in production scenarios to provide credentials to your application. 查看代码中所述,对于本地开发,如果你为身份验证安装并配置了 Azure CLI,请使用 auth.NewAuthorizerFromCLIWithResourceAs mentioned in Review the code, for local development, use auth.NewAuthorizerFromCLIWithResource if you have Azure CLI installed and configured for authentication. 在这种情况下,不需要创建服务主体。In that situation, you don't need to create a service principal.

  10. 运行该程序:Run the program:

    go run main.go
    

    你会看到类似于以下内容的输出:You'll get an output similar to the following:

    waiting for cluster creation to complete - fooADXTestCluster
    created cluster fooADXTestCluster
    listing clusters in resource group <your resource group>
    +-------------------+---------+----------------+-----------+-------------------------------------------------------------+
    |       NAME        |  STATE  |    LOCATION    | INSTANCES |                            URI                              |
    +-------------------+---------+----------------+-----------+-------------------------------------------------------------+
    | fooADXTestCluster | Running | China East 2   |         1 | https://fooADXTestCluster.chinaeast2.kusto.chinacloudapi.cn |
    +-------------------+---------+----------------+-----------+-------------------------------------------------------------+
    
    waiting for database creation to complete - barADXTestDB
    created DB fooADXTestCluster/barADXTestDB with ID /subscriptions/<your subscription ID>/resourceGroups/<your resource group>/providers/Microsoft.Kusto/Clusters/fooADXTestCluster/Databases/barADXTestDB and type Microsoft.Kusto/Clusters/Databases
    
    listing databases in cluster fooADXTestCluster
    +--------------------------------+-----------+----------------+------------------------------------+
    |              NAME              |   STATE   |    LOCATION    |                TYPE                |
    +--------------------------------+-----------+----------------+------------------------------------+
    | fooADXTestCluster/barADXTestDB | Succeeded |  China East 2  | Microsoft.Kusto/Clusters/Databases |
    +--------------------------------+-----------+----------------+------------------------------------+
    
    waiting for database deletion to complete - barADXTestDB
    deleted DB barADXTestDB from cluster fooADXTestCluster
    
    waiting for cluster deletion to complete - fooADXTestCluster
    deleted ADX cluster fooADXTestCluster from resource group <your resource group>
    

清理资源Clean up resources

如果未使用本文中的示例代码以编程方式删除群集,请使用 Azure CLI 手动删除它们。If you didn't delete the cluster programmatically using the sample code in this article, delete them manually using Azure CLI.

后续步骤Next steps

使用 Azure 数据资源管理器 Go SDK 引入数据Ingest data using the Azure Data Explorer Go SDK