在 Azure 数据资源管理器中使用后继数据库来附加数据库Use follower database to attach databases in Azure Data Explorer

使用后继数据库功能可将另一群集中的数据库附加到 Azure 数据资源管理器群集。The follower database feature allows you to attach a database located in a different cluster to your Azure Data Explorer cluster. 后继数据库以只读模式附加,因此可以查看其数据,并针对已引入先导数据库的数据运行查询。The follower database is attached in read-only mode, making it possible to view the data and run queries on the data that was ingested into the leader database. 后继数据库会同步先导数据库中的更改。The follower database synchronizes changes in the leader databases. 由于同步,需要经过几秒到几分钟的延迟之后,才会提供数据。Due to the synchronization, there's a data lag of a few seconds to a few minutes in data availability. 具体的延迟时长取决于先导数据库元数据的总体大小。The length of the time lag depends on the overall size of the leader database metadata. 先导数据库和后继数据库使用相同的存储帐户来提取数据。The leader and follower databases use the same storage account to fetch the data. 存储由先导数据库拥有。The storage is owned by the leader database. 后继数据库无需引入数据即可查看数据。The follower database views the data without needing to ingest it. 由于附加的数据库是只读的数据库,因此无法修改数据库中除缓存策略主体权限以外的其他数据、表和策略。Since the attached database is a read-only database, the data, tables, and policies in the database can't be modified except for caching policy, principals, and permissions. 无法删除附加的数据库。Attached databases can't be deleted. 这些数据库必须先由先导数据库或后继数据库分离,然后才能将其删除。They must be detached by the leader or follower and only then they can be deleted.

使用后继功能将数据库附加到不同群集是在组织与团队之间共享数据的基础结构。Attaching a database to a different cluster using the follower capability is used as the infrastructure to share data between organizations and teams. 此功能可用于隔离计算资源,以防止将生产环境用于非生产用例。The feature is useful to segregate compute resources to protect a production environment from non-production use cases. 后继功能还可用于将 Azure 数据资源管理器群集的成本关联到对数据运行查询的一方。Follower can also be used to associate the cost of Azure Data Explorer cluster to the party that runs queries on the data.

哪些数据库是后继数据库?Which databases are followed?

  • 一个群集可以后继先导群集中的一个数据库、多个数据库或所有数据库。A cluster can follow one database, several databases, or all databases of a leader cluster.
  • 单个群集可以后继多个先导群集中的数据库。A single cluster can follow databases from multiple leader clusters.
  • 一个群集可以同时包含后继数据库和先导数据库A cluster can contain both follower databases and leader databases

先决条件Prerequisites

  1. 如果没有 Azure 订阅,请在开始前创建一个试用帐户If you don't have an Azure subscription, create a trial account before you begin.
  2. 为先导和后继数据库创建群集和数据库Create cluster and DB for the leader and follower.
  3. 使用引入概述中所述的多种方法之一将数据引入先导数据库。Ingest data to leader database using one of various methods discussed in ingestion overview.

附加数据库Attach a database

可以使用多种方法来附加数据库。There are various methods you can use to attach a database. 本文介绍如何使用 C#、Python 或 Azure 资源管理器模板附加数据库。In this article, we discuss attaching a database using C#, Python or an Azure Resource Manager template. 若要附加数据库,必须在先导群集和后继群集上拥有至少具有参与者角色的用户、组、服务主体或托管标识。To attach a database, you must have user, group, service principal, or managed identity with at least contributor role on the leader cluster and the follower cluster. 可以使用 Azure 门户PowerShellAzure CLIARM 模板添加或删除角色分配。You can add or remove role assignments using Azure Portal, PowerShell, Azure CLI and ARM template. 可以深入了解 Azure 基于角色的访问控制 (Azure RBAC)不同角色You can learn more about Azure role-based access control (Azure RBAC) and the different roles.

使用 C# 附加数据库Attach a database using C#

所需的 NuGetNeeded NuGets

代码示例Code Example

var tenantId = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx";//Directory (tenant) ID
var clientId = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx";//Application ID
var clientSecret = "xxxxxxxxxxxxxx";//Client secret
var leaderSubscriptionId = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx";
var followerSubscriptionId = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx";

var serviceCreds = await ApplicationTokenProvider.LoginSilentAsync(tenantId, clientId, clientSecret);
var resourceManagementClient = new KustoManagementClient(serviceCreds){
    SubscriptionId = followerSubscriptionId
};

var followerResourceGroupName = "followerResouceGroup";
var leaderResourceGroup = "leaderResouceGroup";
var leaderClusterName = "leader";
var followerClusterName = "follower";
var attachedDatabaseConfigurationName = "uniqueNameForAttachedDatabaseConfiguration";
var databaseName = "db"; // Can be specific database name or * for all databases
var defaultPrincipalsModificationKind = "Union"; 
var location = "China East 2";

AttachedDatabaseConfiguration attachedDatabaseConfigurationProperties = new AttachedDatabaseConfiguration()
{
    ClusterResourceId = $"/subscriptions/{leaderSubscriptionId}/resourceGroups/{leaderResourceGroup}/providers/Microsoft.Kusto/Clusters/{leaderClusterName}",
    DatabaseName = databaseName,
    DefaultPrincipalsModificationKind = defaultPrincipalsModificationKind,
    Location = location
};

var attachedDatabaseConfigurations = resourceManagementClient.AttachedDatabaseConfigurations.CreateOrUpdate(followerResourceGroupName, followerClusterName, attachedDatabaseConfigurationName, attachedDatabaseConfigurationProperties);

使用 Python 附加数据库Attach a database using Python

所需的模块Needed Modules

pip install azure-common
pip install azure-mgmt-kusto

代码示例Code Example

from azure.mgmt.kusto import KustoManagementClient
from azure.mgmt.kusto.models import AttachedDatabaseConfiguration
from azure.common.credentials import ServicePrincipalCredentials
import datetime

#Directory (tenant) ID
tenant_id = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx"
#Application ID
client_id = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx"
#Client Secret
client_secret = "xxxxxxxxxxxxxx"
follower_subscription_id = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx"
leader_subscription_id = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx"
credentials = ServicePrincipalCredentials(
        client_id=client_id,
        secret=client_secret,
        tenant=tenant_id
    )
kusto_management_client = KustoManagementClient(credentials, follower_subscription_id)

follower_resource_group_name = "followerResouceGroup"
leader_resouce_group_name = "leaderResouceGroup"
follower_cluster_name = "follower"
leader_cluster_name = "leader"
attached_database_Configuration_name = "uniqueNameForAttachedDatabaseConfiguration"
database_name  = "db" # Can be specific database name or * for all databases
default_principals_modification_kind  = "Union"
location = "China East 2"
cluster_resource_id = "/subscriptions/" + leader_subscription_id + "/resourceGroups/" + leader_resouce_group_name + "/providers/Microsoft.Kusto/Clusters/" + leader_cluster_name

attached_database_configuration_properties = AttachedDatabaseConfiguration(cluster_resource_id = cluster_resource_id, database_name = database_name, default_principals_modification_kind = default_principals_modification_kind, location = location)

#Returns an instance of LROPoller, see https://docs.microsoft.com/python/api/msrest/msrest.polling.lropoller?view=azure-python
poller = kusto_management_client.attached_database_configurations.create_or_update(follower_resource_group_name, follower_cluster_name, attached_database_Configuration_name, attached_database_configuration_properties)

使用 Azure 资源管理器模板附加数据库Attach a database using an Azure Resource Manager template

本部分介绍如何使用 Azure 资源管理器模板将数据库附加到现有群集。In this section, you learn to attach a database to an existing cluser by using an Azure Resource Manager template.

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "followerClusterName": {
            "type": "string",
            "defaultValue": "",
            "metadata": {
                "description": "Name of the cluster to which the database will be attached."
            }
        },
        "attachedDatabaseConfigurationsName": {
            "type": "string",
            "defaultValue": "",
            "metadata": {
                "description": "Name of the attached database configurations to create."
            }
        },
        "databaseName": {
            "type": "string",
            "defaultValue": "",
            "metadata": {
                "description": "The name of the database to follow. You can follow all databases by using '*'."
            }
        },
        "leaderClusterResourceId": {
            "type": "string",
            "defaultValue": "",
            "metadata": {
                "description": "The resource ID of the leader cluster."
            }
        },
        "defaultPrincipalsModificationKind": {
            "type": "string",
            "defaultValue": "Union",
            "metadata": {
                "description": "The default principal modification kind."
            }
        },
        "location": {
            "type": "string",
            "defaultValue": "",
            "metadata": {
                "description": "Location for all resources."
            }
        }
    },
    "variables": {},
    "resources": [
        {
            "name": "[concat(parameters('followerClusterName'), '/', parameters('attachedDatabaseConfigurationsName'))]",
            "type": "Microsoft.Kusto/clusters/attachedDatabaseConfigurations",
            "apiVersion": "2020-02-15",
            "location": "[parameters('location')]",
            "properties": {
                "databaseName": "[parameters('databaseName')]",
                "clusterResourceId": "[parameters('leaderClusterResourceId')]",
                "defaultPrincipalsModificationKind": "[parameters('defaultPrincipalsModificationKind')]"
            }
        }
    ]
}

部署模板Deploy the template

可以使用 Azure 门户或 PowerShell 部署 Azure 资源管理器模板。You can deploy the Azure Resource Manager template by using the Azure portal or using powershell.

模板部署

设置Setting 说明Description
后继群集名称Follower Cluster Name 模板将部署到的后继群集的名称。The name of the follower cluster; where the template will be deployed.
附加的数据库配置名称Attached Database Configurations Name 附加的数据库配置对象的名称。The name of the attached database configurations object. 该名称可以是在群集级别唯一的任何字符串。The name can be any string that is unique at the cluster level.
数据库名称Database Name 要后继的数据库的名称。The name of the database to be followed. 若要后继所有先导数据库,请使用“*”。If you want to follow all the leader's databases, use '*'.
先导群集资源 IDLeader Cluster Resource ID 先导群集的资源 ID。The resource ID of the leader cluster.
默认主体修改类型Default Principals Modification Kind 默认的主体修改类型。The default principal modification kind. 可以是 UnionReplaceNoneCan be Union, Replace or None. 有关默认主体修改类型的详细信息,请参阅主体修改类型控制命令For more information about default principal modification kind, see principal modification kind control command.
位置Location 所有资源的位置。The location of all the resources. 先导和后继数据库必须位于同一位置。The leader and the follower must be in the same location.

验证是否已成功附加数据库Verify that the database was successfully attached

若要验证是否已成功附加数据库,请在 Azure 门户中找到附加的数据库。To verify that the database was successfully attached, find your attached databases in the Azure portal.

  1. 导航到后继群集并选择“数据库”Navigate to the follower cluster and select Databases

  2. 在数据库列表中搜索新的只读数据库。Search for new Read-only databases in the database list.

    只读的后继数据库

也可使用以下命令:Alternatively:

  1. 导航到先导群集并选择“数据库”Navigate to the leader cluster and select Databases

  2. 检查相关数据库是否标记为“与其他人共享” > “是” Check that the relevant databases are marked as SHARED WITH OTHERS > Yes

    读取和写入附加的数据库

使用 C# 分离后继数据库Detach the follower database using C#

从后继群集中分离已附加的后继数据库Detach the attached follower database from the follower cluster

后继群集可按如下所示拆离任何附加的数据库:The follower cluster can detach any attached database as follows:

var tenantId = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx";//Directory (tenant) ID
var clientId = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx";//Application ID
var clientSecret = "xxxxxxxxxxxxxx";//Client secret
var leaderSubscriptionId = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx";
var followerSubscriptionId = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx";

var serviceCreds = await ApplicationTokenProvider.LoginSilentAsync(tenantId, clientId, clientSecret);
var resourceManagementClient = new KustoManagementClient(serviceCreds){
    SubscriptionId = followerSubscriptionId
};

var followerResourceGroupName = "testrg";
//The cluster and database that are created as part of the prerequisites
var followerClusterName = "follower";
var attachedDatabaseConfigurationsName = "uniqueName";

resourceManagementClient.AttachedDatabaseConfigurations.Delete(followerResourceGroupName, followerClusterName, attachedDatabaseConfigurationsName);

若要从从后继方拆离数据库,必须在后继群集上拥有至少具有参与者角色的用户、组、服务主体或托管标识。To detach a database from the follower side, you must have user, group, service principal, or managed identity with at least contributor role on the follower cluster. 在上面的示例中,我们使用服务主体。In the example above we use service principal.

从先导群集中分离已附加的后继数据库Detach the attached follower database from the leader cluster

先导群集可按如下所示分离任何附加的数据库:The leader cluster can detach any attached database as follows:

var tenantId = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx";//Directory (tenant) ID
var clientId = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx";//Application ID
var clientSecret = "xxxxxxxxxxxxxx";//Client secret
var leaderSubscriptionId = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx";
var followerSubscriptionId = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx";

var serviceCreds = await ApplicationTokenProvider.LoginSilentAsync(tenantId, clientId, clientSecret);
var resourceManagementClient = new KustoManagementClient(serviceCreds){
    SubscriptionId = leaderSubscriptionId
};

var leaderResourceGroupName = "testrg";
var followerResourceGroupName = "followerResouceGroup";
var leaderClusterName = "leader";
var followerClusterName = "follower";
//The cluster and database that are created as part of the Prerequisites
var followerDatabaseDefinition = new FollowerDatabaseDefinition()
    {
        AttachedDatabaseConfigurationName = "uniqueName",
        ClusterResourceId = $"/subscriptions/{followerSubscriptionId}/resourceGroups/{followerResourceGroupName}/providers/Microsoft.Kusto/Clusters/{followerClusterName}"
    };

resourceManagementClient.Clusters.DetachFollowerDatabases(leaderResourceGroupName, leaderClusterName, followerDatabaseDefinition);

若要从从先导方拆离数据库,必须在先导群集上拥有至少具有参与者角色的用户、组、服务主体或托管标识。To detach a database from the leader side, you must have user, group, service principal, or managed identity with at least contributor role on the leader cluster. 在上面的示例中,我们使用服务主体。In the example above we use service principal.

使用 Python 拆离后继数据库Detach the follower database using Python

从后继群集中分离已附加的后继数据库Detach the attached follower database from the follower cluster

后继群集可按如下所示拆离任何附加的数据库:The follower cluster can detach any attached database as follows:

from azure.mgmt.kusto import KustoManagementClient
from azure.common.credentials import ServicePrincipalCredentials
import datetime

#Directory (tenant) ID
tenant_id = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx"
#Application ID
client_id = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx"
#Client Secret
client_secret = "xxxxxxxxxxxxxx"
follower_subscription_id = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx"
credentials = ServicePrincipalCredentials(
        client_id=client_id,
        secret=client_secret,
        tenant=tenant_id
    )
kusto_management_client = KustoManagementClient(credentials, follower_subscription_id)

follower_resource_group_name = "followerResouceGroup"
follower_cluster_name = "follower"
attached_database_configurationName = "uniqueName"

#Returns an instance of LROPoller, see https://docs.microsoft.com/python/api/msrest/msrest.polling.lropoller?view=azure-python
poller = kusto_management_client.attached_database_configurations.delete(follower_resource_group_name, follower_cluster_name, attached_database_configurationName)

若要从从后继方拆离数据库,必须在后继群集上拥有至少具有参与者角色的用户、组、服务主体或托管标识。To detach a database from the follower side, you must have user, group, service principal, or managed identity with at least contributor role on the follower cluster. 在上面的示例中,我们使用服务主体。In the example above we use service principal.

从先导群集中分离已附加的后继数据库Detach the attached follower database from the leader cluster

先导群集可按如下所示分离任何附加的数据库:The leader cluster can detach any attached database as follows:


from azure.mgmt.kusto import KustoManagementClient
from azure.mgmt.kusto.models import FollowerDatabaseDefinition
from azure.common.credentials import ServicePrincipalCredentials
import datetime

#Directory (tenant) ID
tenant_id = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx"
#Application ID
client_id = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx"
#Client Secret
client_secret = "xxxxxxxxxxxxxx"
follower_subscription_id = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx"
leader_subscription_id = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx"
credentials = ServicePrincipalCredentials(
        client_id=client_id,
        secret=client_secret,
        tenant=tenant_id
    )
kusto_management_client = KustoManagementClient(credentials, follower_subscription_id)

follower_resource_group_name = "followerResourceGroup"
leader_resource_group_name = "leaderResourceGroup"
follower_cluster_name = "follower"
leader_cluster_name = "leader"
attached_database_configuration_name = "uniqueName"
location = "China East 2"
cluster_resource_id = "/subscriptions/" + follower_subscription_id + "/resourceGroups/" + follower_resource_group_name + "/providers/Microsoft.Kusto/Clusters/" + follower_cluster_name


#Returns an instance of LROPoller, see https://docs.microsoft.com/python/api/msrest/msrest.polling.lropoller?view=azure-python
poller = kusto_management_client.clusters.detach_follower_databases(resource_group_name = leader_resource_group_name, cluster_name = leader_cluster_name, cluster_resource_id = cluster_resource_id, attached_database_configuration_name = attached_database_configuration_name)

若要从从先导方拆离数据库,必须在先导群集上拥有至少具有参与者角色的用户、组、服务主体或托管标识。To detach a database from the leader side, you must have user, group, service principal, or managed identity with at least contributor role on the leader cluster. 在上面的示例中,我们使用服务主体。In the example above we use service principal.

管理主体、权限和缓存策略Manage principals, permissions, and caching policy

管理主体Manage principals

附加数据库时,请指定“默认主体修改类型”。When attaching a database, specify the "default principals modification kind". 默认设置会保留已授权主体的先导数据库集合The default is keeping the leader database collection of authorized principals

种类Kind 说明Description
联合Union 附加的数据库主体始终包括原始数据库主体,以及添加到后继数据库的其他新主体。The attached database principals will always include the original database principals plus additional new principals added to the follower database.
Replace 不会从原始数据库继承主体。No inheritance of principals from the original database. 必须为附加的数据库创建新主体。New principals must be created for the attached database.
None 附加的数据库主体只包括原始数据库的主体,而不包括其他主体。The attached database principals include only the principals of the original database with no additional principals.

有关使用控制命令配置已授权主体的详细信息,请参阅用于管理后继群集的控制命令For more information about using control commands to configure the authorized principals, see Control commands for managing a follower cluster.

管理权限Manage permissions

所有只读数据库类型的权限管理方式都是相同的。Managing read-only database permission is the same as for all database types. 请参阅在 Azure 门户中管理权限See manage permissions in the Azure portal.

配置缓存策略Configure caching policy

后继数据库管理员可以修改附加数据库或者该数据库在托管群集上的任何表的缓存策略The follower database administrator can modify the caching policy of the attached database or any of its tables on the hosting cluster. 默认设置是保留数据库和表级缓存策略的先导数据库集合。The default is keeping the leader database collection of database and table-level caching policies. 例如,可对先导数据库使用一个 30 天缓存策略以运行每月报告,并对后继数据库使用一个 3 天缓存策略,以仅查询最近的数据进行故障排除。You can, for example, have a 30 day caching policy on the leader database for running monthly reporting and a three day caching policy on the follower database to query only the recent data for troubleshooting. 有关使用控制命令对后继数据库或表配置缓存策略的详细信息,请参阅用于管理后继群集的控制命令For more information about using control commands to configure the caching policy on the follower database or table, see Control commands for managing a follower cluster.

限制Limitations

  • 后继和先导群集必须位于同一区域。The follower and the leader clusters must be in the same region.
  • 流式引入不能用于先导数据库。Streaming ingestion can't be used on a database that is being followed.
  • 在分离已附加到其他群集的数据库之前,无法删除该数据库。You can't delete a database that is attached to a different cluster before detaching it.
  • 在分离包含已附加到其他群集的数据库的群集之前,无法删除该群集。You can't delete a cluster that has a database attached to a different cluster before detaching it.

后续步骤Next steps