在 Azure 数据资源管理器群集上启用数据清除Enable data purge on your Azure Data Explorer cluster

备注

本文介绍如何删除设备或服务中的个人数据,并且可为 GDPR 下的任务提供支持。This article provides steps for how to delete personal data from the device or service and can be used to support your obligations under the GDPR. 如果正在寻找有关 GDPR 的一般信息,请参阅服务信任门户的 GDPR 部分If you’re looking for general information about GDPR, see the GDPR section of the Service Trust portal.

Azure 数据资源管理器支持删除单个记录。Azure Data Explorer supports the ability to delete individual records. 通过 .purge 命令删除数据可保护个人数据,且不应在其他情况下使用。Data deletion through the .purge command protects personal data and shouldn't be used in other scenarios. 它不是为支持频繁的删除请求或删除大量数据而设计的,并且可能会对服务的性能产生重大影响。It isn't designed to support frequent delete requests, or deletion of massive quantities of data, and may have a significant performance impact on the service.

执行 .purge 命令会触发一个可能需要几天时间才能完成的过程。Executing a .purge command triggers a process that can take a few days to complete. 如果应用了 predicate 的记录的“密度”很大,该过程会重新引入表中的所有数据。If the "density" of records on which the predicate is applied is large, the process will reingest all the data in the table. 此过程会极大影响性能和 COGS。This process has a significant impact on performance and COGS. 有关详细信息,请参阅 Azure 数据资源管理器中的数据清除For more information, see Data purge in Azure Data Explorer.

清除操作调用方法Methods of invoking purge operations

Azure 数据资源管理器 (Kusto) 支持删除单个记录和清除整个表。Azure Data Explorer (Kusto) supports both individual record deletion and purging an entire table. 根据使用方案的不同,.purge 命令可以通过两种方式调用:The .purge command can be invoked in two ways for differing usage scenarios:

  • 编程调用:要由应用程序调用的单个步骤。Programmatic invocation: A single step that is intended to be invoked by applications. 调用此命令将直接触发清除执行序列。Calling this command directly triggers the purge execution sequence.

  • 人为调用:一个两步过程,需要显式确认为单个步骤。Human invocation: A two-step process that requires an explicit confirmation as a separate step. 调用该命令将返回验证令牌,需要提供该令牌才能运行实际清除。Invocation of the command returns a verification token, which should be provided to run the actual purge. 此过程降低了无意中删除错误数据的风险。This process reduces the risk of inadvertently deleting incorrect data. 对于含有重要冷缓存数据的大型表,使用此选项可能需要很长时间才能完成相应过程。Using this option may take a long time to complete on large tables with significant cold cache data.

先决条件Prerequisites

启用群集上的数据清除Enable data purge on your cluster

警告

  • 启用数据清除需要重启服务,这可能会导致查询删除发生。Enabling data purge requires service restart that may result in query drop.
  • 启用数据清除之前,请查看限制Please review the limitations prior to enabling data purge.
  1. 在 Azure 门户中,转到 Azure 数据资源管理器群集。In the Azure portal, go to your Azure Data Explorer cluster.

  2. 在“设置”中选择“配置”。**** ****In Settings, select Configurations.

  3. 在“配置”窗格中,选择“打开”以启用“启用清除”**** **** ****。In the Configurations pane, select On to enable Enable Purge.

  4. 选择“保存” ****。Select Save.

    启用清除

禁用群集上的数据清除Disable data purge on your cluster

  1. 在 Azure 门户中,转到 Azure 数据资源管理器群集。In the Azure portal, go to your Azure Data Explorer cluster.

  2. 在“设置”中选择“配置”。**** ****In Settings, select Configurations.

  3. 在“配置”窗格中,选择“关闭”以禁用“启用清除”**** **** ****。In the Configurations pane, select Off to disable Enable purge.

  4. 选择“保存” ****。Select Save.

    禁用清除

限制Limitations

  • 清除过程是最终的且不可逆的。The purge process is final and irreversible. 无法“撤消”此过程或恢复已清除的数据。It isn't possible to "undo" this process or recover data that has been purged. 因此,undo table drop 等命令无法恢复清除的数据,将数据回滚到以前的版本也无法恢复到清除“之前”。Therefore, commands such as undo table drop can't recover purged data, and rollback of the data to a previous version can't go to "before" the latest purge.
  • .purge 命令对数据管理终结点执行: https://ingest- [YourClusterName].[Region].kusto.chinacloudapi.cn**。The .purge command is executed against the Data Management endpoint: https://ingest-[YourClusterName].[Region].kusto.chinacloudapi.cn. 该命令要求对相关数据库具有数据库管理权限。The command requires database admin permissions on the relevant databases.
  • 由于清除过程的性能影响,调用方需要修改数据架构,以便使最小的表包含相关数据,并对每个表执行批处理命令,以减轻清除过程中产生的较大 COGS 影响。Due to the purge process performance impact, the caller is expected to modify the data schema so that minimal tables include relevant data, and batch commands per table to reduce the significant COGS impact of the purge process.
  • 清除命令的 predicate 参数用于指定要清除的记录。The predicate parameter of the purge command is used to specify which records to purge. Predicate 大小限制为 63 KB。Predicate size is limited to 63 KB.

后续步骤Next steps