容量策略Capacity policy

容量策略用于控制群集上数据管理操作的计算资源。A capacity policy is used for controlling the compute resources of data management operations on the cluster.

容量策略对象The capacity policy object

容量策略由以下内容组成:The capacity policy is made of:

引入容量Ingestion capacity

属性Property 类型Type 说明Description
ClusterMaximumConcurrentOperationsClusterMaximumConcurrentOperations longlong 群集中并发引入操作数的最大值A maximal value for the number of concurrent ingestion operations in a cluster
CoreUtilizationCoefficientCoreUtilizationCoefficient Doubledouble 计算引入容量时使用的核心百分比系数。A coefficient for the percentage of cores to use when calculating the ingestion capacity. 计算得出的结果将始终由 ClusterMaximumConcurrentOperations 进行标准化The calculation's result will always be normalized by ClusterMaximumConcurrentOperations

群集总引入容量(由 .show capacity 显示)的计算方式为:The cluster's total ingestion capacity, as shown by .show capacity, is calculated by:

Minimum(ClusterMaximumConcurrentOperations, Number of nodes in cluster * Maximum(1, Core count per node * CoreUtilizationCoefficient))Minimum(ClusterMaximumConcurrentOperations, Number of nodes in cluster * Maximum(1, Core count per node * CoreUtilizationCoefficient))

备注

在具有三个或以上节点的群集中,管理节点不参与引入操作。In clusters with three ore more nodes, the admin node doesn't participate in ingestion operations. Number of nodes in cluster 减少 1 个。The Number of nodes in cluster is reduced by one.

盘区合并容量Extents merge capacity

属性Property 类型Type 说明Description
MinimumConcurrentOperationsPerNodeMinimumConcurrentOperationsPerNode longlong 单个节点上并发盘区合并/重新生成操作数的最小值。A minimal value for the number of concurrent extents merge/rebuild operations on a single node. 默认值为 1Default is 1
MaximumConcurrentOperationsPerNodeMaximumConcurrentOperationsPerNode longlong 单个节点上并发盘区合并/重新生成操作数的最大值。A maximal value for the number of concurrent extents merge/rebuild operations on a single node. 默认值为 3Default is 3

群集总盘区合并容量(由 .show capacity 显示)的计算方式为:The cluster's total extents merge capacity, as shown by .show capacity, is calculated by:

Number of nodes in cluster x Concurrent operations per nodeNumber of nodes in cluster x Concurrent operations per node

在 [MinimumConcurrentOperationsPerNode,MaximumConcurrentOperationsPerNode] 范围内,系统会自动调整 Concurrent operations per node 的有效值,前提是合并操作的成功率高于 90%。The effective value for Concurrent operations per node gets automatically adjusted by the system in the range [MinimumConcurrentOperationsPerNode,MaximumConcurrentOperationsPerNode], as long as the success rate of the merge operations is above 90%.

备注

  • 在具有三个或以上节点的群集中,管理节点不参与合并操作。In clusters with three or more nodes, the admin node doesn't participate in doing merge operations. Number of nodes in cluster 减少 1 个。The Number of nodes in cluster is reduced by one.

盘区清除重新生成容量Extents purge rebuild capacity

属性Property 类型Type 说明Description
MaximumConcurrentOperationsPerNodeMaximumConcurrentOperationsPerNode longlong 单个节点上清除操作的并发重新生成盘区数的最大值A maximal value for the number of concurrent rebuild extents for purge operations on a single node

群集总盘区清除重新生成容量(由 .show capacity 显示)的计算方式为:The cluster's total extents purge rebuild capacity (as shown by .show capacity) is calculated by:

Number of nodes in cluster x MaximumConcurrentOperationsPerNodeNumber of nodes in cluster x MaximumConcurrentOperationsPerNode

备注

在具有三个或以上节点的群集中,管理节点不参与合并操作。In clusters with three or more nodes, the admin node doesn't participate in doing merge operations. Number of nodes in cluster 减少 1 个。The Number of nodes in cluster is reduced by one.

导出容量Export capacity

属性Property 类型Type 说明Description
ClusterMaximumConcurrentOperationsClusterMaximumConcurrentOperations longlong 群集中并发导出操作数的最大值。A maximal value for the number of concurrent export operations in a cluster.
CoreUtilizationCoefficientCoreUtilizationCoefficient Doubledouble 计算导出容量时使用的核心百分比系数。A coefficient for the percentage of cores to use when calculating the export capacity. 计算得出的结果将始终由 ClusterMaximumConcurrentOperations 进行标准化。The calculation's result will always be normalized by ClusterMaximumConcurrentOperations.

群集总导出容量(由 .show capacity 显示)的计算方式为:The cluster's total export capacity, as shown by .show capacity, is calculated by:

Minimum(ClusterMaximumConcurrentOperations, Number of nodes in cluster * Maximum(1, Core count per node * CoreUtilizationCoefficient))Minimum(ClusterMaximumConcurrentOperations, Number of nodes in cluster * Maximum(1, Core count per node * CoreUtilizationCoefficient))

备注

在具有三个或以上节点的群集中,管理节点不参与导出操作。In clusters with three or more nodes, the admin node doesn't participate in export operations. Number of nodes in cluster 减少 1 个。The Number of nodes in cluster is reduced by one.

盘区分区容量Extents partition capacity

属性Property 类型Type 说明Description
ClusterMinimumConcurrentOperationsClusterMinimumConcurrentOperations longlong 群集中并发盘区分区操作数的最小值。A minimal value for the number of concurrent extents partition operations in a cluster. 默认值:1Default: 1
ClusterMaximumConcurrentOperationsClusterMaximumConcurrentOperations longlong 群集中并发盘区分区操作数的最大值。A maximal value for the number of concurrent extents partition operations in a cluster. 默认值:16Default: 16

群集总盘区分区容量(由 .show capacity 显示)。The cluster's total extents partition capacity (as shown by .show capacity).

在 [ClusterMinimumConcurrentOperations,ClusterMaximumConcurrentOperations] 范围内,系统会自动调整 Concurrent operations 的有效值,前提是分区操作的成功率高于 90%。The effective value for Concurrent operations is automatically adjusted by the system in the range [ClusterMinimumConcurrentOperations,ClusterMaximumConcurrentOperations], as long as the success rate of the partitioning operations is above 90%.

默认值Defaults

默认容量策略具有以下 JSON 表示形式:The default capacity policy has the following JSON representation:

{
  "IngestionCapacity": {
    "ClusterMaximumConcurrentOperations": 512,
    "CoreUtilizationCoefficient": 0.75
  },
  "ExtentsMergeCapacity": {
    "MinimumConcurrentOperationsPerNode": 1,
    "MaximumConcurrentOperationsPerNode": 3
  },
  "ExtentsPurgeRebuildCapacity": {
    "MaximumConcurrentOperationsPerNode": 1
  },
  "ExportCapacity": {
    "ClusterMaximumConcurrentOperations": 100,
    "CoreUtilizationCoefficient": 0.25
  },
  "ExtentsPartitionCapacity": {
    "ClusterMinimumConcurrentOperations": 1,
    "ClusterMaximumConcurrentOperations": 16
  }
}

控制命令Control commands

警告

更改容量策略之前,请咨询 Azure 数据资源管理器团队。Consult with the Azure Data Explorer team before altering a capacity policy.

限制Throttling

Kusto 限制以下用户启动命令的并发请求数:Kusto limits the number of concurrent requests for the following user-initiated commands:

  • 引入(包括此处列出的所有命令)Ingestions (includes all the commands that are listed here)
  • 清除Purges
    • 当前全局固定为每个群集一个。Global is currently fixed at one per cluster.
    • 清除重新生成容量在内部用于确定使用清除命令期间的并发重新生成操作数。The purge rebuild capacity is used internally to determine the number of concurrent rebuild operations during purge commands. 清除命令不会因此过程而被阻止/限制,但将以更快或更慢的速度运行,具体取决于清除重新生成容量。Purge commands won't be blocked/throttled because of this process, but will work faster or slower depending on the purge rebuild capacity.
  • 导出Exports

当群集检测到某个操作具有超过允许数量的并发操作时,它将使用 429“受限制”HTTP 代码进行响应。When the cluster detects that an operation has exceeded the permitted concurrent operation, it will respond with a 429, "Throttled", HTTP code. 回退后重试操作。Retry the operation after some backoff.