如何计算群集中的核心数How to calculate the number of cores in a cluster

如果组织已在群集节点上安装指标服务,则可以使用群集详细信息页上的指标选项卡在工作区 UI 中查看 Azure Databricks 群集中的核心数。If your organization has installed a metrics service on your cluster nodes, you can view the number of cores in an Azure Databricks cluster in the Workspace UI using the Metrics tab on the cluster details page.

如果驱动程序和执行程序的节点类型相同,则还可以使用 Scala 实用程序代码以编程方式确定群集中可用的核心数:If the driver and executors are of the same node type, you can also determine the number of cores available in a cluster programmatically, using Scala utility code:

  1. 使用 sc.statusTracker.getExecutorInfos.length 获取节点总数。Use sc.statusTracker.getExecutorInfos.length to get the total number of nodes. 结果包括驱动程序节点,因此减 1。The result includes the driver node, so subtract 1.

  2. 使用 java.lang.Runtime.getRuntime.availableProcessors 获取每个节点的核心数。Use java.lang.Runtime.getRuntime.availableProcessors to get the number of cores per node.

  3. 将两个结果相乘(从节点总数中减去 1)得到可用总核心数:Multiply both results (subtracting 1 from the total number of nodes) to get the total number of cores available:

    java.lang.Runtime.getRuntime.availableProcessors * (sc.statusTracker.getExecutorInfos.length -1)