发行说明Release notes

本文提供有关最新 Azure HDInsight 版本更新的信息。This article provides information about the most recent Azure HDInsight release updates. 有关较早版本的信息,请参阅 HDInsight 发行说明存档For information on earlier releases, see HDInsight Release Notes Archive.

摘要Summary

Azure HDInsight 是 Azure 中最受企业客户青睐的开源分析服务之一。Azure HDInsight is one of the most popular services among enterprise customers for open-source analytics on Azure.

发行日期:2020/06/11Release date: 06/11/2020

此发行版适用于 HDInsight 3.6 和 4.0。This release applies both for HDInsight 3.6 and 4.0. HDInsight 发行版在几天后即会在所有区域中推出。HDInsight release is made available to all regions over several days. 此处的发行日期是指在第一个区域中的发行日期。The release date here indicates the first region release date. 如果看不到以下更改,请耐心等待,几天后发行版会在你所在的区域推出。If you don't see below changes, wait for the release being live in your region in several days.

新增功能New features

迁移到 Azure 虚拟机规模集Moving to Azure virtual machine scale sets

HDInsight 目前使用 Azure 虚拟机来预配群集。HDInsight uses Azure virtual machines to provision the cluster now. 从此版本起,新创建的 HDInsight 群集开始使用 Azure 虚拟机规模集。From this release, new-created HDInsight clusters start using Azure virtual machine scale set. 此更改将逐步推出。The change is rolling out gradually. 预计不会有中断性变更。You should expect no breaking change. 详细了解 Azure 虚拟机规模集See more about Azure virtual machine scale sets.

重启 HDInsight 群集中的 VMReboot VMs in HDInsight cluster

在此版本中,我们支持重启 HDInsight 群集中的 VM 以重启无响应的节点。In this release, we support rebooting VMs in HDInsight cluster to reboot unresponsive nodes. 目前只能通过 API 完成此操作,即将支持使用 PowerShell 和 CLI。Currently you can only do it through API, PowerShell and CLI support is on the way. 有关此 API 的详细信息,请参阅此文档For more information about the API, see this doc.

弃用Deprecation

弃用 HDInsight 3.6 Spark 群集中的 Spark 2.1 和 2.2Deprecation of Spark 2.1 and 2.2 in HDInsight 3.6 Spark cluster

从 2020 年 7 月 1 日起,客户无法使用 HDInsight 3.6 上的 Spark 2.1 和 2.2 创建新的 Spark 群集。Starting from July 1 2020, customers cannot create new Spark clusters with Spark 2.1 and 2.2 on HDInsight 3.6. 现有群集将在没有 Microsoft 支持的情况下按原样运行。Existing clusters will run as is without the support from Microsoft. 请考虑在 2020 年 6 月 30 日之前转移到 HDInsight 3.6 上的 Spark 2.3,以避免潜在的系统/支持中断。Consider to move to Spark 2.3 on HDInsight 3.6 by June 30 2020 to avoid potential system/support interruption.

弃用 HDInsight 4.0 Spark 群集中的 Spark 2.3Deprecation of Spark 2.3 in HDInsight 4.0 Spark cluster

从 2020 年 7 月 1 日起,客户无法使用 HDInsight 4.0 上的 Spark 2.3 创建新的 Spark 群集。Starting from July 1 2020, customers cannot create new Spark clusters with Spark 2.3 on HDInsight 4.0. 现有群集将在没有 Microsoft 支持的情况下按原样运行。Existing clusters will run as is without the support from Microsoft. 请考虑在 2020 年 6 月 30 日之前转移到 HDInsight 4.0 上的 Spark 2.4,避免出现潜在的系统/支持中断。Consider moving to Spark 2.4 on HDInsight 4.0 by June 30 2020 to avoid potential system/support interruption.

弃用 HDInsight 4.0 Kafka 群集中的 Kafka 1.1Deprecation of Kafka 1.1 in HDInsight 4.0 Kafka cluster

从 2020 年 7 月 1 日开始,客户将无法使用 HDInsight 4.0 上的 Kafka 1.1 创建新的 Kafka 群集。Starting from July 1 2020, customers will not be able to create new Kafka clusters with Kafka 1.1 on HDInsight 4.0. 现有群集将在没有 Microsoft 支持的情况下按原样运行。Existing clusters will run as is without the support from Microsoft. 请考虑在 2020 年 6 月 30 日之前转移到 HDInsight 4.0 上的 Spark 2.1,避免出现潜在的系统/支持中断。Consider moving to Kafka 2.1 on HDInsight 4.0 by June 30 2020 to avoid potential system/support interruption.

行为更改Behavior changes

提供至少有 4 个核心的 VM 作为头节点A minimum 4-core VM is required for Head Node

头节点至少需要 4 核 VM,以确保 HDInsight 群集的高可用性和可靠性。A minimum 4-core VM is required for Head Node to ensure the high availability and reliability of HDInsight clusters. 从 2020 年 4 月 6 日开始,客户只能选择至少有 4 个核心的 VM 作为新 HDInsight 群集的头节点。Starting from April 6 2020, customers can only choose 4-core or above VM as Head Node for the new HDInsight clusters. 现有群集将继续按预期方式运行。Existing clusters will continue to run as expected.

群集工作器节点预配更改Cluster worker node provisioning change

当 80% 的工作器节点准备就绪时,群集将进入可运行阶段。When 80% of the worker nodes are ready, the cluster enters operational stage. 在此阶段中,客户可以执行所有数据平面操作,例如运行脚本和作业。At this stage, customers can do all the data plane operations like running scripts and jobs. 但客户不能执行任何控制平面操作,例如纵向扩展/缩减。But customers can't do any control plane operation like scaling up/down. 仅支持删除。Only deletion is supported.

在进入可运行阶段后,群集会再等待 60 分钟,等待的对象是其余的 20% 的工作器节点。After the operational stage, the cluster waits another 60 minutes for the remaining 20% worker nodes. 在 60 分钟结束时,即使仍有部分工作节点不可用,群集也会进入正在运行阶段。At the end of this 60 minutes, the cluster moves to the running stage, even if all of worker nodes are still not available. 在群集进入正在运行阶段后,你可以正常使用它。Once a cluster enters the running stage, you can use it as normal. 控制平面操作(例如纵向扩展/缩减)和数据平面操作(例如运行脚本和作业)都会被接受。Both control plan operations like scaling up/down, and data plan operations like running scripts and jobs are accepted. 如果所请求的某些工作器节点不可用,则群集会被标记为部分成功。If some of the requested worker nodes are not available, the cluster will be marked as partial success. 你需要为已成功部署的节点付费。You are charged for the nodes that were deployed successfully.

通过 HDInsight 创建新的服务主体Create new service principal through HDInsight

以前,在创建群集的过程中,客户可以创建新的服务主体来访问 Azure 门户中已连接的 ADLS 第 1 代帐户。Previously, with cluster creation, customers can create a new service principal to access the connected ADLS Gen 1 account in Azure portal. 从 2020 年 6 月 15 日起,客户无法在 HDInsight 创建工作流中创建新的服务主体,我们只支持现有的服务主体。Starting June 15 2020, customers cannot create new service principal in HDInsight creation workflow, only existing service principal is supported. 请参阅使用 Azure Active Directory 创建服务主体和证书See Create Service Principal and Certificates using Azure Active Directory.

即将推出的更改Upcoming changes

没有需要注意的即将发生的中断性变更。No upcoming breaking changes that you need to pay attention to.

Bug 修复Bug fixes

HDInsight 会持续改善群集的可靠性和性能。HDInsight continues to make cluster reliability and performance improvements.

组件版本更改Component version change

HBase 2.0 到 2.1.6HBase 2.0 to 2.1.6

HBase 版本已从 2.0 升级到 2.1.6。HBase version is upgraded from version 2.0 to 2.1.6.

Spark 2.4.0 到 2.4.4Spark 2.4.0 to 2.4.4

Spark 版本已从 2.4.0 升级到 2.4.4。Spark version is upgraded from version 2.4.0 to 2.4.4.

Kafka 2.1.0 到 2.1.1Kafka 2.1.0 to 2.1.1

Kafka 版本已从 2.1.0 升级到 2.1.1。Kafka version is upgraded from version 2.1.0 to 2.1.1.

可以在此文档中查找 HDInsight 4.0 和 HDInsight 3.6 的当前组件版本You can find the current component versions for HDInsight 4.0 ad HDInsight 3.6 in this doc