Databricks Runtime 4.1 ML(不受支持)Databricks Runtime 4.1 ML (Unsupported)

Databricks Runtime 4.1 ML 为机器学习和数据科学提供随时可用的环境。Databricks Runtime 4.1 ML provides a ready-to-go environment for machine learning and data science. 它包含多个热门库,其中包括 TensorFlow、Keras 和 XGBoost。It contains multiple popular libraries, including TensorFlow, Keras, and XGBoost. 它还支持使用 Horovod 进行分布式 TensorFlow 训练。It also supports distributed TensorFlow training using Horovod.


此版本已于 2019 年 1 月 17 日弃用。This release was deprecated on January 17, 2019. 建议使用较新的 Databricks Runtime ML 版本,这取决于你要使用的库版本。We recommend that you use a newer version of Databricks Runtime ML, depending on which library versions you want to use.

有关详细信息,包括有关创建 Databricks Runtime ML 群集的说明,请参阅用于机器学习的 Databricks RuntimeFor more information, including instructions for creating a Databricks Runtime ML cluster, see Databricks Runtime for Machine Learning.


Databricks Runtime ML 版本会获取基础 Databricks Runtime 版本的所有维护更新。Databricks Runtime ML releases pick up all maintenance updates to the base Databricks Runtime release. 有关所有维护更新的列表,请参阅 Databricks 运行时维护更新For a list of all maintenance updates, see Databricks runtime maintenance updates.


Databricks Runtime 4.1 ML 是基于 Databricks Runtime 4.1 构建的。Databricks Runtime 4.1 ML is built on top of Databricks Runtime 4.1. 若要了解 Databricks Runtime 4.1 中的新增功能,请参阅 Databricks Runtime 4.1(不受支持)发行说明。For information on what’s new in Databricks Runtime 4.1, see the Databricks Runtime 4.1 (Unsupported) release notes. 除了 Databricks Runtime 4.1 中的新功能,Databricks Runtime 4.1 ML 还包括以下库来支持机器学习。In addition to the new features in Databricks Runtime 4.1, Databricks Runtime 4.1 ML includes the following libraries to support machine learning. 其中一些功能也包含在基础 Databricks Runtime 4.1 中,并同样进行了说明。Some of these are also included in the base Databricks Runtime 4.1 and are noted as such.

类别Category Libraries
分布式深度学习Distributed Deep Learning 使用 Horovod 和 Spark 进行分布式训练Distributed training with Horovod and Spark:

* HorovodEstimator* HorovodEstimator
* horovod 0.12.1* horovod 0.12.1
* openmpi 3.0.0* openmpi 3.0.0
* paramiko 2.4.1* paramiko 2.4.1
* cloudpickle 0.5.2* cloudpickle 0.5.2

分布式 TensorFlow 和 Keras 预测Distributed TensorFlow and Keras prediction:

* spark-deep-learning 1.0 pre-release* spark-deep-learning 1.0 pre-release
* tensorframes 0.3.0* tensorframes 0.3.0
深度学习Deep Learning [Keras]:

* keras 2.1.5* keras 2.1.5
* h5py 2.7.1* h5py 2.7.1


* (CPU clusters) tensorflow 1.7.1* (CPU clusters) tensorflow 1.7.1
* (GPU clusters) tensorflow-gpu 1.7.1* (GPU clusters) tensorflow-gpu 1.7.1

GPU 库:GPU libraries:

* CUDA 9.0(也包括在基础 Databricks Runtime 中)* CUDA 9.0 (also installed in base Databricks Runtime)
* cuDNN 7.0(也包括在基础 Databricks Runtime 中)* cuDNN 7.0 (also installed in base Databricks Runtime)
* NCCL 2.0.5-3* NCCL 2.0.5-3
XGBoostXGBoost * XGBoost4j 0.8-spark2.3-s_2.11* XGBoost4j 0.8-spark2.3-s_2.11
其他机器学习库Other machine learning libraries * Numpy 1.14.2(也包括在基础 Databricks Runtime 中;版本可能不同)* numpy 1.14.2 (also installed in base Databricks Runtime; version may differ)
* Scikit-learn 0.18.1(也已在基本 Databricks Runtime 中安装)* scikit-learn 0.18.1 (also installed in base Databricks Runtime)
* Scipy(也包括在基础 Databricks Runtime 中)* scipy (also installed in base Databricks Runtime)