Databricks Runtime 6.2 ML(不受支持)Databricks Runtime 6.2 ML (Unsupported)

Databricks 于 2019 年 12 月发布了此映像。Databricks released this image in December 2019.

Databricks Runtime 6.2 ML 基于 Databricks Runtime 6.2(不受支持),为机器学习和数据科学提供了随时可用的环境。Databricks Runtime 6.2 ML provides a ready-to-go environment for machine learning and data science based on Databricks Runtime 6.2 (Unsupported). Databricks Runtime ML 包含许多常用的机器学习库,包括 TensorFlow、PyTorch、Keras 和 XGBoost。Databricks Runtime ML contains many popular machine learning libraries, including TensorFlow, PyTorch, Keras, and XGBoost. 它还支持使用 Horovod 进行分布式深度学习训练。It also supports distributed deep learning training using Horovod.

有关详细信息,包括有关创建 Databricks Runtime ML 群集的说明,请参阅用于机器学习的 Databricks RuntimeFor more information, including instructions for creating a Databricks Runtime ML cluster, see Databricks Runtime for Machine Learning.

新增功能New features

Databricks Runtime 6.2 ML 是基于 Databricks Runtime 6.2 构建的。Databricks Runtime 6.2 ML is built on top of Databricks Runtime 6.2. 若要了解 Databricks Runtime 6.2 中的新增功能,请参阅 Databricks Runtime 6.2(不受支持)发行说明。For information on what’s new in Databricks Runtime 6.2, see the Databricks Runtime 6.2 (Unsupported) release notes.

改进Improvements

升级了机器学习库Upgraded machine learning libraries

  • TensorFlow 和 TensorBoard:1.14.0 到 1.15.0。TensorFlow and TensorBoard: 1.14.0 to 1.15.0. 存在两个已知问题:There are two known issues:

    • 你可能需要在函数中显式导入 tensorflow 模块,以避免 PySpark、HorovodRunner、HyperOpt 和其他机器学习库中发生 pickling 问题。You might need to import tensorflow modules explicitly in your function to avoid pickling issues in PySpark, HorovodRunner, HyperOpt, and other machine learning libraries. 请参阅 TensorFlow 的 2 个已知问题See TensorFlow 2 known issues.
    • TensorBoard 中的“投影仪”选项卡为空白。The Projector tab in TensorBoard is blank. 若要直接访问投影仪页,解决方法是,将 URL 中的 #projector 替换为 data/plugin/projector/projector_binary.htmlAs a workaround, to visit the projector page directly, you can replace #projector in the URL by data/plugin/projector/projector_binary.html.
  • Keras:2.2.4 到 2.2.5。Keras: 2.2.4 to 2.2.5.

    备注

    如果为 Keras 使用 TensorFlow 后端,则 Databricks 建议改用 tf.kerasIf you use the TensorFlow backend for Keras, Databricks recommends using tf.keras instead.

  • PyTorch:1.2.0 到 1.3.0。PyTorch: 1.2.0 to 1.3.0.

  • tensorboardX:1.8 到 1.9。tensorboardX: 1.8 to 1.9.

    备注

    由于 PyTorch 现已正式支持 TensorBoard,因此我们将在下一个主要版本中删除 tensorboardX。Because PyTorch now officially supports TensorBoard, we will remove tensorboardX in the next major release.

  • MLflow:1.3.0 到 1.4.0。MLflow: 1.3.0 to 1.4.0.

    • Keras 和 TensorFlow autologging 及 Keras 模型持久性 API 现在与 TensorFlow 2.0 兼容。Keras and TensorFlow autologging and Keras model persistence APIs are now compatible with TensorFlow 2.0.
    • get_runget_experimentget_experiment_by_name 函数New get_run, get_experiment, get_experiment_by_name functions
  • Hyperopt:0.2-db1 与 Azure Databricks MLflow 集成。Hyperopt: 0.2-db1 with Azure Databricks MLflow integrations.

  • mleap-databricks-runtime 到 0.15.0,包括 mleap-xgboost-runtime。mleap-databricks-runtime to 0.15.0 and includes mleap-xgboost-runtime.

向 SparkTrials 添加了对广播变量的支持Added support for broadcast variables to SparkTrials

以前,具有 SparkTrials 的 Hyperopt 无法与 PySpark 广播变量一起使用。Previously, Hyperopt with SparkTrials could not be used with PySpark broadcast variables. 现在,可将广播变量包括在传递给 fmin() 的函数 fn 中。Now, broadcast variables can be included in the function fn passed to fmin().

弃用功能Deprecations

除了 Databricks Runtime 6.2 中的弃用功能之外,以下包也已弃用,并将从下一个主要版本中删除:In addition to deprecations in Databricks Runtime 6.2, the following packages are deprecated and will be removed in the next major release:

  • TensorFrames。TensorFrames. 改用 pandas UDFUse pandas UDF instead.
  • Python 包 sparkdl 中的一些模块和类。A few modules and classes in the Python package sparkdl. 主要有:The major ones are:
    • sparkdl.HorovodEstimator.sparkdl.HorovodEstimator. 改用 sparkdl.HorovodRunnerUse sparkdl.HorovodRunner instead.
    • sparkdl.graph.sparkdl.graph. 改用 pandas UDFUse a pandas UDF instead.
    • sparkdl.udf.sparkdl.udf. 改用 pandas UDFUse a pandas UDF instead.
    • Spark ML 管道中使用的转换器和估算器已弃用。The Transformers and Estimators used in Spark ML pipelines are deprecated. 使用以下替代方法:Use the following alternatives:
      • 使用 pandas UDF 作为以下转换器的替代项:Use a pandas UDF as a replacement for the following Transformers:
        • TFImageTransformer
        • TFTransformer
        • DeepImagePredictor
        • DeepImageFeaturizer
        • KerasImageFileTransformer
        • KerasTransformer
      • KerasImageFileEstimator:若要优化深度学习模型,请改用 HyperoptKerasImageFileEstimator: For tuning deep learning models, use Hyperopt instead.

如需了解更多详细信息和建议的替代方法,可在笔记本中使用这些包时查看弃用消息。For more details and recommended alternatives, look at the deprecation messages when you use these packages in a notebook.

Bug 修复Bug fixes

在 Databricks Community Edition 中,PySpark 辅助角色现在可以找到预安装的 Spark 包。In Databricks Community Edition, PySpark workers can now find pre-installed Spark Packages.

系统环境System environment

Databricks Runtime 6.2 ML 中的系统环境与 Databricks Runtime 6.2 不同,如下所示:The system environment in Databricks Runtime 6.2 ML differs from Databricks Runtime 6.2 as follows:

  • DBUtils:不包含库实用工具DBUtils: Does not contain Library utilities.
  • 对于 GPU 群集,以下 NVIDIA GPU 库:For GPU clusters, the following NVIDIA GPU libraries:
    • NVIDIA 驱动程序 418.40NVIDIA driver 418.40
    • CUDA 10.0CUDA 10.0
    • cuDNN 7.6.4cuDNN 7.6.4
    • NCCL 2.4.7NCCL 2.4.7

Libraries

以下部分列出了 Databricks Runtime 6.2 ML 中包含的库,这些库不同于 Databricks Runtime 6.2 中包含的库。The following sections list the libraries included in Databricks Runtime 6.2 ML that differ from those included in Databricks Runtime 6.2.

本节内容:In this section:

顶层库Top-tier libraries

Databricks Runtime 6.2 ML 包含以下顶层Databricks Runtime 6.2 ML includes the following top-tier libraries:

Python 库Python libraries

Databricks Runtime 6.2 ML 使用 Conda 进行 Python 包管理,并且包含许多常用的 ML 包。Databricks Runtime 6.2 ML uses Conda for Python package management and includes many popular ML packages. 下一部分介绍用于 Databricks Runtime 6.2 ML 的 Conda 环境。The following section describes the Conda environment for Databricks Runtime 6.2 ML.

CPU 群集上的 PythonPython on CPU clusters

name: databricks-ml
channels:
  - Databricks
  - pytorch
  - defaults
dependencies:
  - _libgcc_mutex=0.1=main
  - _py-xgboost-mutex=2.0=cpu_0
  - _tflow_select=2.3.0=mkl
  - absl-py=0.8.1=py37_0
  - asn1crypto=0.24.0=py37_0
  - astor=0.8.0=py37_0
  - backcall=0.1.0=py37_0
  - backports=1.0=py_2
  - bcrypt=3.1.7=py37h7b6447c_0
  - blas=1.0=mkl
  - boto=2.49.0=py37_0
  - boto3=1.9.162=py_0
  - botocore=1.12.163=py_0
  - c-ares=1.15.0=h7b6447c_1001
  - ca-certificates=2019.1.23=0
  - certifi=2019.3.9=py37_0
  - cffi=1.12.2=py37h2e261b9_1
  - chardet=3.0.4=py37_1003
  - click=7.0=py_0
  - cloudpickle=0.8.0=py37_0
  - colorama=0.4.1=py_0
  - configparser=3.7.4=py37_0
  - cpuonly=1.0=0
  - cryptography=2.6.1=py37h1ba5d50_0
  - cycler=0.10.0=py37_0
  - cython=0.29.6=py37he6710b0_0
  - decorator=4.4.0=py37_1
  - docutils=0.14=py37_0
  - entrypoints=0.3=py37_0
  - et_xmlfile=1.0.1=py37_0
  - flask=1.0.2=py37_1
  - freetype=2.9.1=h8a8886c_1
  - future=0.17.1=py37_0
  - gast=0.2.2=py37_0
  - gitdb2=2.0.6=py_0
  - gitpython=2.1.11=py37_0
  - google-pasta=0.1.8=py_0
  - grpcio=1.16.1=py37hf8bcb03_1
  - gunicorn=19.9.0=py37_0
  - h5py=2.9.0=py37h7918eee_0
  - hdf5=1.10.4=hb1b8bf9_0
  - html5lib=1.0.1=py_0
  - icu=58.2=h9c2bf20_1
  - idna=2.8=py37_0
  - intel-openmp=2019.3=199
  - ipython=7.4.0=py37h39e3cac_0
  - ipython_genutils=0.2.0=py37_0
  - itsdangerous=1.1.0=py_0
  - jdcal=1.4=py37_0
  - jedi=0.13.3=py37_0
  - jinja2=2.10=py37_0
  - jmespath=0.9.4=py_0
  - jpeg=9b=h024ee3a_2
  - keras-applications=1.0.8=py_0
  - keras-preprocessing=1.1.0=py_1
  - kiwisolver=1.0.1=py37hf484d3e_0
  - krb5=1.16.1=h173b8e3_7
  - libedit=3.1.20181209=hc058e9b_0
  - libffi=3.2.1=hd88cf55_4
  - libgcc-ng=8.2.0=hdf63c60_1
  - libgfortran-ng=7.3.0=hdf63c60_0
  - libpng=1.6.36=hbc83047_0
  - libpq=11.2=h20c2e04_0
  - libprotobuf=3.9.2=hd408876_0
  - libsodium=1.0.16=h1bed415_0
  - libstdcxx-ng=8.2.0=hdf63c60_1
  - libtiff=4.0.10=h2733197_2
  - libxgboost=0.90=he6710b0_1
  - libxml2=2.9.9=hea5a465_1
  - libxslt=1.1.33=h7d1a2b0_0
  - llvmlite=0.28.0=py37hd408876_0
  - lxml=4.3.2=py37hefd8a0e_0
  - mako=1.0.10=py_0
  - markdown=3.1.1=py37_0
  - markupsafe=1.1.1=py37h7b6447c_0
  - mkl=2019.3=199
  - mkl_fft=1.0.10=py37ha843d7b_0
  - mkl_random=1.0.2=py37hd81dba3_0
  - ncurses=6.1=he6710b0_1
  - networkx=2.2=py37_1
  - ninja=1.9.0=py37hfd86e86_0
  - nose=1.3.7=py37_2
  - numba=0.43.1=py37h962f231_0
  - numpy=1.16.2=py37h7e9f1db_0
  - numpy-base=1.16.2=py37hde5b4d6_0
  - olefile=0.46=py_0
  - openpyxl=2.6.1=py37_1
  - openssl=1.1.1b=h7b6447c_1
  - opt_einsum=3.1.0=py_0
  - pandas=0.24.2=py37he6710b0_0
  - paramiko=2.4.2=py37_0
  - parso=0.3.4=py37_0
  - pathlib2=2.3.3=py37_0
  - patsy=0.5.1=py37_0
  - pexpect=4.6.0=py37_0
  - pickleshare=0.7.5=py37_0
  - pillow=5.4.1=py37h34e0f95_0
  - pip=19.0.3=py37_0
  - ply=3.11=py37_0
  - prompt_toolkit=2.0.9=py37_0
  - protobuf=3.9.2=py37he6710b0_0
  - psutil=5.6.1=py37h7b6447c_0
  - psycopg2=2.7.6.1=py37h1ba5d50_0
  - ptyprocess=0.6.0=py37_0
  - py-xgboost=0.90=py37he6710b0_1
  - py-xgboost-cpu=0.90=py37_1
  - pyasn1=0.4.8=py_0
  - pycparser=2.19=py_0
  - pygments=2.3.1=py37_0
  - pymongo=3.8.0=py37he6710b0_1
  - pynacl=1.3.0=py37h7b6447c_0
  - pyopenssl=19.0.0=py37_0
  - pyparsing=2.3.1=py37_0
  - pysocks=1.6.8=py37_0
  - python=3.7.3=h0371630_0
  - python-dateutil=2.8.0=py37_0
  - python-editor=1.0.4=py_0
  - pytorch=1.3.0=py3.7_cpu_0
  - pytz=2018.9=py37_0
  - pyyaml=5.1=py37h7b6447c_0
  - readline=7.0=h7b6447c_5
  - requests=2.21.0=py37_0
  - s3transfer=0.2.1=py37_0
  - scikit-learn=0.20.3=py37hd81dba3_0
  - scipy=1.2.1=py37h7c811a0_0
  - setuptools=40.8.0=py37_0
  - simplejson=3.16.0=py37h14c3975_0
  - singledispatch=3.4.0.3=py37_0
  - six=1.12.0=py37_0
  - smmap2=2.0.5=py_0
  - sqlite=3.27.2=h7b6447c_0
  - sqlparse=0.3.0=py_0
  - statsmodels=0.9.0=py37h035aef0_0
  - tabulate=0.8.3=py37_0
  - tensorboard=1.15.0+db2=pyhb230dea_0
  - tensorflow=1.15.0+db2=mkl_py37hc5fbf04_0
  - tensorflow-base=1.15.0+db2=mkl_py37h2ae1e84_0
  - tensorflow-estimator=1.15.1+db2=pyh2649769_0
  - tensorflow-mkl=1.15.0+db2=h4fcabd2_0
  - termcolor=1.1.0=py37_1
  - tk=8.6.8=hbc83047_0
  - torchvision=0.4.1=py37_cpu
  - tqdm=4.31.1=py37_1
  - traitlets=4.3.2=py37_0
  - urllib3=1.24.1=py37_0
  - virtualenv=16.0.0=py37_0
  - wcwidth=0.1.7=py37_0
  - webencodings=0.5.1=py37_1
  - websocket-client=0.56.0=py37_0
  - werkzeug=0.14.1=py37_0
  - wheel=0.33.1=py37_0
  - wrapt=1.11.1=py37h7b6447c_0
  - xz=5.2.4=h14c3975_4
  - yaml=0.1.7=had09818_2
  - zlib=1.2.11=h7b6447c_3
  - zstd=1.3.7=h0b5b093_0
  - pip:
    - argparse==1.4.0
    - databricks-cli==0.9.1
    - deprecated==1.2.7
    - docker==4.1.0
    - fusepy==2.0.4
    - gorilla==0.3.0
    - horovod==0.18.2
    - hyperopt==0.2.1.db1
    - keras==2.2.5
    - matplotlib==3.0.3
    - mleap==0.8.1
    - mlflow==1.4.0
    - nose-exclude==0.5.0
    - pyarrow==0.13.0
    - querystring-parser==1.2.4
    - seaborn==0.9.0
    - tensorboardx==1.9
prefix: /databricks/conda/envs/databricks-ml

GPU 群集上的 PythonPython on GPU clusters

name: databricks-ml-gpu
channels:
  - Databricks
  - pytorch
  - defaults
dependencies:
  - _libgcc_mutex=0.1=main
  - _py-xgboost-mutex=1.0=gpu_0
  - _tflow_select=2.1.0=gpu
  - absl-py=0.8.1=py37_0
  - asn1crypto=0.24.0=py37_0
  - astor=0.8.0=py37_0
  - backcall=0.1.0=py37_0
  - backports=1.0=py_2
  - bcrypt=3.1.7=py37h7b6447c_0
  - blas=1.0=mkl
  - boto=2.49.0=py37_0
  - boto3=1.9.162=py_0
  - botocore=1.12.163=py_0
  - c-ares=1.15.0=h7b6447c_1001
  - ca-certificates=2019.1.23=0
  - certifi=2019.3.9=py37_0
  - cffi=1.12.2=py37h2e261b9_1
  - chardet=3.0.4=py37_1003
  - click=7.0=py_0
  - cloudpickle=0.8.0=py37_0
  - colorama=0.4.1=py_0
  - configparser=3.7.4=py37_0
  - cryptography=2.6.1=py37h1ba5d50_0
  - cudatoolkit=10.0.130=0
  - cudnn=7.6.4=cuda10.0_0
  - cupti=10.0.130=0
  - cycler=0.10.0=py37_0
  - cython=0.29.6=py37he6710b0_0
  - decorator=4.4.0=py37_1
  - docutils=0.14=py37_0
  - entrypoints=0.3=py37_0
  - et_xmlfile=1.0.1=py37_0
  - flask=1.0.2=py37_1
  - freetype=2.9.1=h8a8886c_1
  - future=0.17.1=py37_0
  - gast=0.2.2=py37_0
  - gitdb2=2.0.6=py_0
  - gitpython=2.1.11=py37_0
  - google-pasta=0.1.8=py_0
  - grpcio=1.16.1=py37hf8bcb03_1
  - gunicorn=19.9.0=py37_0
  - h5py=2.9.0=py37h7918eee_0
  - hdf5=1.10.4=hb1b8bf9_0
  - html5lib=1.0.1=py_0
  - icu=58.2=h9c2bf20_1
  - idna=2.8=py37_0
  - intel-openmp=2019.3=199
  - ipython=7.4.0=py37h39e3cac_0
  - ipython_genutils=0.2.0=py37_0
  - itsdangerous=1.1.0=py_0
  - jdcal=1.4=py37_0
  - jedi=0.13.3=py37_0
  - jinja2=2.10=py37_0
  - jmespath=0.9.4=py_0
  - jpeg=9b=h024ee3a_2
  - keras-applications=1.0.8=py_0
  - keras-preprocessing=1.1.0=py_1
  - kiwisolver=1.0.1=py37hf484d3e_0
  - krb5=1.16.1=h173b8e3_7
  - libedit=3.1.20181209=hc058e9b_0
  - libffi=3.2.1=hd88cf55_4
  - libgcc-ng=8.2.0=hdf63c60_1
  - libgfortran-ng=7.3.0=hdf63c60_0
  - libpng=1.6.36=hbc83047_0
  - libpq=11.2=h20c2e04_0
  - libprotobuf=3.9.2=hd408876_0
  - libsodium=1.0.16=h1bed415_0
  - libstdcxx-ng=8.2.0=hdf63c60_1
  - libtiff=4.0.10=h2733197_2
  - libxgboost=0.90=h688424c_0
  - libxml2=2.9.9=hea5a465_1
  - libxslt=1.1.33=h7d1a2b0_0
  - llvmlite=0.28.0=py37hd408876_0
  - lxml=4.3.2=py37hefd8a0e_0
  - mako=1.0.10=py_0
  - markdown=3.1.1=py37_0
  - markupsafe=1.1.1=py37h7b6447c_0
  - mkl=2019.3=199
  - mkl_fft=1.0.10=py37ha843d7b_0
  - mkl_random=1.0.2=py37hd81dba3_0
  - ncurses=6.1=he6710b0_1
  - networkx=2.2=py37_1
  - ninja=1.9.0=py37hfd86e86_0
  - nose=1.3.7=py37_2
  - numba=0.43.1=py37h962f231_0
  - numpy=1.16.2=py37h7e9f1db_0
  - numpy-base=1.16.2=py37hde5b4d6_0
  - olefile=0.46=py_0
  - openpyxl=2.6.1=py37_1
  - openssl=1.1.1b=h7b6447c_1
  - opt_einsum=3.1.0=py_0
  - pandas=0.24.2=py37he6710b0_0
  - paramiko=2.4.2=py37_0
  - parso=0.3.4=py37_0
  - pathlib2=2.3.3=py37_0
  - patsy=0.5.1=py37_0
  - pexpect=4.6.0=py37_0
  - pickleshare=0.7.5=py37_0
  - pillow=5.4.1=py37h34e0f95_0
  - pip=19.0.3=py37_0
  - ply=3.11=py37_0
  - prompt_toolkit=2.0.9=py37_0
  - protobuf=3.9.2=py37he6710b0_0
  - psutil=5.6.1=py37h7b6447c_0
  - psycopg2=2.7.6.1=py37h1ba5d50_0
  - ptyprocess=0.6.0=py37_0
  - py-xgboost=0.90=py37h688424c_0
  - py-xgboost-gpu=0.90=py37h28bbb66_0
  - pyasn1=0.4.8=py_0
  - pycparser=2.19=py_0
  - pygments=2.3.1=py37_0
  - pymongo=3.8.0=py37he6710b0_1
  - pynacl=1.3.0=py37h7b6447c_0
  - pyopenssl=19.0.0=py37_0
  - pyparsing=2.3.1=py37_0
  - pysocks=1.6.8=py37_0
  - python=3.7.3=h0371630_0
  - python-dateutil=2.8.0=py37_0
  - python-editor=1.0.4=py_0
  - pytorch=1.3.0=py3.7_cuda10.0.130_cudnn7.6.3_0
  - pytz=2018.9=py37_0
  - pyyaml=5.1=py37h7b6447c_0
  - readline=7.0=h7b6447c_5
  - requests=2.21.0=py37_0
  - s3transfer=0.2.1=py37_0
  - scikit-learn=0.20.3=py37hd81dba3_0
  - scipy=1.2.1=py37h7c811a0_0
  - setuptools=40.8.0=py37_0
  - simplejson=3.16.0=py37h14c3975_0
  - singledispatch=3.4.0.3=py37_0
  - six=1.12.0=py37_0
  - smmap2=2.0.5=py_0
  - sqlite=3.27.2=h7b6447c_0
  - sqlparse=0.3.0=py_0
  - statsmodels=0.9.0=py37h035aef0_0
  - tabulate=0.8.3=py37_0
  - tensorboard=1.15.0+db2=pyhb230dea_0
  - tensorflow=1.15.0+db2=gpu_py37h9fd0ff8_0
  - tensorflow-base=1.15.0+db2=gpu_py37hd56f5dd_0
  - tensorflow-estimator=1.15.1+db2=pyh2649769_0
  - tensorflow-gpu=1.15.0+db2=h0d30ee6_0
  - termcolor=1.1.0=py37_1
  - tk=8.6.8=hbc83047_0
  - torchvision=0.4.1=py37_cu100
  - tqdm=4.31.1=py37_1
  - traitlets=4.3.2=py37_0
  - urllib3=1.24.1=py37_0
  - virtualenv=16.0.0=py37_0
  - wcwidth=0.1.7=py37_0
  - webencodings=0.5.1=py37_1
  - websocket-client=0.56.0=py37_0
  - werkzeug=0.14.1=py37_0
  - wheel=0.33.1=py37_0
  - wrapt=1.11.1=py37h7b6447c_0
  - xz=5.2.4=h14c3975_4
  - yaml=0.1.7=had09818_2
  - zlib=1.2.11=h7b6447c_3
  - zstd=1.3.7=h0b5b093_0
  - pip:
    - argparse==1.4.0
    - databricks-cli==0.9.1
    - deprecated==1.2.7
    - docker==4.1.0
    - fusepy==2.0.4
    - gorilla==0.3.0
    - horovod==0.18.2
    - hyperopt==0.2.1.db1
    - keras==2.2.5
    - matplotlib==3.0.3
    - mleap==0.8.1
    - mlflow==1.4.0
    - nose-exclude==0.5.0
    - pyarrow==0.13.0
    - querystring-parser==1.2.4
    - seaborn==0.9.0
    - tensorboardx==1.9
prefix: /databricks/conda/envs/databricks-ml-gpu

包含 Python 模块的 Spark 包Spark packages containing Python modules

Spark 包Spark Package Python 模块Python Module 版本Version
graphframesgraphframes graphframesgraphframes 0.7.0-db1-spark2.40.7.0-db1-spark2.4
spark-deep-learningspark-deep-learning sparkdlsparkdl 1.5.0-db12-spark2.41.5.0-db12-spark2.4
tensorframestensorframes tensorframestensorframes 0.8.2-s_2.110.8.2-s_2.11

R 库R libraries

R 库与 Databricks Runtime 6.2 中的 R 库完全相同。The R libraries are identical to the R Libraries in Databricks Runtime 6.2.

Java 库和 Scala 库(Scala 2.11 群集)Java and Scala libraries (Scala 2.11 cluster)

除了 Databricks Runtime 6.2 中的 Java 库和 Scala 库之外,Databricks Runtime 6.2 ML 还包含以下 JAR:In addition to Java and Scala libraries in Databricks Runtime 6.2, Databricks Runtime 6.2 ML contains the following JARs:

组 IDGroup ID 项目 IDArtifact ID 版本Version
com.databrickscom.databricks spark-deep-learningspark-deep-learning 1.5.0-db12-spark2.41.5.0-db12-spark2.4
com.typesafe.akkacom.typesafe.akka akka-actor_2.11akka-actor_2.11 2.3.112.3.11
ml.combust.mleapml.combust.mleap mleap-databricks-runtime_2.11mleap-databricks-runtime_2.11 0.15.00.15.0
ml.dmlcml.dmlc xgboost4jxgboost4j 0.900.90
ml.dmlcml.dmlc xgboost4j-sparkxgboost4j-spark 0.900.90
org.graphframesorg.graphframes graphframes_2.11graphframes_2.11 0.7.0-db1-spark2.40.7.0-db1-spark2.4
org.mlfloworg.mlflow mlflow-clientmlflow-client 1.4.01.4.0
org.tensorfloworg.tensorflow libtensorflowlibtensorflow 1.15.01.15.0
org.tensorfloworg.tensorflow libtensorflow_jnilibtensorflow_jni 1.15.01.15.0
org.tensorfloworg.tensorflow spark-tensorflow-connector_2.11spark-tensorflow-connector_2.11 1.15.01.15.0
org.tensorfloworg.tensorflow tensorflowtensorflow 1.15.01.15.0
org.tensorframesorg.tensorframes tensorframestensorframes 0.8.2-s_2.110.8.2-s_2.11