PyTorchPyTorch

PyTorch 项目是一个 Python 包,可提供 GPU 加速的张量计算和用于构建深度学习网络的高级功能。PyTorch project is a Python package that provides GPU accelerated tensor computation and high level functionalities for building deep learning networks. 如需许可详细信息,请参阅 GitHub 上的 PyTorch 许可文档For licensing details, see the PyTorch license doc on GitHub.

若要监视和调试 PyTorch 模型,请考虑使用 TensorBoardTo monitor and debug your PyTorch models, consider using TensorBoard.

以下部分提供了有关在 Azure Databricks 上安装 PyTorch 的指导,并提供了运行 PyTorch 程序的示例。The following sections provide guidance on installing PyTorch on Azure Databricks and give an example of running PyTorch programs.

备注

本文并不是 PyTorch 的全面指南。This is not a comprehensive guide to PyTorch. 请参阅 PyTorch 网站Refer to the PyTorch website.

安装 PyTorchInstall PyTorch

用于 ML 的 Databricks RuntimeDatabricks Runtime for ML

用于机器学习的 Databricks Runtime 包括 PyTorch,因此你可以创建群集并开始使用 PyTorch。Databricks Runtime for Machine Learning includes PyTorch so you can create the cluster and start using PyTorch. 下面是包括的 Pytorch 版本:Here are the Pytorch versions included:

Databricks Runtime ML 版Databricks Runtime ML Version PyTorch 版本PyTorch Version
Databricks Runtime 7.3 MLDatabricks Runtime 7.3 ML 1.6.01.6.0
Databricks Runtime 7.2 MLDatabricks Runtime 7.2 ML 1.5.11.5.1
Databricks Runtime 7.1 MLDatabricks Runtime 7.1 ML 1.5.11.5.1
Databricks Runtime 7.0 MLDatabricks Runtime 7.0 ML 1.5.01.5.0
Databricks Runtime 6.6 MLDatabricks Runtime 6.6 ML 1.4.01.4.0
Databricks Runtime 6.5 MLDatabricks Runtime 6.5 ML 1.4.01.4.0
Databricks Runtime 6.4 MLDatabricks Runtime 6.4 ML 1.4.01.4.0
Databricks Runtime 6.3 ML(不受支持)Databricks Runtime 6.3 ML (Unsupported) 1.3.11.3.1
Databricks Runtime 5.5 LTS MLDatabricks Runtime 5.5 LTS ML 1.1.01.1.0

Databricks RuntimeDatabricks Runtime

建议使用用于机器学习的 Databricks Runtime 中包含的 PyTorch。We recommend using the PyTorch included on Databricks Runtime for Machine Learning. 但是,如果必须使用 Databricks Runtime,则可以以 Databricks PyPI 库的形式安装 PyTorch。However, if you must use Databricks Runtime, PyTorch can be installed as a Databricks PyPI library. 下面的示例演示如何安装 PyTorch 1.5.0:The following example shows how to install PyTorch 1.5.0:

  • 在 GPU 群集上,通过指定以下内容来安装 pytorchtorchvisionOn GPU clusters, install pytorch and torchvision by specifying the following:
    • torch==1.5.0
    • torchvision==0.6.0
  • 在 CPU 群集上,使用以下 wheel 文件安装 pytorchtorchvisionOn CPU clusters, install pytorch and torchvision by using the following wheel files:
    • https://download.pytorch.org/whl/cpu/torch-1.5.0%2Bcpu-cp37-cp37m-linux_x86_64.whl
    • https://download.pytorch.org/whl/cpu/torchvision-0.6.0%2Bcpu-cp37-cp37m-linux_x86_64.whl

在单节点上使用 PyTorchUse PyTorch on a single node

若要测试和迁移单机 PyTorch 工作流,可以通过将工作器数设置为零,在 Azure Databricks 上从仅限驱动程序的群集着手。To test and migrate single-machine PyTorch workflows, you can start with a driver-only cluster on Azure Databricks by setting the number of workers to zero. 尽管 Apache Spark 在此设置下不起作用,但这是运行单机 PyTorch 工作流的一种经济高效的方法。Though Apache Spark is not functional under this setting, it is a cost-effective way to run single-machine PyTorch workflows.

PyTorch 笔记本PyTorch notebook

获取笔记本Get notebook