教程:在你的开发环境中开始使用 Azure 机器学习(第 1 部分,共 4 部分)Tutorial: Get started with Azure Machine Learning in your development environment (part 1 of 4)

在这个由四部分组成的教程系列中,你将了解 Azure 机器学习的基础知识,并在 Azure 云平台上完成基于作业的 Python 机器学习任务。In this four-part tutorial series, you'll learn the fundamentals of Azure Machine Learning and complete jobs-based Python machine learning tasks on the Azure cloud platform.

在本教程系列的第 1 部分中,你将执行以下操作:In part 1 of this tutorial series, you will:

  • 安装 Azure 机器学习 SDK。Install the Azure Machine Learning SDK.
  • 设置代码的目录结构。Set up the directory structure for code.
  • 创建 Azure 机器学习工作区。Create an Azure Machine Learning workspace.
  • 配置本地开发环境。Configure your local development environment.
  • 设置计算群集。Set up a compute cluster.

备注

本教程系列重点介绍提交批处理作业所需的 Azure 机器学习概念(在批处理作业中,代码提交到云以在后台运行,而无需任何用户交互)。This tutorial series focuses on the Azure Machine Learning concepts required to submit batch jobs - this is where the code is submitted to the cloud to run in the background without any user interaction. 这适用于要重复运行的已完成脚本或代码或者计算密集型机器学习任务。This is useful for finished scripts or code you wish to run repeatedly, or for compute-intensive machine learning tasks. 如果对探索性工作流更感兴趣,可以改用 Azure 机器学习计算实例上的 Jupyter 或 RStudioIf you are more interested in an exploratory workflow, you could instead use Jupyter or RStudio on an Azure Machine Learning compute instance.

先决条件Prerequisites

安装 Azure 机器学习 SDKInstall the Azure Machine Learning SDK

本教程从头到尾都将使用适用于 Python 的 Azure 机器学习 SDK。Throughout this tutorial, you will use the Azure Machine Learning SDK for Python. 为了避免 Python 依赖项问题,你将创建一个隔离的环境。To avoid Python dependency issues, you'll create an isolated environment. 本教程系列使用 conda 创建该环境。This tutorial series uses conda to create that environment. 如果你希望使用其他解决方案(如 venvvirtualenv 或 docker),请确保使用的 Python 版本 >=3.5 且 < 3.9。If you prefer to use other solutions, such as venv, virtualenv, or docker, make sure you use a Python version >=3.5 and < 3.9.

检查系统是否安装了 conda:Check if you have conda installed on your system:

conda --version

如果此命令返回 conda not found 错误,则下载并安装 MinicondaIf this command returns a conda not found error, download and install Miniconda.

安装 Conda 后,使用终端或 Anaconda 提示符窗口创建新环境:Once you have installed Conda, use a terminal or Anaconda Prompt window to create a new environment:

conda create -n tutorial python=3.8

接下来,将 Azure 机器学习 SDK 安装到已创建的 conda 环境中:Next, install the Azure Machine Learning SDK into the conda environment you created:

conda activate tutorial
pip install azureml-core

备注

完成 Azure 机器学习 SDK 安装大约需要 2 分钟。It takes approximately 2 minutes for the Azure Machine Learning SDK install to complete.

如果遇到超时错误,请改为尝试 pip install --default-timeout=100 azureml-coreIf you get a timeout error, try pip install --default-timeout=100 azureml-core instead.

创建代码的目录结构Create a directory structure for code

建议为本教程设置以下简单目录结构:We recommend that you set up the following simple directory structure for this tutorial:

目录结构:包含 .azureml 子目录的教程顶层

  • tutorial:项目的顶级目录。tutorial: Top-level directory of the project.
  • .azureml:用于存储 Azure 机器学习配置文件的隐藏子目录。.azureml: Hidden subdirectory for storing Azure Machine Learning configuration files.

例如,在 bash 窗口中创建此内容:For example, to create this in a bash window:

mkdir tutorial
cd tutorial
mkdir .azureml

提示

若要在图形窗口中创建或查看结构,首先要启用查看和创建隐藏文件和文件夹的功能:To create or view the structure in a graphical window, first enable the ability to see and create hidden files and folders:

  • 在 Mac 查找器窗口中使用 Command+Shift+.In a Mac Finder window use Command + Shift + . 切换显示隐藏文件/文件夹。to toggle the display of hidden files/folders.
  • 在 Windows 10 文件资源管理器中,请参阅如何查看隐藏的文件和文件夹In a Windows 10 File Explorer, see how to view hidden files and folders.
  • 在 Linux 图形界面中,使用 Ctrl+h 或“查看”菜单,并选中“显示隐藏的文件”复选框。 In the Linux Graphical Interface, use Ctrl + h or the View menu and check the box to Show hidden files.

创建 Azure 机器学习工作区Create an Azure Machine Learning workspace

工作区是 Azure 机器学习的顶级资源,可集中执行以下操作:A workspace is a top-level resource for Azure Machine Learning and is a centralized place to:

  • 管理资源(如计算)。Manage resources such as compute.
  • 存储资产,如笔记本、环境、数据集、管道、模型和终结点。Store assets like notebooks, environments, datasets, pipelines, models, and endpoints.
  • 与其他团队成员协作。Collaborate with other team members.

在顶级目录 tutorial 中,通过使用以下代码来添加名为“01-create-workspace.py”的新 Python 文件。In the top-level directory, tutorial, add a new Python file called 01-create-workspace.py by using the following code. 根据自己的需要对参数(名称、订阅 ID、资源组和位置)进行调整。Adapt the parameters (name, subscription ID, resource group, and location) with your preferences.

代码可以在交互式会话中运行,或者也可以作为 Python 文件运行。You can run the code in an interactive session or as a Python file.

备注

如果使用本地开发环境(例如你的计算机),在首次运行以下代码时,系统会提示你通过使用设备代码进行工作区身份验证。When you're using a local development environment (for example, your computer), you'll be asked to authenticate to your workspace by using a device code the first time you run the following code. 按照屏幕上的说明进行操作。Follow the on-screen instructions.

# tutorial/01-create-workspace.py
from azureml.core import Workspace

ws = Workspace.create(name='<my_workspace_name>', # provide a name for your workspace
                      subscription_id='<azure-subscription-id>', # provide your subscription ID
                      resource_group='<myresourcegroup>', # provide a resource group name
                      create_resource_group=True,
                      location='<NAME_OF_REGION>') 

# write out the workspace details to a configuration file: .azureml/config.json
ws.write_config(path='.azureml')

在已激活 tutorial1 conda 环境的窗口中,从 tutorial 目录运行此代码。In the window that has the activated tutorial1 conda environment, run this code from the tutorial directory.

cd <path/to/tutorial>
python ./01-create-workspace.py

提示

如果运行此代码返回错误“你没有访问订阅的权限”,请参阅创建工作区,以获取有关身份验证选项的信息。If running this code gives you an error that you do not have access to the subscription, see Create a workspace for information on authentication options.

成功运行 01-create-workspace.py 后,文件夹结构将如下所示:After you've successfully run 01-create-workspace.py, your folder structure will look like:

运行 01-create-workspace.py 后文件 config.json 显示在 .azureml 子目录中

文件 .azureml/config.json 包含连接到 Azure 机器学习工作区所需的元数据。The file .azureml/config.json contains the metadata necessary to connect to your Azure Machine Learning workspace. 也就是说,其中包含订阅 ID、资源组和工作区名称。Namely, it contains your subscription ID, resource group, and workspace name.

备注

config.json 的内容不是机密。The contents of config.json are not secrets. 可以共享这些详细信息。It's fine to share these details.

与 Azure 机器学习工作区交互仍需要进行身份验证。Authentication is still required to interact with your Azure Machine Learning workspace.

创建 Azure 机器学习计算群集Create an Azure Machine Learning compute cluster

在顶级目录 tutorial 中创建名为“02-create-compute.py”的 Python 脚本。Create a Python script in the tutorial top-level directory called 02-create-compute.py. 使用以下代码填充该脚本,以创建 Azure 机器学习计算群集,该群集将会在 0 到 4 个节点之间自动缩放:Populate it with the following code to create an Azure Machine Learning compute cluster that will autoscale between zero and four nodes:

# tutorial/02-create-compute.py
from azureml.core import Workspace
from azureml.core.compute import ComputeTarget, AmlCompute
from azureml.core.compute_target import ComputeTargetException

ws = Workspace.from_config() # This automatically looks for a directory .azureml

# Choose a name for your CPU cluster
cpu_cluster_name = "cpu-cluster"

# Verify that the cluster does not exist already
try:
    cpu_cluster = ComputeTarget(workspace=ws, name=cpu_cluster_name)
    print('Found existing cluster, use it.')
except ComputeTargetException:
    compute_config = AmlCompute.provisioning_configuration(vm_size='STANDARD_D2_V2',
                                                           idle_seconds_before_scaledown=2400,
                                                           min_nodes=0,
                                                           max_nodes=4)
    cpu_cluster = ComputeTarget.create(ws, cpu_cluster_name, compute_config)

cpu_cluster.wait_for_completion(show_output=True)

在已激活 tutorial1 conda 环境的窗口中,运行 Python 文件:In the window that has the activated tutorial1 conda environment, run the Python file:

python ./02-create-compute.py

备注

该群集在创建之后将会预配 0 个节点。When the cluster is created, it will have 0 nodes provisioned. 因此在你提交作业之前,该群集不会产生成本。The cluster does not incur costs until you submit a job. 此群集在空闲 2,400 秒(40 分钟)之后将会纵向缩减。This cluster will scale down when it has been idle for 2,400 seconds (40 minutes).

文件夹结构现在将如下所示:Your folder structure will now look as follows:

将 02-create-compute.py 添加到教程目录

在工作室中查看View in the studio

登录到 Azure 机器学习工作室,查看已创建的工作区和计算实例。Sign in to Azure Machine Learning studio to view the workspace and compute instance you created.

  1. 选择曾经用于创建工作区的订阅。Select the Subscription you used to create the workspace.
  2. 选择已创建的机器学习工作区“tutorial-ws”。Select the Machine Learning workspace you created, tutorial-ws.
  3. 加载工作区后,在左侧选择“计算”。Once the workspace loads, on the left side, select Compute.
  4. 在顶部,选择“计算群集”选项卡。At the top, select the Compute clusters tab.

屏幕截图:查看工作区中的计算实例。

此视图显示预配的计算群集,以及空闲节点、忙碌节点和未预配节点的数量。This view shows the provisioned compute cluster, along with the number of idle nodes, busy nodes, and unprovisioned nodes. 由于你尚未使用群集,因此所有节点当前都未预配。Since you haven't used the cluster yet, all the nodes are currently unprovisioned.

后续步骤Next steps

在本设置教程中,你已经:In this setup tutorial, you have:

  • 创建了 Azure 机器学习工作区。Created an Azure Machine Learning workspace.
  • 设置了本地开发环境。Set up your local development environment.
  • 创建了 Azure 机器学习计算群集。Created an Azure Machine Learning compute cluster.

在本教程的其他部分,你将学习:In the other parts of this tutorial you will learn:

  • 第 2 部分。Part 2. 通过使用适用于 Python 的 Azure 机器学习 SDK 在云中运行代码。Run code in the cloud by using the Azure Machine Learning SDK for Python.
  • 第 3 部分。Part 3. 管理用于模型训练的 Python 环境。Manage the Python environment that you use for model training.
  • 第 4 部分。Part 4. 将数据上传到 Azure,并在训练中使用该数据。Upload data to Azure and consume that data in training.

继续学习下一教程,了解如何将脚本提交到 Azure 机器学习计算群集。Continue to the next tutorial, to walk through submitting a script to the Azure Machine Learning compute cluster.