使用 Azure 机器学习 SDK 创建和运行机器学习管道Create and run machine learning pipelines with Azure Machine Learning SDK

本文介绍如何通过使用 Azure 机器学习 SDK 来创建并运行机器学习管道In this article, you learn how to create and run a machine learning pipeline by using the Azure Machine Learning SDK. 使用 ML 管道 来创建将各种不同 ML 阶段拼结在一起的工作流。Use ML pipelines to create a workflow that stitches together various ML phases. 然后发布该管道,以便以后访问或与他人共享。Then, publish that pipeline for later access or sharing with others. 跟踪 ML 管道,以了解模型在实际应用场合的表现并检测数据偏移。Track ML pipelines to see how your model is performing in the real world and to detect data drift. ML 管道非常适合用于批量评分方案,它们可以使用各种计算,重复使用步骤而不是重新运行步骤,以及与其他人共享 ML 工作流。ML pipelines are ideal for batch scoring scenarios, using various computes, reusing steps instead of rerunning them, as well as sharing ML workflows with others.

尽管可以使用一种称作 Azure 管道的不同类型的管道来实现 ML 任务的 CI/CD 自动化,但这种类型的管道并不会存储在工作区中。While you can use a different kind of pipeline called an Azure Pipeline for CI/CD automation of ML tasks, that type of pipeline is not stored in your workspace. 比较这些不同的管道Compare these different pipelines.

Azure 机器学习工作区的成员可以看到创建的 ML 管道。The ML pipelines you create are visible to the members of your Azure Machine Learning workspace.

ML 管道在计算目标上执行(请参阅 Azure 机器学习中的计算目标是什么)。ML pipelines execute on compute targets (see What are compute targets in Azure Machine Learning). 管道可以在支持的 Azure 存储文档位置读取和写入数据。Pipelines can read and write data to and from supported Azure Storage locations.

如果没有 Azure 订阅,请在开始前创建一个试用帐户。If you don't have an Azure subscription, create a trial account before you begin. 试用试用版 Azure 机器学习Try the trial version of Azure Machine Learning.

先决条件Prerequisites

首先附加工作区:Start by attaching your workspace:

import azureml.core
from azureml.core import Workspace, Datastore

ws = Workspace.from_config()

设置机器学习资源Set up machine learning resources

创建运行 ML 管道所需的资源:Create the resources required to run an ML pipeline:

  • 设置一个数据存储,用于访问管道步骤中所需的数据。Set up a datastore used to access the data needed in the pipeline steps.

  • 配置 Dataset 对象,使之指向驻留在数据存储中的或可在数据存储中访问的持久性数据。Configure a Dataset object to point to persistent data that lives in, or is accessible in, a datastore. 为在管道步骤间传递的临时数据配置 PipelineData 对象。Configure a PipelineData object for temporary data passed between pipeline steps.

    提示

    公共预览版类 OutputFileDatasetConfig 改进了在管道步骤之间传递临时数据的体验。An improved experience for passing temporary data between pipeline steps is available in the public preview class, OutputFileDatasetConfig. 此类是一个试验性预览功能,可能会随时更改。This class is an experimental preview feature, and may change at any time.

  • 设置计算目标,管道步骤将在其上运行。Set up the compute targets on which your pipeline steps will run.

设置数据存储Set up a datastore

数据存储用于存储可供管道访问的数据。A datastore stores the data for the pipeline to access. 每个工作区有一个默认的数据存储。Each workspace has a default datastore. 可以注册其他数据存储。You can register additional datastores.

创建工作区时,会将 Azure 文件存储Azure Blob 存储附加到该工作区。When you create your workspace, Azure Files and Azure Blob storage are attached to the workspace. 已注册一个用于连接 Azure Blob 存储的默认数据存储。A default datastore is registered to connect to the Azure Blob storage. 要了解详细信息,请参阅确定何时使用 Azure 文件存储、Azure Blob 或 Azure 磁盘To learn more, see Deciding when to use Azure Files, Azure Blobs, or Azure Disks.

# Default datastore 
def_data_store = ws.get_default_datastore()

# Get the blob storage associated with the workspace
def_blob_store = Datastore(ws, "workspaceblobstore")

# Get file storage associated with the workspace
def_file_store = Datastore(ws, "workspacefilestore")

步骤通常使用数据并生成输出数据。Steps generally consume data and produce output data. 步骤可以创建数据,例如模型、包含模型和依赖文件的目录,或临时数据。A step can create data such as a model, a directory with model and dependent files, or temporary data. 然后,此数据可供管道中的其他后续步骤使用。This data is then available for other steps later in the pipeline. 若要详细了解如何将管道连接到数据,请参阅如何访问数据如何注册数据集这两篇文章。To learn more about connecting your pipeline to your data, see the articles How to Access Data and How to Register Datasets.

使用 DatasetPipelineData 对象配置数据Configure data with Dataset and PipelineData objects

向管道提供数据的首选方法是使用 Dataset 对象。The preferred way to provide data to a pipeline is a Dataset object. Dataset 对象指向驻留在数据存储中的或者可从数据存储或 Web URL 访问的数据。The Dataset object points to data that lives in or is accessible from a datastore or at a Web URL. Dataset 类是抽象类,因此,你将创建一个 FileDataset 的实例(引用一个或多个文件)或一个 TabularDataset 的实例(基于一个或多个包含分隔数据列的文件创建)。The Dataset class is abstract, so you will create an instance of either a FileDataset (referring to one or more files) or a TabularDataset that's created by from one or more files with delimited columns of data.

使用 from_filesfrom_delimited_files 之类的方法来创建 DatasetYou create a Dataset using methods like from_files or from_delimited_files.

from azureml.core import Dataset

my_dataset = Dataset.File.from_files([(def_blob_store, 'train-images/')])

中间数据(或步骤输出)由 PipelineData 对象表示。Intermediate data (or output of a step) is represented by a PipelineData object. output_data1 生成为步骤的输出,并用作一个或多个后续步骤的输入。output_data1 is produced as the output of a step, and used as the input of one or more future steps. PipelineData 在步骤之间引入数据依赖项,并在管道中创建隐式执行顺序。PipelineData introduces a data dependency between steps, and creates an implicit execution order in the pipeline. 稍后在创建管道步骤时将使用此对象。This object will be used later when creating pipeline steps.

from azureml.pipeline.core import PipelineData

output_data1 = PipelineData(
    "output_data1",
    datastore=def_blob_store,
    output_name="output_data1")

提示

也可以通过公共预览版类保留 OutputFileDatasetConfig 管道步骤之间的中间数据。Persisting intermediate data between pipeline steps is also possible with the public preview class, OutputFileDatasetConfig. 有关使用 OutputFileDatasetConfig 类的代码示例,请参阅如何生成两步 ML 管道For a code example using the OutputFileDatasetConfig class, see how to build a two step ML pipeline.

设置计算目标Set up a compute target

在 Azure 机器学习中,术语“计算”(或“计算目标”)是指在机器学习管道中执行计算步骤的计算机或群集 。In Azure Machine Learning, the term compute (or compute target) refers to the machines or clusters that perform the computational steps in your machine learning pipeline. 请参阅用于模型训练的计算目标以获取计算目标的完整列表,并参阅创建计算目标以了解如何创建计算目标并将其附加到工作区。See compute targets for model training for a full list of compute targets and Create compute targets for how to create and attach them to your workspace. 无论是要训练模型还是要运行管道步骤,创建和/或附加计算目标的过程都是相同的。The process for creating and or attaching a compute target is the same whether you are training a model or running a pipeline step. 创建并附加计算目标后,请使用管道步骤中的 ComputeTarget 对象。After you create and attach your compute target, use the ComputeTarget object in your pipeline step.

重要

内部远程作业不支持对计算目标执行管理操作。Performing management operations on compute targets is not supported from inside remote jobs. 由于机器学习管道作为远程作业提交,因此请勿对管道内的计算目标使用管理操作。Since machine learning pipelines are submitted as a remote job, do not use management operations on compute targets from inside the pipeline.

Azure 机器学习计算Azure Machine Learning compute

可以创建 Azure 机器学习计算用于运行步骤。You can create an Azure Machine Learning compute for running your steps. 其他计算目标的代码非常相似,参数会因类型的不同而略有差异。The code for other compute targets is very similar, with slightly different parameters, depending on the type.

from azureml.core.compute import ComputeTarget, AmlCompute

compute_name = "aml-compute"
vm_size = "STANDARD_NC6"
if compute_name in ws.compute_targets:
    compute_target = ws.compute_targets[compute_name]
    if compute_target and type(compute_target) is AmlCompute:
        print('Found compute target: ' + compute_name)
else:
    print('Creating a new compute target...')
    provisioning_config = AmlCompute.provisioning_configuration(vm_size=vm_size,  # STANDARD_NC6 is GPU-enabled
                                                                min_nodes=0,
                                                                max_nodes=4)
    # create the compute target
    compute_target = ComputeTarget.create(
        ws, compute_name, provisioning_config)

    # Can poll for a minimum number of nodes and for a specific timeout.
    # If no min node count is provided it will use the scale settings for the cluster
    compute_target.wait_for_completion(
        show_output=True, min_node_count=None, timeout_in_minutes=20)

    # For a more detailed view of current cluster status, use the 'status' property
    print(compute_target.status.serialize())

配置训练运行的环境Configure the training run's environment

下一步是确保远程训练运行包含训练步骤所需的所有依赖项。The next step is making sure that the remote training run has all the dependencies needed by the training steps. 通过创建和配置 RunConfiguration 对象来设置依赖项和运行时上下文。Dependencies and the runtime context are set by creating and configuring a RunConfiguration object.

from azureml.core.runconfig import RunConfiguration
from azureml.core.conda_dependencies import CondaDependencies
from azureml.core import Environment 

aml_run_config = RunConfiguration()
# `compute_target` as defined in "Azure Machine Learning compute" section above
aml_run_config.target = compute_target

USE_CURATED_ENV = True
if USE_CURATED_ENV :
    curated_environment = Environment.get(workspace=ws, name="AzureML-Tutorial")
    aml_run_config.environment = curated_environment
else:
    aml_run_config.environment.python.user_managed_dependencies = False
    
    # Add some packages relied on by data prep step
    aml_run_config.environment.python.conda_dependencies = CondaDependencies.create(
        conda_packages=['pandas','scikit-learn'], 
        pip_packages=['azureml-sdk', 'azureml-dataprep[fuse,pandas]'], 
        pin_sdk_version=False)

以上代码显示了处理依赖项的两个选项。The code above shows two options for handling dependencies. 如前所述,当 USE_CURATED_ENV = True,配置基于特选环境。As presented, with USE_CURATED_ENV = True, the configuration is based on a curated environment. 特选环境中“预先准备”有常见的互依赖库,可以大大加快联机速度。Curated environments are "prebaked" with common inter-dependent libraries and can be significantly faster to bring online. 特选环境在 Microsoft 容器注册表中具有预先生成的 Docker 映像。Curated environments have prebuilt Docker images in the Microsoft Container Registry. 有关详细信息,请参阅 Azure 机器学习特选环境For more information, see Azure Machine Learning curated environments.

USE_CURATED_ENV 更改为 False 所采用的路径显示了显式设置依赖项的模式。The path taken if you change USE_CURATED_ENV to False shows the pattern for explicitly setting your dependencies. 在这种情况下,将在资源组内的 Azure 容器注册表中创建和注册新的自定义 Docker 映像(请参阅 Azure 中的专用 Docker 容器注册表简介)。In that scenario, a new custom Docker image will be created and registered in an Azure Container Registry within your resource group (see Introduction to private Docker container registries in Azure). 创建和注册此映像可能需要几分钟的时间。Building and registering this image can take quite a few minutes.

构造管道步骤Construct your pipeline steps

创建计算资源和环境后,即可定义管道的步骤。Once you have the compute resource and environment created, you are ready to define your pipeline's steps. Azure 机器学习 SDK 提供了许多内置步骤,如 azureml.pipeline.steps 包的参考文档中所示。There are many built-in steps available via the Azure Machine Learning SDK, as you can see on the reference documentation for the azureml.pipeline.steps package. 最灵活的类是运行 Python 脚本的 PythonScriptStepThe most flexible class is PythonScriptStep, which runs a Python script.

from azureml.pipeline.steps import PythonScriptStep

dataprep_source_dir = "./dataprep_src"
entry_point = "prepare.py"

# `my_dataset` as defined above
ds_input = my_dataset.as_named_input('input1')

# `output_data1`, `compute_target`, `aml_run_config` as defined above
data_prep_step = PythonScriptStep(
    script_name=entry_point,
    source_directory=dataprep_source_dir,
    arguments=["--input", ds_input.as_download(), "--output", output_data1],
    inputs=[ds_input],
    outputs=[output_data1],
    compute_target=compute_target,
    runconfig=aml_run_config,
    allow_reuse=True
)

上述代码展示了典型的初始管道步骤。The above code shows a typical initial pipeline step. 数据准备代码位于子目录中(在此示例中,"prepare.py" 位于目录 "./dataprep.src" 中)。Your data preparation code is in a subdirectory (in this example, "prepare.py" in the directory "./dataprep.src"). 在管道创建的过程中,此目录将被压缩并上传到 compute_target,该步骤会运行指定为 script_name 的值的脚本。As part of the pipeline creation process, this directory is zipped and uploaded to the compute_target and the step runs the script specified as the value for script_name.

argumentsinputsoutputs 值指定该步骤的输入和输出。The arguments, inputs, and outputs values specify the inputs and outputs of the step. 在上面的示例中,基线数据是 my_dataset 数据集。In the example above, the baseline data is the my_dataset dataset. 相应数据会下载到计算资源,因为代码将其指定为 as_download()The corresponding data will be downloaded to the compute resource since the code specifies it as as_download(). 脚本 prepare.py 执行与手头任务相应的任何数据转换任务,并将数据输出到 PipelineData 类型的 output_data1The script prepare.py does whatever data-transformation tasks are appropriate to the task at hand and outputs the data to output_data1, of type PipelineData. 有关详细信息,请参阅将数据移入 ML 管道和在 ML 管道之间移动数据的步骤 (Python)For more information, see Moving data into and between ML pipeline steps (Python).

该步骤将使用配置 aml_run_config 在由 compute_target 定义的计算机上运行。The step will run on the machine defined by compute_target, using the configuration aml_run_config.

在协作环境中使用管道时,重复使用以前的结果 (allow_reuse) 非常关键,因为消除不必要的重新运行可以提高敏捷性。Reuse of previous results (allow_reuse) is key when using pipelines in a collaborative environment since eliminating unnecessary reruns offers agility. 当步骤的 script_name、输入和参数保持不变时,“重复使用”是默认行为。Reuse is the default behavior when the script_name, inputs, and the parameters of a step remain the same. 允许重复使用时,上一次运行的结果将立即发送到下一步骤。When reuse is allowed, results from the previous run are immediately sent to the next step. 如果 allow_reuse 设置为 False,则在管道执行过程中将始终为此步骤生成新的运行。If allow_reuse is set to False, a new run will always be generated for this step during pipeline execution.

虽然可以通过单个步骤创建管道,但你几乎始终会选择将整个过程拆分为几个步骤。It's possible to create a pipeline with a single step, but almost always you'll choose to split your overall process into several steps. 例如,你可能有用于数据准备、训练、模型比较和部署的步骤。For instance, you might have steps for data preparation, training, model comparison, and deployment. 例如,可以想象,在上面指定 data_prep_step 之后,下一步可能是训练:For instance, one might imagine that after the data_prep_step specified above, the next step might be training:

train_source_dir = "./train_src"
train_entry_point = "train.py"

training_results = PipelineData(name = "training_results", 
                                datastore=def_blob_store,
                                output_name="training_results")

    
train_step = PythonScriptStep(
    script_name=train_entry_point,
    source_directory=train_source_dir,
    arguments=["--prepped_data", output_data1.as_input(), "--training_results", training_results],
    compute_target=compute_target,
    runconfig=aml_run_config,
    allow_reuse=True
)

上述代码与数据准备步骤的代码非常相似。The above code is very similar to that for the data preparation step. 训练代码位于与数据准备代码不同的目录中。The training code is in a directory separate from that of the data preparation code. 数据准备步骤的 PipelineData 输出 output_data1 用作训练步骤的输入。The PipelineData output of the data preparation step, output_data1 is used as the input to the training step. 将创建一个新的 PipelineData 对象 training_results用于保存结果,以便进行后续比较或部署步骤。A new PipelineData object, training_results is created to hold the results for a subsequent comparison or deployment step.

提示

为了获得更好的体验,以及为了能够在管道运行结束时将中间数据写回到数据存储,请使用公共预览版类 OutputFileDatasetConfigFor an improved experience, and the ability to write intermediate data back to your datastores at the end of your pipeline run, use the public preview class, OutputFileDatasetConfig. 有关代码示例,请参阅如何生成两步 ML 管道如何在运行完成时将数据写回数据存储For code examples, see how to build a two step ML pipeline and how to write data back to datastores upon run completion.

定义步骤后,使用其中的部分或所有步骤生成管道。After you define your steps, you build the pipeline by using some or all of those steps.

备注

定义步骤或生成管道时,不会将任何文件或数据上传到 Azure 机器学习。No file or data is uploaded to Azure Machine Learning when you define the steps or build the pipeline. 文件会在你调用 Experiment.submit() 时上传。The files are uploaded when you call Experiment.submit().

# list of steps to run (`compare_step` definition not shown)
compare_models = [data_prep_step, train_step, compare_step]

from azureml.pipeline.core import Pipeline

# Build the pipeline
pipeline1 = Pipeline(workspace=ws, steps=[compare_models])

Python 环境如何处理管道参数How Python environments work with pipeline parameters

如先前在配置训练运行的环境中所讨论的一样,环境状态和 Python 库依赖项是使用 Environment 对象指定的。As discussed previously in Configure the training run's environment, environment state and Python library dependencies are specified using an Environment object. 通常,可以通过引用现有 Environment 的名称和版本(可选)来指定它:Generally, you can specify an existing Environment by referring to its name and, optionally, a version:

aml_run_config = RunConfiguration()
aml_run_config.environment.name = 'MyEnvironment'
aml_run_config.environment.version = '1.0'

但是,如果选择使用 PipelineParameter 对象在运行时为管道步骤动态设置变量,则不能使用这种引用现有 Environment 的方法。However, if you choose to use PipelineParameter objects to dynamically set variables at runtime for your pipeline steps, you cannot use this technique of referring to an existing Environment. 而如果想要使用 PipelineParameter 对象,则必须将 RunConfigurationenvironment 字段设置为 Environment 对象。Instead, if you want to use PipelineParameter objects, you must set the environment field of the RunConfiguration to an Environment object. 你需要负责确保此类 Environment 正确设置了它对外部 Python 包的依赖项。It is your responsibility to ensure that such an Environment has its dependencies on external Python packages properly set.

使用数据集Use a dataset

从 Azure Blob 存储、Azure 文件存储、Azure Data Lake Storage Gen1、Azure Data Lake Storage Gen2、Azure SQL 数据库和 Azure Database for PostgreSQL 创建的数据集可以用作任何管道步骤的输入。Datasets created from Azure Blob storage, Azure Files, Azure Data Lake Storage Gen1, Azure Data Lake Storage Gen2, Azure SQL Database, and Azure Database for PostgreSQL can be used as input to any pipeline step. 可以将输出写入 DataTransferStepDatabricksStep,如果要将数据写入特定数据存储,请使用 PipelineDataYou can write output to a DataTransferStep, DatabricksStep, or if you want to write data to a specific datastore use PipelineData.

重要

仅 Azure Blob 和 Azure 文件共享数据存储支持使用 PipelineData 将输出数据写回到数据存储。Writing output data back to a datastore using PipelineData is only supported for Azure Blob and Azure File share datastores.

若要将输出数据写回到 Azure Blob、Azure 文件共享、ADLS Gen2 数据存储,请使用公共预览版类 OutputFileDatasetConfigTo write output data back to Azure Blob, Azure File share, ADLS Gen 2 datastores use the public preview class, OutputFileDatasetConfig.

dataset_consuming_step = PythonScriptStep(
    script_name="iris_train.py",
    inputs=[iris_tabular_dataset.as_named_input("iris_data")],
    compute_target=compute_target,
    source_directory=project_folder
)

然后,使用 Run.input_datasets 字典检索管道中的数据集。You then retrieve the dataset in your pipeline by using the Run.input_datasets dictionary.

# iris_train.py
from azureml.core import Run, Dataset

run_context = Run.get_context()
iris_dataset = run_context.input_datasets['iris_data']
dataframe = iris_dataset.to_pandas_dataframe()

Run.get_context() 行值得强调。The line Run.get_context() is worth highlighting. 此函数检索 Run(表示当前试验运行)。This function retrieves a Run representing the current experimental run. 在上面的示例中,我们使用它来检索已注册数据集。In the above sample, we use it to retrieve a registered dataset. Run 对象的另一个常见用途是检索试验本身和试验所在的工作区:Another common use of the Run object is to retrieve both the experiment itself and the workspace in which the experiment resides:

# Within a PythonScriptStep

ws = Run.get_context().experiment.workspace

有关更多详细信息(包括传递数据和访问数据的替代方法),请参阅将数据移入 ML 管道步骤和在 ML 管道步骤之间移动数据 (Python)For more detail, including alternate ways to pass and access data, see Moving data into and between ML pipeline steps (Python).

缓存和重复使用Caching & reuse

若要优化和自定义管道的行为,可以围绕缓存和重复使用采取某些措施。In order to optimize and customize the behavior of your pipelines, you can do a few things around caching and reuse. 例如,可以选择:For example, you can choose to:

  • 步骤定义期间通过设置 allow_reuse=False禁用默认的重复使用步骤运行输出的行为Turn off the default reuse of the step run output by setting allow_reuse=False during step definition. 在协作环境中使用管道时,“重复使用”非常关键,因为消除不必要的运行可以提高敏捷性。Reuse is key when using pipelines in a collaborative environment since eliminating unnecessary runs offers agility. 但是,可以选择禁用重复使用。However, you can opt out of reuse.
  • 使用 pipeline_run = exp.submit(pipeline, regenerate_outputs=False) 强制对运行中的所有步骤重新生成输出Force output regeneration for all steps in a run with pipeline_run = exp.submit(pipeline, regenerate_outputs=False)

默认情况下,已启用步骤的 allow_reuse,步骤定义中指定的 source_directory 将进行哈希处理。By default, allow_reuse for steps is enabled and the source_directory specified in the step definition is hashed. 因此,如果给定步骤的脚本保持不变(script_name、输入和参数),并且 source_directory 中未发生任何其他更改,则会重复使用前一个步骤运行的输出,不会将作业提交到计算,并且前一运行的结果立即可供下一步骤使用。So, if the script for a given step remains the same (script_name, inputs, and the parameters), and nothing else in the source_directory has changed, the output of a previous step run is reused, the job is not submitted to the compute, and the results from the previous run are immediately available to the next step instead.

step = PythonScriptStep(name="Hello World",
                        script_name="hello_world.py",
                        compute_target=aml_compute,
                        source_directory=source_directory,
                        allow_reuse=False,
                        hash_paths=['hello_world.ipynb'])

提交管道Submit the pipeline

提交管道时,Azure 机器学习将检查每个步骤的依赖项,并上传指定的源目录的快照。When you submit the pipeline, Azure Machine Learning checks the dependencies for each step and uploads a snapshot of the source directory you specified. 如果未指定源目录,则上传当前的本地目录。If no source directory is specified, the current local directory is uploaded. 该快照也作为试验的一部分存储在工作区中。The snapshot is also stored as part of the experiment in your workspace.

重要

为了防止在快照中包含不必要的文件,请在目录中创建 ignore 文件(.gitignore.amlignore)。To prevent unnecessary files from being included in the snapshot, make an ignore file (.gitignore or .amlignore) in the directory. 将要排除的文件和目录添加到此文件中。Add the files and directories to exclude to this file. 有关此文件中使用的语法的详细信息,请参阅 .gitignore语法和模式For more information on the syntax to use inside this file, see syntax and patterns for .gitignore. .amlignore 文件使用相同的语法。The .amlignore file uses the same syntax. 如果这两个文件都存在,则 .amlignore 文件的优先级更高。If both files exist, the .amlignore file takes precedence.

有关详细信息,请参阅快照For more information, see Snapshots.

from azureml.core import Experiment

# Submit the pipeline to be run
pipeline_run1 = Experiment(ws, 'Compare_Models_Exp').submit(pipeline1)
pipeline_run1.wait_for_completion()

第一次运行管道时,Azure 机器学习会:When you first run a pipeline, Azure Machine Learning:

  • 将项目快照从与工作区关联的 Blob 存储下载到计算目标。Downloads the project snapshot to the compute target from the Blob storage associated with the workspace.
  • 生成对应于管道中每个步骤的 Docker 映像。Builds a Docker image corresponding to each step in the pipeline.
  • 将每个步骤的 Docker 映像从容器注册表下载到计算目标。Downloads the Docker image for each step to the compute target from the container registry.
  • 配置对 DatasetPipelineData 对象的访问权限。Configures access to Dataset and PipelineData objects. as_mount() 访问模式下,FUSE 用于提供虚拟访问。For as as_mount() access mode, FUSE is used to provide virtual access. 如果不支持装载,或者用户将访问权限指定为 as_download(),则改为将数据复制到计算目标。If mount is not supported or if the user specified access as as_download(), the data is instead copied to the compute target.
  • 运行在步骤定义中指定的计算目标中的步骤。Runs the step in the compute target specified in the step definition.
  • 创建项目,例如日志、stdout 和 stderr、指标以及步骤指定的输出。Creates artifacts, such as logs, stdout and stderr, metrics, and output specified by the step. 然后上传这些项目并将其保存在用户的默认数据存储中。These artifacts are then uploaded and kept in the user's default datastore.

以管道方式运行实验的图

有关详细信息,请参阅 Experiment 类参考。For more information, see the Experiment class reference.

对推理时更改的参数使用管道参数Use pipeline parameters for arguments that change at inference time

查看管道的结果View results of a pipeline

在工作室中查看所有管道的列表及其运行详细信息:See the list of all your pipelines and their run details in the studio:

  1. 登录到 Azure 机器学习工作室Sign in to Azure Machine Learning studio.

  2. 查看工作区View your workspace.

  3. 在左侧,选择“管道”以查看所有管道运行。On the left, select Pipelines to see all your pipeline runs. 机器学习管道列表list of machine learning pipelines

  4. 选择特定的管道以查看运行结果。Select a specific pipeline to see the run results.

Git 跟踪与集成Git tracking and integration

如果以本地 Git 存储库作为源目录开始训练运行,有关存储库的信息将存储在运行历史记录中。When you start a training run where the source directory is a local Git repository, information about the repository is stored in the run history. 有关详细信息,请参阅 Azure 机器学习的 Git 集成For more information, see Git integration for Azure Machine Learning.

后续步骤Next steps