使用 Azure 机器学习部署模型Deploy models with Azure Machine Learning

适用于:是基本版是企业版               (升级到企业版APPLIES TO: yesBasic edition yesEnterprise edition                    (Upgrade to Enterprise edition)

了解如何将机器学习模型作为 Web 服务部署在 Azure 云或 Azure IoT Edge 设备中。Learn how to deploy your machine learning model as a web service in the Azure cloud or to Azure IoT Edge devices.

无论你在何处部署模型,工作流都是类似的:The workflow is similar no matter where you deploy your model:

  1. 注册模型。Register the model.
  2. 准备推理配置Prepare an inference configuration
  3. 准备入口脚本(使用无代码部署的情况除外)Prepare an entry script (unless using no-code deployment)
  4. 将模型部署到计算目标。Deploy the model to the compute target.
  5. 测试已部署的模型(也称为“Web 服务”)。Test the deployed model, also called a web service.

有关部署工作流涉及的概念的详细信息,请参阅使用 Azure 机器学习管理、部署和监视模型For more information on the concepts involved in the deployment workflow, see Manage, deploy, and monitor models with Azure Machine Learning.

先决条件Prerequisites

连接到工作区Connect to your workspace

按照 Azure CLI 文档中的说明设置订阅上下文Follow the directions in the Azure CLI documentation for setting your subscription context.

然后执行以下命令:Then do:

az ml workspace list --resource-group=<my resource group>

以便查看有权访问的工作区。to see the workspaces you have access to.

注册模型Register your model

已注册的模型是组成模型的一个或多个文件的逻辑容器。A registered model is a logical container for one or more files that make up your model. 例如,如果有一个存储在多个文件中的模型,则可以在工作区中将这些文件注册为单个模型。For example, if you have a model that's stored in multiple files, you can register them as a single model in the workspace. 注册这些文件后,可以下载或部署已注册的模型,并接收注册的所有文件。After you register the files, you can then download or deploy the registered model and receive all the files that you registered.

提示

注册模型时,请提供云位置(来自训练运行)或本地目录的路径。When you register a model, you provide the path of either a cloud location (from a training run) or a local directory. 此路径仅用于在注册过程中查找要上传的文件。This path is just to locate the files for upload as part of the registration process. 它不需要与入口脚本中使用的路径匹配。It doesn't need to match the path used in the entry script. 有关详细信息,请参阅在入口脚本中查找模型文件For more information, see Locate model files in your entry script.

机器学习模型会注册到 Azure 机器学习工作区中。Machine learning models are registered in your Azure Machine Learning workspace. 模型可以来自 Azure 机器学习或其他位置。The model can come from Azure Machine Learning or from somewhere else. 注册模型时,可以选择提供有关模型的元数据。When registering a model, you can optionally provide metadata about the model. 然后,可将应用于模型注册的 tagsproperties 字典用于筛选模型。The tags and properties dictionaries that you apply to a model registration can then be used to filter models.

以下示例演示如何注册模型。The following examples demonstrate how to register a model.

通过 Azure ML 训练运行注册一个模型Register a model from an Azure ML training run

az ml model register -n sklearn_mnist  --asset-path outputs/sklearn_mnist_model.pkl  --experiment-name myexperiment --run-id myrunid --tag area=mnist

提示

如果收到一条错误消息,指出未安装 ml 扩展,请使用以下命令进行安装:If you get an error message stating that the ml extension isn't installed, use the following command to install it:

az extension add -n azure-cli-ml

--asset-path 参数表示模型的云位置。The --asset-path parameter refers to the cloud location of the model. 本示例使用的是单个文件的路径。In this example, the path of a single file is used. 若要在模型注册中包含多个文件,请将 --asset-path 设置为包含文件的文件夹的路径。To include multiple files in the model registration, set --asset-path to the path of a folder that contains the files.

通过本地文件注册模型Register a model from a local file

az ml model register -n onnx_mnist -p mnist/model.onnx

若要在模型注册中包含多个文件,请将 -p 设置为包含文件的文件夹的路径。To include multiple files in the model registration, set -p to the path of a folder that contains the files.

有关 az ml model register 的详细信息,请参阅参考文档For more information on az ml model register, consult the reference documentation.

定义入口脚本Define an entry script

The entry script receives data submitted to a deployed web service and passes it to the model. It then takes the response returned by the model and returns that to the client. The script is specific to your model. It must understand the data that the model expects and returns.

The script contains two functions that load and run the model:

  • init(): Typically, this function loads the model into a global object. This function is run only once, when the Docker container for your web service is started.

  • run(input_data): This function uses the model to predict a value based on the input data. Inputs and outputs of the run typically use JSON for serialization and deserialization. You can also work with raw binary data. You can transform the data before sending it to the model or before returning it to the client.

The REST API expects the body of the request to be a JSON document with the following structure:

{
    "data":
        [
            <model-specific-data-structure>
        ]
}

The following example demonstrates how to load a registered scikit-learn model and score it with numpy data:

#Example: scikit-learn and Swagger
import json
import numpy as np
import os
from sklearn.externals import joblib
from sklearn.linear_model import Ridge

from inference_schema.schema_decorators import input_schema, output_schema
from inference_schema.parameter_types.numpy_parameter_type import NumpyParameterType


def init():
    global model
    # AZUREML_MODEL_DIR is an environment variable created during deployment. Join this path with the filename of the model file.
    # It holds the path to the directory that contains the deployed model (./azureml-models/$MODEL_NAME/$VERSION).
    # If there are multiple models, this value is the path to the directory containing all deployed models (./azureml-models).
    model_path = os.path.join(os.getenv('AZUREML_MODEL_DIR'), 'sklearn_mnist_model.pkl')

    # If your model were stored in the same directory as your score.py, you could also use the following:
    # model_path = os.path.abspath(os.path.join(os.path.dirname(__file_), 'sklearn_mnist_model.pkl')

    # Deserialize the model file back into a sklearn model
    model = joblib.load(model_path)


input_sample = np.array([[10, 9, 8, 7, 6, 5, 4, 3, 2, 1]])
output_sample = np.array([3726.995])


@input_schema('data', NumpyParameterType(input_sample))
@output_schema(NumpyParameterType(output_sample))
def run(data):
    try:
        result = model.predict(data)
        # You can return any data type, as long as it is JSON serializable.
        return result.tolist()
    except Exception as e:
        error = str(e)
        return error

For more examples, see the following scripts:

定义推理配置Define an inference configuration

inferenceconfig.json 文档中的条目映射到 InferenceConfig 类的参数。The entries in the inferenceconfig.json document map to the parameters for the InferenceConfig class. 下表描述了 JSON 文档中的实体与方法参数之间的映射:The following table describes the mapping between entities in the JSON document and the parameters for the method:

JSON 实体JSON entity 方法参数Method parameter 说明Description
entryScript entry_script 包含要为映像运行的代码的本地文件路径。Path to a local file that contains the code to run for the image.
runtime runtime 可选。Optional. 用于映像的运行时。Which runtime to use for the image. 支持的运行时为 spark-pypythonSupported runtimes are spark-py and python. 如果设置 environment,则会忽略此项。If environment is set, this entry is ignored.
condaFile conda_file 可选。Optional. 包含要用于映像的 Conda 环境定义的本地文件路径。Path to a local file that contains a Conda environment definition to use for the image. 如果设置 environment,则会忽略此项。If environment is set, this entry is ignored.
extraDockerFileSteps extra_docker_file_steps 可选。Optional. 包含设置映像时要运行的附加 Docker 步骤的本地文件路径。Path to a local file that contains additional Docker steps to run when setting up the image. 如果设置 environment,则会忽略此项。If environment is set, this entry is ignored.
sourceDirectory source_directory 可选。Optional. 包含用于创建映像的所有文件的文件夹路径,因此可以轻松地访问此文件夹或子文件夹中的任何文件。Path to folders that contain all files to create the image, which makes it easy to access any files within this folder or subfolder. 可以从本地计算机上传整个文件夹,作为 Webservice 的依赖项。You can upload an entire folder from your local machine as dependencies for the Webservice. 注意:entry_script、conda_file 和 extra_docker_file_steps 路径是 source_directory 路径的相对路径。Note: your entry_script, conda_file, and extra_docker_file_steps paths are relative paths to the source_directory path.
enableGpu enable_gpu 可选。Optional. 是否在映像中启用 GPU 支持。Whether to enable GPU support in the image. GPU 映像必须在 Azure 服务上运行,如 Azure 容器实例。The GPU image must be used on an Azure service, like Azure Container Instances. 例如,Azure 机器学习计算、Azure 虚拟机和 Azure Kubernetes 服务。For example, Azure Machine Learning Compute, Azure Virtual Machines, and Azure Kubernetes Service. 默认值为 False。The default is False. 如果设置 environment,则会忽略此项。If environment is set, this entry is ignored.
baseImage base_image 可选。Optional. 要用作基础映像的自定义映像。Custom image to be used as a base image. 如果未提供基础映像,则该映像将基于提供的运行时参数。If no base image is provided, the image will be based on the provided runtime parameter. 如果设置 environment,则会忽略此项。If environment is set, this entry is ignored.
baseImageRegistry base_image_registry 可选。Optional. 包含基础映像的映像注册表。Image registry that contains the base image. 如果设置 environment,则会忽略此项。If environment is set, this entry is ignored.
cudaVersion cuda_version 可选。Optional. 要为需要 GPU 支持的映像安装的 CUDA 版本。Version of CUDA to install for images that need GPU support. 必须在 Azure 服务上使用 GPU 映像。The GPU image must be used on an Azure service. 例如,Azure 容器实例、Azure 机器学习计算、Azure 虚拟机和 Azure Kubernetes 服务。For example, Azure Container Instances, Azure Machine Learning Compute, Azure Virtual Machines, and Azure Kubernetes Service. 支持的版本为 9.0、9.1 和 10.0。Supported versions are 9.0, 9.1, and 10.0. 如果设置 enable_gpu,则默认为 9.1。If enable_gpu is set, the default is 9.1. 如果设置 environment,则会忽略此项。If environment is set, this entry is ignored.
description description 映像的说明。A description for the image. 如果设置 environment,则会忽略此项。If environment is set, this entry is ignored.
environment environment 可选。Optional. Azure 机器学习环境Azure Machine Learning environment.

以下 JSON 是用于 CLI 的推理配置示例:The following JSON is an example inference configuration for use with the CLI:

{
    "entryScript": "score.py",
    "runtime": "python",
    "condaFile": "myenv.yml",
    "extraDockerfileSteps": null,
    "sourceDirectory": null,
    "enableGpu": false,
    "baseImage": null,
    "baseImageRegistry": null
}

可以在推理配置文件中包含 Azure 机器学习环境的完整规范。You can include full specifications of an Azure Machine Learning environment in the inference configuration file. 如果工作区中不存在此环境,Azure 机器学习会创建它。If this environment doesn't exist in your workspace, Azure Machine Learning will create it. 否则,Azure 机器学习会更新环境(如有必要)。Otherwise, Azure Machine Learning will update the environment if necessary. 以下 JSON 是一个示例:The following JSON is an example:

{
    "entryScript": "score.py",
    "environment": {
        "docker": {
            "arguments": [],
            "baseDockerfile": null,
            "baseImage": "mcr.microsoft.com/azureml/base:intelmpi2018.3-ubuntu16.04",
            "enabled": false,
            "sharedVolumes": true,
            "shmSize": null
        },
        "environmentVariables": {
            "EXAMPLE_ENV_VAR": "EXAMPLE_VALUE"
        },
        "name": "my-deploy-env",
        "python": {
            "baseCondaEnvironment": null,
            "condaDependencies": {
                "channels": [
                    "conda-forge"
                ],
                "dependencies": [
                    "python=3.6.2",
                    {
                        "pip": [
                            "azureml-defaults",
                            "azureml-telemetry",
                            "scikit-learn",
                            "inference-schema[numpy-support]"
                        ]
                    }
                ],
                "name": "project_environment"
            },
            "condaDependenciesFile": null,
            "interpreterPath": "python",
            "userManagedDependencies": false
        },
        "version": "1"
    }
}

还可以在单独的 CLI 参数中使用现有 Azure 机器学习环境,并从推理配置文件中删除“环境”密钥。You can also use an existing Azure Machine Learning environment in separated CLI parameters and remove the "environment" key from the inference configuration file. 将 -e 用于环境名称,将 --ev 用于环境版本。Use -e for the environment name, and --ev for the environment version. 如果未指定 --ev,则会使用最新版本。If you don't specify --ev, the latest version will be used. 下面是推理配置文件的示例:Here is an example of an inference configuration file:

{
    "entryScript": "score.py",
    "sourceDirectory": null
}

以下命令演示如何使用以前的推理配置文件(名为 myInferenceConfig.json)来部署模型。The following command demonstrates how to deploy a model using the previous inference configuration file (named myInferenceConfig.json).

它还使用最新版本的现有 Azure 机器学习环境(名为 AzureML-Minimal)。It also uses the latest version of an existing Azure Machine Learning environment (named AzureML-Minimal).

az ml model deploy -m mymodel:1 --ic myInferenceConfig.json -e AzureML-Minimal --dc deploymentconfig.json

以下命令演示如何使用 CLI 部署模型:The following command demonstrates how to deploy a model by using the CLI:

az ml model deploy -n myservice -m mymodel:1 --ic inferenceconfig.json

此实例中的配置指定以下设置:In this example, the configuration specifies the following settings:

  • 需要 Python 的模型That the model requires Python
  • 入口脚本,用于处理发送到已部署服务的 Web 请求The entry script, which is used to handle web requests sent to the deployed service
  • 用于描述推理所需的 Python 包的 Conda 文件The Conda file that describes the Python packages needed for inference

若要详细了解如何将自定义 Docker 映像与推理配置结合使用,请参阅如何使用自定义 Docker 映像部署模型For information on using a custom Docker image with an inference configuration, see How to deploy a model using a custom Docker image.

选择计算目标Choose a compute target

计算目标Compute target 用于Used for GPU 支持GPU support FPGA 支持FPGA support 说明Description
本地 web 服务Local web service 测试/调试Testing/debugging     用于有限的测试和故障排除。Use for limited testing and troubleshooting. 硬件加速依赖于本地系统中库的使用情况。Hardware acceleration depends on use of libraries in the local system.
Azure 机器学习计算实例 web 服务Azure Machine Learning compute instance web service 测试/调试Testing/debugging     用于有限的测试和故障排除。Use for limited testing and troubleshooting.
Azure 容器实例Azure Container Instances 测试或开发Testing or development     用于需要小于 48 GB RAM 的基于 CPU 的小规模工作负载。Use for low-scale CPU-based workloads that require less than 48 GB of RAM.
Azure 机器学习计算群集Azure Machine Learning compute clusters (预览)批处理 推理(Preview) Batch inference (机器学习管道)Yes (machine learning pipeline)   对无服务器计算运行批量评分。Run batch scoring on serverless compute. 支持普通 VM 和低优先级 VM。Supports normal and low-priority VMs.
Azure FunctionsAzure Functions (预览)实时推理(Preview) Real-time inference      
Azure IoT EdgeAzure IoT Edge (预览)IoT 模块(Preview) IoT module     在 IoT 设备上部署和提供 ML 模型。Deploy and serve ML models on IoT devices.

备注

尽管计算目标(例如本地、Azure 机器学习计算实例和 Azure 机器学习计算群集)支持用于训练和试验的 GPU,但在__部署为 Web 服务__时,使用 GPU 进行推理仅在 Azure Kubernetes 服务中受支持。Although compute targets like local, Azure Machine Learning compute instance, and Azure Machine Learning compute clusters support GPU for training and experimentation, using GPU for inference when deployed as a web service is supported only on Azure Kubernetes Service.

只有在 Azure 机器学习计算上,才能__在通过机器学习管道评分时__使用 GPU 进行推理。Using a GPU for inference when scoring with a machine learning pipeline is supported only on Azure Machine Learning Compute.

定义部署配置Define a deployment configuration

适用于部署配置的选项因所选计算目标而异。The options available for a deployment configuration differ depending on the compute target you choose.

deploymentconfig.json 文档中的条目对应于 LocalWebservice.deploy_configuration 的参数。The entries in the deploymentconfig.json document map to the parameters for LocalWebservice.deploy_configuration. 下表描述了 JSON 文档中的实体与方法参数之间的映射:The following table describes the mapping between the entities in the JSON document and the parameters for the method:

JSON 实体JSON entity 方法参数Method parameter 说明Description
computeType NANA 计算目标。The compute target. 对于本地目标,值必须是 localFor local targets, the value must be local.
port port 用于公开服务的 HTTP 终结点的本地端口。The local port on which to expose the service's HTTP endpoint.

此 JSON 是用于 CLI 的部署配置示例:This JSON is an example deployment configuration for use with the CLI:

{
    "computeType": "local",
    "port": 32267
}

有关详细信息,请参阅 az ml 模型部署文档。For more information, see the az ml model deploy documentation.

部署模型Deploy your model

现在已准备好部署模型。You are now ready to deploy your model.

使用已注册的模型Using a registered model

如果在 Azure 机器学习工作区中注册了模型,请将“mymodel:1”替换为模型的名称及其版本号。If you registered your model in your Azure Machine Learning workspace, replace "mymodel:1" with the name of your model and its version number.

az ml model deploy -m mymodel:1 --ic inferenceconfig.json --dc deploymentconfig.json

使用本地模型Using a local model

如果不想注册模型,则可在 inferenceconfig.json 中传递 "sourceDirectory" 参数,以指定用于提供模型的本地目录。If you would prefer not to register your model, you can pass the "sourceDirectory" parameter in your inferenceconfig.json to specify a local directory from which to serve your model.

az ml model deploy --ic inferenceconfig.json --dc deploymentconfig.json

了解服务状态Understanding service state

在模型部署期间,当模型完全部署时,你可能会看到服务状态发生更改。During model deployment, you may see the service state change while it fully deploys.

下表描述了各种服务状态:The following table describes the different service states:

Webservice 状态Webservice state 说明Description 最终状态?Final state?
正在转换Transitioning 此服务正在进行部署。The service is in the process of deployment. No
不正常Unhealthy 此服务已部署,但当前无法访问。The service has deployed but is currently unreachable. No
不可安排Unschedulable 由于缺少资源,此时无法部署此服务。The service cannot be deployed at this time due to lack of resources. No
已失败Failed 由于出现错误或崩溃,服务未能部署。The service has failed to deploy due to an error or crash. Yes
正常Healthy 服务正常,终结点可用。The service is healthy and the endpoint is available. Yes

批量推理Batch inference

Azure 机器学习计算目标由 Azure 机器学习创建和管理。Azure Machine Learning Compute targets are created and managed by Azure Machine Learning. 它们可用于 Azure 机器学习管道中的批量预测。They can be used for batch prediction from Azure Machine Learning pipelines.

若要查看使用 Azure 机器学习计算进行批量推理的演练,请参阅如何运行批量预测For a walkthrough of batch inference with Azure Machine Learning Compute, see How to run batch predictions.

IoT Edge 推理IoT Edge inference

对部署到边缘的支持处于预览阶段。Support for deploying to the edge is in preview. 有关详细信息,请参阅将 Azure 机器学习部署为 IoT Edge 模块For more information, see Deploy Azure Machine Learning as an IoT Edge module.

删除资源Delete resources

若要删除已部署的 webservice,请使用 az ml service <name of webservice>To delete a deployed webservice, use az ml service <name of webservice>.

若要从工作区中删除已注册的模型,请使用 az ml model delete <model id>To delete a registered model from your workspace, use az ml model delete <model id>

详细了解如何删除 webservice删除模型Read more about deleting a webservice and deleting a model.

后续步骤Next steps