将部署终结点升级到 SDK v2

使用 SDK/CLI v1,可以在 ACI 或 AKS 上部署模型作为 Web 服务。 现有的 v1 模型部署和 Web 服务将继续正常运行,但使用 SDK/CLI v1 在 ACI 或 AKS 上部署模型作为 Web 服务现在已被视为旧版。 对于新的模型部署,建议升级到 v2。

在 v2 中,我们提供托管终结点或 Kubernetes 终结点。 有关 v1 和 v2 的比较,请参阅终结点和部署

有多个部署漏斗,如 v2 中的托管联机终结点、kubernetes 联机终结点(包括 Azure Kubernetes 服务),v1 中的 Azure 容器实例 (ACI) 和 Kubernetes 服务 (AKS) Web 服务。 本文重点介绍部署到 ACI Web 服务 (v1) 和托管联机终结点 (v2) 的过程比较。

本文中的示例演示如何执行以下操作:

  • 将模型部署到 Azure
  • 使用终结点进行评分
  • 删除 Web 服务/终结点

创建推理资源

  • SDK v1
    1. 配置模型、环境和评分脚本:

      # configure a model. example for registering a model 
      from azureml.core.model import Model
      model = Model.register(ws, model_name="bidaf_onnx", model_path="./model.onnx")
      
      # configure an environment
      from azureml.core import Environment
      env = Environment(name='myenv')
      python_packages = ['nltk', 'numpy', 'onnxruntime']
      for package in python_packages:
          env.python.conda_dependencies.add_pip_package(package)
      
      # configure an inference configuration with a scoring script
      from azureml.core.model import InferenceConfig
      inference_config = InferenceConfig(
          environment=env,
          source_directory="./source_dir",
          entry_script="./score.py",
      )
      
    2. 配置和部署 ACI Web 服务:

      from azureml.core.webservice import AciWebservice
      
      # defince compute resources for ACI
      deployment_config = AciWebservice.deploy_configuration(
          cpu_cores=0.5, memory_gb=1, auth_enabled=True
      )
      
      # define an ACI webservice
      service = Model.deploy(
          ws,
          "myservice",
          [model],
          inference_config,
          deployment_config,
          overwrite=True,
      )
      
      # create the service 
      service.wait_for_deployment(show_output=True)
      

有关注册模型的详细信息,请参阅从本地文件注册模型

  • SDK v2

    1. 配置模型、环境和评分脚本:

      from azure.ai.ml.entities import Model
      # configure a model
      model = Model(path="../model-1/model/sklearn_regression_model.pkl")
      
      # configure an environment
      from azure.ai.ml.entities import Environment
      env = Environment(
          conda_file="../model-1/environment/conda.yml",
          image="mcr.microsoft.com/azureml/openmpi3.1.2-ubuntu18.04:20210727.v1",
      )
      
      # configure an inference configuration with a scoring script
      from azure.ai.ml.entities import CodeConfiguration
      code_config = CodeConfiguration(
              code="../model-1/onlinescoring", scoring_script="score.py"
          )
      
    2. 配置和创建联机终结点:

      import datetime
      from azure.ai.ml.entities import ManagedOnlineEndpoint
      
      # create a unique endpoint name with current datetime to avoid conflicts
      online_endpoint_name = "endpoint-" + datetime.datetime.now().strftime("%m%d%H%M%f")
      
      # define an online endpoint
      endpoint = ManagedOnlineEndpoint(
          name=online_endpoint_name,
          description="this is a sample online endpoint",
          auth_mode="key",
          tags={"foo": "bar"},
      )
      
      # create the endpoint:
      ml_client.begin_create_or_update(endpoint)
      
    3. 配置和创建联机部署:

      from azure.ai.ml.entities import ManagedOnlineDeployment
      
      # define a deployment
      blue_deployment = ManagedOnlineDeployment(
          name="blue",
          endpoint_name=online_endpoint_name,
          model=model,
          environment=env,
          code_configuration=code_config,
          instance_type="Standard_F2s_v2",
          instance_count=1,
      )
      
      # create the deployment:
      ml_client.begin_create_or_update(blue_deployment)
      
      # blue deployment takes 100 traffic
      endpoint.traffic = {"blue": 100}
      ml_client.begin_create_or_update(endpoint)
      

有关终结点和部署概念的详细信息,请参阅什么是联机终结点?

提交请求

  • SDK v1

    import json
    data = {
        "query": "What color is the fox",
        "context": "The quick brown fox jumped over the lazy dog.",
    }
    data = json.dumps(data)
    predictions = service.run(input_data=data)
    print(predictions)
    
  • SDK v2

    # test the endpoint (the request will route to blue deployment as set above)
    ml_client.online_endpoints.invoke(
        endpoint_name=online_endpoint_name,
        request_file="../model-1/sample-request.json",
    )
    
    # test the specific (blue) deployment
    ml_client.online_endpoints.invoke(
        endpoint_name=online_endpoint_name,
        deployment_name="blue",
        request_file="../model-1/sample-request.json",
    )
    

删除资源

  • SDK v1

    service.delete()
    
  • SDK v2

    ml_client.online_endpoints.begin_delete(name=online_endpoint_name)
    

SDK v1 和 SDK v2 中关键功能的映射

SDK v1 中的功能 SDK v2 中的粗略映射
azureml.core.model.Model 类 azure.ai.ml.entities.Model 类
azureml.core.Environment 类 azure.ai.ml.entities.Environment 类
azureml.core.model.InferenceConfig 类 azure.ai.ml.entities.CodeConfiguration 类
azureml.core.webservice.AciWebservice 类 azure.ai.ml.entities.OnlineDeployment 类(和 azure.ai.ml.entities.ManagedOnlineEndpoint 类
Model.deployWebservice.deploy ml_client.begin_create_or_update(online_deployment)
Webservice.run ml_client.online_endpoints.invoke
Webservice.delete ml_client.online_endpoints.delete

有关详细信息,请参阅

v2 文档:

v1 文档: