Create model packages (preview)

Model package is a capability in Azure Machine Learning that allows you to collect all the dependencies required to deploy a machine learning model to a serving platform. Creating packages before deploying models provides robust and reliable deployment and a more efficient MLOps workflow. Packages can be moved across workspaces and even outside of Azure Machine Learning.

In this article, you learn how to package a model for deployment.

Prerequisites

Before following the steps in this article, make sure you have the following prerequisites:

• An Azure subscription. If you don't have an Azure subscription, create a Trial before you begin. Try the Azure Machine Learning.

• An Azure Machine Learning workspace. If you don't have one, use the steps in the How to manage workspacesarticle to create one.

• Azure role-based access controls (Azure RBAC) are used to grant access to operations in Azure Machine Learning. To perform the steps in this article, your user account must be assigned the owner or contributor role for the Azure Machine Learning workspace, or a custom role. For more information, see Manage access to an Azure Machine Learning workspace.

• A model to package. This example, uses an MLflow model registered in the workspace.

About this example

In this example, you will learn how to package models in Azure Machine Learning.

Clone the repository

The example in this article is based on code samples contained in the azureml-examples repository. To run the commands locally without having to copy/paste YAML and other files, first clone the repo and then change directories to the folder:

git clone https://github.com/Azure/azureml-examples --depth 1
cd azureml-examples/cli

!git clone https://github.com/Azure/azureml-examples --depth 1
!cd azureml-examples/sdk/python

This section uses the example in the folder endpoints/online/deploy-packages/custom-model.

Connect to your workspace

Connect to the Azure Machine Learning workspace where you'll do your work.

az account set --subscription <subscription>
az configure --defaults workspace=<workspace> group=<resource-group> location=<location>

Package a model

You can create model packages explicitly to allow you to control how the packaging operation is done. Use this workflow when:

You can create model packages by specifying the:

• Model to package: Each model package can contain only a single model. Azure Machine Learning doesn't support packaging of multiple models under the same model package.
• Base environment: Environments are used to indicate the base image, and in Python packages dependencies your model need. For MLflow models, Azure Machine Learning automatically generates the base environment. For custom models, you need to specify it.
• Serving technology: The inferencing stack used to run the model.

Register the model

Model packages require the model to be registered in either your workspace or in an Azure Machine Learning registry. In this example, you already have a local copy of the model in the repository, so you only need to publish the model to the registry in the workspace. You can skip this section if the model you're trying to deploy is already registered.

MODEL_NAME='sklearn-regression'
MODEL_PATH='model'
az ml model create --name $MODEL_NAME --path $MODEL_PATH --type custom_model

model_name = "sklearn-regression"
model_path = "model"

model = ml_client.models.create_or_update(Model(name=model_name, path=model_path))

Create the base environment

Base environments are used to indicate the base image and the model Python package dependencies. Our model requires the following packages to be used as indicated in the conda file:

conda.yaml

name: model-env
channels:
  - conda-forge
dependencies:
  - python=3.9
  - numpy=1.23.5
  - pip=23.0.1
  - scikit-learn=1.2.2
  - scipy=1.10.1
  - xgboost==1.3.3

Create the environment as follows:

$schema: https://azuremlschemas.azureedge.net/latest/environment.schema.json
name: sklearn-regression-env
image: mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu22.04
conda_file: conda.yaml
description: An environment for models built with XGBoost and Scikit-learn.

az ml environment create -f environment/sklearn-regression-env.yml

base_environment = ml_client.environments.create_or_update(
    Environment(
        name=f"{model_name}-env",
        image="mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu22.04",
        conda_file="environment/conda.yml",
    )
)

Create a package specification

You can create model packages in Azure Machine Learning, using the Azure CLI or the Azure Machine Learning SDK for Python. The custom package specification supports the following attributes:

1. Create a package specification as follows:

   • Azure CLI
   • Python

   package-moe.yml

    $schema: http://azureml/sdk-2-0/ModelVersionPackage.json
    base_environment_source:
   	type: environment_asset
       resource_id: azureml:sklearn-regression-env:1
    target_environment: sklearn-regression-online-pkg
    inferencing_server: 
   	type: azureml_online
   	code_configuration:
     	  code: src
         scoring_script: score.py
   

    
1. Start the model package operation:

    # [Azure CLI](#tab/cli)
    
    ```azurecli
	az ml model package -n $MODEL_NAME -v $MODEL_VERSION --file package-moe.yml
	```
    
    # [Python](#tab/sdk)
    
    ```python
	model_package = ml_client.models.package(model_name, model.version, pakage_config)
	```
    
1. The result of the package operation is an environment.


## Package a model that has dependencies in private Python feeds

Model packages can resolve Python dependencies that are available in private feeds. To use this capability, you need to create a connection from your workspace to the feed and specify the PAT token configuration. The following Python code shows how you can configure the workspace where you're running the package operation.

```python
from azure.ai.ml.entities import WorkspaceConnection
from azure.ai.ml.entities import PatTokenConfiguration

# fetching secrets from env var to secure access, these secrets can be set outside or source code
git_pat = os.environ["GIT_PAT"]

credentials = PatTokenConfiguration(pat=git_pat)

ws_connection = WorkspaceConnection(
    name="<workspace_connection_name>",
    target="<git_url>",
    type="git",
    credentials=credentials,
)

ml_client.connections.create_or_update(ws_connection)

Once the connection is created, build the model package as described in the section for Package a model. In the following example, the base environment of the package uses a private feed for the Python dependency bar, as specified in the following conda file:

conda.yml

name: foo
channels:
  - defaults
dependencies:
  - python
  - pip
  - pip:
    - --extra-index-url <python_feed_url>
    - bar

If you're using an MLflow model, model dependencies are indicated inside the model itself, and hence a base environment isn't needed. Instead, specify private feed dependencies when logging the model, as explained in Logging models with a custom signature, environment or samples.

Package a model that is hosted in a registry

Model packages provide a convenient way to collect dependencies before deployment. However, when models are hosted in registries, the deployment target is usually another workspace. When creating packages in this setup, use the target_environment property to specify the full location where you want the model package to be created, instead of just its name.

The following code creates a package of the t5-base model from a registry:

1. Connect to the registry where the model is located and the workspace in which you need the model package to be created:

   • Azure CLI
   • Python

   az login
    Connect-AzAccount -Environment AzureChinaCloud
   

2. Get a reference to the model you want to package. In this case we are packaging the model t5-base from azureml registry.

   • Azure CLI
   • Python

    MODEL_NAME="t5-base"
    MODEL_VERSION=$(az ml model show --name $MODEL_NAME --label latest --registry-name azureml | jq .version -r)
   

3. Configure a package specification. Since the model we want to package is MLflow, base environment and scoring script is optional.

   • Azure CLI
   • Python

   package.yml

    $schema: http://azureml/sdk-2-0/ModelVersionPackage.json
    target_environment: pkg-t5-base-online
    inferencing_server: 
   	type: azureml_online
   

4. Start the operation to create the model package:

   • Azure CLI
   • Python

    #<get_model>
    MODEL_NAME="t5-base"
    MODEL_VERSION=$(az ml model show --name $MODEL_NAME --label latest --registry-name azureml | jq .version -r)
    #</get_model>
   
    #<build_package>
    az ml model package --name $MODEL_NAME \
                        --version $MODEL_VERSION \
                        --registry-name azureml \
                        --file package.yml
    #</build_package>
   

5. The package is now created in the target workspace and ready to be deployed.

Package models to deploy outside of Azure Machine Learning

Model packages can be deployed outside of Azure Machine Learning if needed. To guarantee portability, you only need to ensure that the model configuration in your package has the mode set to copy so that the model itself is copied inside the generated docker image instead of referenced from the model registry in Azure Machine Learning.

The following code shows how to configure copy in a model package:

$schema: http://azureml/sdk-2-0/ModelVersionPackage.json
base_environment_source:
    type: environment_asset
    resource_id: azureml:sklearn-regression-env:1
target_environment: sklearn-regression-docker-pkg
inferencing_server: 
    type: azureml_online
    code_configuration:
      code: src
      scoring_script: score.py
model_configuration:
  mode: copy

pakage_config = ModelPackage(
    target_environment_name="sklearn-regression-online-pkg",
    base_environment_source=BaseEnvironment(
        type="asset",
        resource_id=f"azureml:{base_environment.name}:{base_environment.version}",
    ),
    inferencing_server=AzureMLOnlineInferencingServer(
        code_configuration=CodeConfiguration(code="src", scoring_script="score.py")
    ),
    model_configuration=ModelConfiguration(mode="copy"),
)

Next step

• Package and deploy a model to Online Endpoints.
• Package and deploy a model to App Service.