Manage Azure Machine Learning registries

Article
09/13/2024

Azure Machine Learning entities can be grouped into two broad categories:

Assets such as models, environments, components, and datasets are durable entities that are workspace agnostic. For example, a model can be registered with any workspace and deployed to any endpoint.
Resources such as compute, job, and endpoints are transient entities that are workspace specific. For example, an online endpoint has a scoring URI that is unique to a specific instance in a specific workspace. Similarly, a job runs for a known duration and generates logs and metrics each run.

Assets lend themselves to being stored in a central repository and used in different workspaces, possibly in different regions. Resources are workspace specific.

Azure Machine Learning registries enable you to create and use those assets in different workspaces. Registries support multi-region replication for low latency access to assets, so you can use assets in workspaces located in different Azure regions. Creating a registry provisions Azure resources required to facilitate replication. First, Azure blob storage accounts in each supported region. Second, a single Azure Container Registry with replication enabled to each supported region.

Diagram of the relationships between assets in workspace and registry.

Prerequisites

Before following the steps in this article, make sure you have the following prerequisites:

The Azure CLI and the ml extension to the Azure CLI. For more information, see Install, set up, and use the CLI (v2).

Important

The CLI examples in this article assume that you are using the Bash (or compatible) shell. For example, from a Linux system or Windows Subsystem for Linux.
An Azure Machine Learning workspace. If you don't have one, use the steps in the Install, set up, and use the CLI (v2) to create one.

Tip

If you are using an older version of the ml extension for CLI, you may need to update it to the latest version before working with this feature. To update the latest version, use the following command:

az extension update -n ml

>
> For more information, see [Install, set up, and use the CLI (v2)](../how-to-configure-cli.md).

<!--Update_Description: new article about new feature cli-->
<!--NEW.date: 05/01/2023-->

Prepare to create registry

You need to decide the following information carefully before proceeding to create a registry:

Choose a name

Consider the following factors before picking a name.

Registries are meant to facilitate sharing of ML assets across teams within your organization across all workspaces. Choose a name that is reflective of the sharing scope. The name should help identify your group, division, or organization.
Registry name is unique with your organization (Microsoft Entra tenant). For example, you might prefix your team or organization name and avoid generic names.
Registry names can't be changed once created because they're used in IDs of models, environments, and components that are referenced in code.
- Length can be 2-32 characters.
- Alphanumerics, underscore, hyphen are allowed. No other special characters. No spaces - registry names are part of model, environment, and component IDs that can be referenced in code.
- Name can contain underscore or hyphen but can't start with an underscore or hyphen. Needs to start with an alphanumeric.

Choose Azure regions

Registries enable sharing of assets across workspaces. To do so, a registry replicates content across multiple Azure regions. You need to define the list of regions that a registry supports when creating the registry. Create a list of all regions in which you have workspaces today and plan to add in near future. This list is a good set of regions to start with. When creating a registry, you define a primary region and a set of other regions. The primary region can't be changed after registry creation, but the other regions can be updated at a later point.

Check permissions

Make sure you're the "Owner" or "Contributor" of the subscription or resource group in which you plan to create the registry. If you don't have one of these built-in roles, review the section on permissions toward the end of this article.

Create a registry

Create the YAML definition and name it registry.yml.

Note

The primary location is listed twice in the YAML file. In the following example, chinaeast2 is listed first as the primary location (location item) and also in the replication_locations list.

name: DemoRegistry1
tags:
  description: Basic registry with one primary region and to additional regions
  foo: bar
location: chinaeast2
replication_locations:
  - location: chinaeast2
  - location: chinaeast2
  - location: chinaeast2

For more information on the structure of the YAML file, see the registry YAML reference article.

Tip

You typically see display names of Azure regions such as 'China East 2' in the Azure Portal but the registry creation YAML needs names of regions without spaces and lower case letters. Use az account list-locations -o table to find the mapping of region display names to the name of the region that can be specified in YAML.

Run the registry create command.

az ml registry create --file registry.yml

Tip

You need the curl utility to complete this step. The curl program is available in the Windows Subsystem for Linux or any UNIX distribution. In PowerShell, curl is an alias for Invoke-WebRequest and curl -d "key=val" -X POST uri becomes Invoke-WebRequest -Body "key=val" -Method POST -Uri uri.

To authenticate REST API calls, you need an authentication token for your Azure user account. You can use the following command to retrieve a token:

az account get-access-token

The response should provide an access token good for one hour. Make note of the token, as you use it to authenticate all administrative requests. The following JSON is a sample response:

Tip

The value of the access_token field is the token.

{
    "access_token": "YOUR-ACCESS-TOKEN",
    "expiresOn": "<expiration-time>",
    "subscription": "<subscription-id>",
    "tenant": "your-tenant-id",
    "tokenType": "Bearer"
}

To create a registry, use the following command. You can edit the JSON to change the inputs as needed. Replace the <YOUR-ACCESS-TOKEN> value with the access token retrieved previously:

Tip

We recommend using the latest API version when working with the REST API. For a list of the current REST API versions for Azure Machine Learning, see the Machine Learning REST API reference. The current API versions are listed in the table of contents on the left side of the page.

curl -X PUT https://management.chinacloudapi.cn/subscriptions/<your-subscription-id>/resourceGroups/<your-resource-group>/providers/Microsoft.MachineLearningServices/registries/reg-from-rest?api-version=2024-04-01 -H "Authorization:Bearer <YOUR-ACCESS-TOKEN>" -H 'Content-Type: application/json' -d ' 
{
    "properties":
    {
        "regionDetails": 
        [
            {
                "location": "chinaeast2",
                "storageAccountDetails":
                [
                    {
                        "systemCreatedStorageAccount": 
                        {
                            "storageAccountType": "Standard_LRS"
                        }
                    }
                ],
                "acrDetails": 
                [
                    {
                        "systemCreatedAcrAccount":
                        {
                            "acrAccountSku": "Premium"
                        }
                    }
                ]
            }
        ]
    },
    "identity": {
        "type": "SystemAssigned"
        },
    "location": "chinaeast2"
}
'

You should receive a 202 Accepted response.

Specify storage account type and SKU (optional)

Tip

Specifying the Azure Storage Account type and SKU is only available from the Azure CLI.

Azure storage offers several types of storage accounts with different features and pricing. For more information, see the Types of storage accounts article. Once you identify the optimal storage account SKU that best suites your needs, find the value for the appropriate SKU type. In the YAML file, use your selected SKU type as the value of the storage_account_type field. This field is under each location in the replication_locations list.

Next, decide if you want to use an Azure Blob storage account or Azure Data Lake Storage Gen2. To create Azure Data Lake Storage Gen2, set storage_account_hns to true. To create Azure Blob Storage, set storage_account_hns to false. The storage_account_hns field is under each location in the replication_locations list.

Note

The hns portion of storage_account_hns refers to the hierarchical namespace capability of Azure Data Lake Storage Gen2 accounts.

The following example YAML file demonstrates this advanced storage configuration:

name: DemoRegistry2
tags:
  description: Registry with additional configuration for storage accounts
  foo: bar
location: chinaeast2
replication_locations:
  - location: chinaeast2
    storage_config:
      storage_account_hns: False
      storage_account_type: Standard_LRS
  - location: chinaeast2
    storage_config:
      storage_account_hns: False
      storage_account_type: Standard_LRS
  - location: chinaeast2
    storage_config:
      storage_account_hns: False
      storage_account_type: Standard_LRS

Add users to the registry

Decide if you want to allow users to only use assets (models, environments, and components) from the registry or both use and create assets in the registry. Review steps to assign a role if you aren't familiar how to manage permissions using Azure role-based access control.

Allow users to use assets from the registry

To let a user only read assets, you can grant the user the built-in Reader role. If you don't want to use the built-in role, create a custom role with the following permissions

Permission	Description
Microsoft.MachineLearningServices/registries/read	Allows the user to list registries and get registry metadata
Microsoft.MachineLearningServices/registries/assets/read	Allows the user to browse assets and use the assets in a workspace

Allow users to create and use assets from the registry

To let the user both read and create or delete assets, grant the following write permission in addition to the previous read permissions.

Permission	Description
Microsoft.MachineLearningServices/registries/assets/write	Create assets in registries
Microsoft.MachineLearningServices/registries/assets/delete	Delete assets in registries

Warning

The built-in Contributor and Owner roles allow users to create, update and delete registries. You must create a custom role if you want the user to create and use assets from the registry, but not create or update registries. Review custom roles to learn how to create custom roles from permissions.

Allow users to create and manage registries

To let users create, update, and delete registries, grant them the built-in Contributor or Owner role. If you don't want to use built-in roles, create a custom role with the following permissions, in addition to all the above permissions to read, create, and delete assets in registry.

Permission	Description
Microsoft.MachineLearningServices/registries/write	Allows the user to create or update registries
Microsoft.MachineLearningServices/registries/delete	Allows the user to delete registries