Authenticate clients for online endpoints

  • Article

APPLIES TO: Azure CLI ml extension v2 (current) Python SDK azure-ai-ml v2 (current)

This article covers how to authenticate clients to perform control plane and data plane operations on online endpoints.

A control plane operation controls an endpoint and changes it. Control plane operations include create, read, update, and delete (CRUD) operations on online endpoints and online deployments.

A data plane operation uses data to interact with an online endpoint without changing the endpoint. For example, a data plane operation could consist of sending a scoring request to an online endpoint and getting a response.

Prerequisites

Before following the steps in this article, make sure you have the following prerequisites:

  • An Azure Machine Learning workspace. If you don't have one, use the steps in the Quickstart: Create workspace resources article to create one.

  • The Azure CLI and the ml extension or the Azure Machine Learning Python SDK v2:

Prepare a user identity

You need a user identity to perform control plane operations (that is, CRUD operations) and data plane operations (that is, send scoring requests) on the online endpoint. You can use the same user identity or different user identities for the control plane and data plane operations. In this article, you use the same user identity for both control plane and data plane operations.

To create a user identity under Microsoft Entra ID, see Set up authentication. You'll need the identity ID later.

Assign permissions to the identity

In this section, you assign permissions to the user identity that you use for interacting with the endpoint. You begin by using either a built-in role or by creating a custom role. Thereafter, you assign the role to your user identity.

Use a built-in role

The AzureML Data Scientist built-in role can be used to manage and use endpoints and deployments and it uses wildcards to include the following control plane RBAC actions:

  • Microsoft.MachineLearningServices/workspaces/onlineEndpoints/write
  • Microsoft.MachineLearningServices/workspaces/onlineEndpoints/delete
  • Microsoft.MachineLearningServices/workspaces/onlineEndpoints/read
  • Microsoft.MachineLearningServices/workspaces/onlineEndpoints/token/action
  • Microsoft.MachineLearningServices/workspaces/onlineEndpoints/listKeys/action
  • Microsoft.MachineLearningServices/workspaces/onlineEndpoints/regenerateKeys/action

and to include the following data plane RBAC action:

  • Microsoft.MachineLearningServices/workspaces/onlineEndpoints/score/action

Optionally, the Azure Machine Learning Workspace Connection Secrets Reader built-in role can be used to access secrets from workspace connections and it include the following control plane RBAC actions:

  • Microsoft.MachineLearningServices/workspaces/connections/listsecrets/action
  • Microsoft.MachineLearningServices/workspaces/metadata/secrets/read

If you use these built-in roles, there's no action needed at this step.

(Optional) Create a custom role

You can skip this step if you're using built-in roles or other pre-made custom roles.

  1. Define the scope and actions for custom roles by creating JSON definitions of the roles. For example, the following role definition allows the user to CRUD an online endpoint, under a specified workspace.

    custom-role-for-control-plane.json:

    {
    "Name": "Custom role for control plane operations - online endpoint",
    "IsCustom": true,
    "Description": "Can CRUD against online endpoints.",
    "Actions": [
        "Microsoft.MachineLearningServices/workspaces/onlineEndpoints/write",
        "Microsoft.MachineLearningServices/workspaces/onlineEndpoints/delete",
        "Microsoft.MachineLearningServices/workspaces/onlineEndpoints/read",
        "Microsoft.MachineLearningServices/workspaces/onlineEndpoints/token/action",
        "Microsoft.MachineLearningServices/workspaces/onlineEndpoints/listKeys/action",
        "Microsoft.MachineLearningServices/workspaces/onlineEndpoints/regenerateKeys/action"
    ],
    "NotActions": [
    ],
    "AssignableScopes": [
        "/subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>"
    ]
}

    The following role definition allows the user to send scoring requests to an online endpoint, under a specified workspace.

    custom-role-for-scoring.json:

    {
    "Name": "Custom role for scoring - online endpoint",
    "IsCustom": true,
    "Description": "Can score against online endpoints.",
    "Actions": [
        "Microsoft.MachineLearningServices/workspaces/onlineEndpoints/*/action"
    ],
    "NotActions": [
    ],
    "AssignableScopes": [
        "/subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>"
    ]
}

  2. Use the JSON definitions to create custom roles:

    az role definition create --role-definition custom-role-for-control-plane.json --subscription <subscriptionId>

az role definition create --role-definition custom-role-for-scoring.json --subscription <subscriptionId>

    Note

    To create custom roles, you need one of three roles:

    • owner
    • user access administrator
    • a custom role with Microsoft.Authorization/roleDefinitions/write permission (to create/update/delete custom roles) and Microsoft.Authorization/roleDefinitions/read permission (to view custom roles).

    For more information on creating custom roles, see Azure custom roles.

  3. Verify the role definition:

    az role definition list --custom-role-only -o table

az role definition list -n "Custom role for control plane operations - online endpoint"
az role definition list -n "Custom role for scoring - online endpoint"

export role_definition_id1=`(az role definition list -n "Custom role for control plane operations - online endpoint" --query "[0].id" | tr -d '"')`

export role_definition_id2=`(az role definition list -n "Custom role for scoring - online endpoint" --query "[0].id" | tr -d '"')`

Assign the role to the identity

  1. If you're using the AzureML Data Scientist built-in role, use the following code to assign the role to your user identity.

    az role assignment create --assignee <identityId> --role "AzureML Data Scientist" --scope /subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Microsoft.MachineLearningServices/workspaces/<workspaceName>

  2. Optionally, if you're using the Azure Machine Learning Workspace Connection Secrets Reader built-in role, use the following code to assign the role to your user identity.

    az role assignment create --assignee <identityId> --role "Azure Machine Learning Workspace Connection Secrets Reader" --scope /subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Microsoft.MachineLearningServices/workspaces/<workspaceName>

  3. If you're using a custom role, use the following code to assign the role to your user identity.

    az role assignment create --assignee <identityId> --role "Custom role for control plane operations - online endpoint" --scope /subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Microsoft.MachineLearningServices/workspaces/<workspaceName>

az role assignment create --assignee <identityId> --role "Custom role for scoring - online endpoint" --scope /subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Microsoft.MachineLearningServices/workspaces/<workspaceName>

    Note

    To assign custom roles to the user identity, you need one of three roles:

    • owner
    • user access administrator
    • a custom role that allows Microsoft.Authorization/roleAssignments/write permission (to assign custom roles) and Microsoft.Authorization/roleAssignments/read (to view role assignments).

    For more information on the different Azure roles and their permissions, see Azure roles and Assigning Azure roles using Azure Portal.

  4. Confirm the role assignment:

    az role assignment list --scope /subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Microsoft.MachineLearningServices/workspaces/<workspaceName>

Get the Microsoft Entra token for control plane operations

Perform this step if you plan to perform control plane operations with REST API, which will directly use the token.

If you plan to use other ways such as Azure Machine Learning CLI (v2), Python SDK (v2), or the Azure Machine Learning studio, you don't need to get the Microsoft Entra token manually. Rather, during sign in, your user identity would already be authenticated, and the token would automatically be retrieved and passed for you.

You can retrieve the Microsoft Entra token for control plane operations from the Azure resource endpoint: https://management.chinacloudapi.cn.

  1. Sign in to Azure.

    az login

  2. If you want to use a specific identity, use the following code to sign in with the identity:

    az login --identity --username <identityId>

  3. Use this context to get the token.

    export CONTROL_PLANE_TOKEN=`(az account get-access-token --resource https://management.chinacloudapi.cn --query accessToken | tr -d '"')`

(Optional) Verify the resource endpoint and client ID for Microsoft Entra token

After you retrieve the Microsoft Entra token, you can verify that the token is for the right Azure resource endpoint management.chinacloudapi.cn and the right client ID by decoding the token via jwt.ms, which will return a json response with the following information:

{
    "aud": "https://management.chinacloudapi.cn",
    "oid": "<your-object-id>"
}

Create an endpoint

The following example creates the endpoint with a system-assigned identity (SAI) as the endpoint identity. The SAI is the default identity type of the managed identity for endpoints. Some basic roles are automatically assigned for the SAI. For more information on role assignment for a system-assigned identity, see Automatic role assignment for endpoint identity.

The CLI doesn't require you to explicitly provide the control plane token. Instead, the CLI az login authenticates you during sign in, and the token is automatically retrieved and passed for you.

  1. Create an endpoint definition YAML file.

    endpoint.yml:

    $schema: https://azuremlschemas.azureedge.net/latest/managedOnlineEndpoint.schema.json
name: my-endpoint
auth_mode: aad_token

  2. You can replace auth_mode with key for key auth, or aml_token for Azure Machine Learning token auth. In this example, you use aad_token for Microsoft Entra token auth.

    az ml online-endpoint create -f endpoint.yml

  3. Check the endpoint's status:

    az ml online-endpoint show -n my-endpoint

  4. If you want to override auth_mode (for example, to aad_token) when creating an endpoint, run the following code:

    az ml online-endpoint create -n my-endpoint --auth_mode aad_token

  5. If you want to update the existing endpoint and specify auth_mode (for example, to aad_token), run the following code:

    az ml online-endpoint update -n my-endpoint --set auth_mode=aad_token

Create a deployment

To create a deployment, see Deploy an ML model with an online endpoint or Use REST to deploy an model as an online endpoint. There's no difference in how you create deployments for different auth modes.

The following code is an example of how to create a deployment. For more information on deploying online endpoints, see Deploy an ML model with an online endpoint (via CLI)

  1. Create a deployment definition YAML file.

    blue-deployment.yml:

    $schema: https://azuremlschemas.azureedge.net/latest/managedOnlineDeployment.schema.json
name: blue
endpoint_name: my-aad-auth-endp1
model:
  path: ../../model-1/model/
code_configuration:
  code: ../../model-1/onlinescoring/
  scoring_script: score.py
environment: 
  conda_file: ../../model-1/environment/conda.yml
  image: mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:latest
instance_type: Standard_DS3_v2
instance_count: 1

  2. Create the deployment using the YAML file. For this example, set all traffic to the new deployment.

    az ml online-deployment create -f blue-deployment.yml --all-traffic

Get the scoring URI for the endpoint

If you plan to use the CLI to invoke the endpoint, you're not required to get the scoring URI explicitly, as the CLI handles it for you. However, you can still use the CLI to get the scoring URI so that you can use it with other channels, such as REST API.

scoringUri=$(az ml online-endpoint show -n my-endpoint --query "scoring_uri")

Get the key or token for data plane operations

A key or token can be used for data plane operations, even though the process of getting the key or token is a control plane operation. In other words, you use a control plane token to get the key or token that you later use to perform your data plane operations.

Getting the key or Azure Machine Learning token requires that the correct role is assigned to the user identity that is requesting it, as described in authorization for control plane operations. Getting the Microsoft Entra token doesn't require any extra roles for the user identity.

If you plan to use the CLI to invoke the endpoint, you're not required to get the keys or token for data plane operations explicitly, as the CLI handles it for you. However, you can still use the CLI to get the keys or token for data plane operation so that you can use it with other channels, such as REST API.

To get the keys or token for data plane operations, use the az ml online-endpoint get-credentials command. This command returns a JSON output that contains the keys, token, and/or additional information.

Tip

To extract a specific information from the JSON output, the --query parameter of the CLI command is used as an example. However, you can use any suitable tool for this purpose.

When auth_mode of the endpoint is key

  • Keys are returned in the primaryKey and secondaryKey fields.
export DATA_PLANE_TOKEN=$(az ml online-endpoint get-credentials -n $ENDPOINT_NAME -g $RESOURCE_GROUP -w $WORKSPACE_NAME -o tsv --query primaryKey)
export DATA_PLANE_TOKEN2=$(az ml online-endpoint get-credentials -n $ENDPOINT_NAME -g $RESOURCE_GROUP -w $WORKSPACE_NAME -o tsv --query secondaryKey)

When auth_mode of the endpoint is aml_token

  • Token is returned in the accessToken field.
  • Token expiration time is returned in the expiryTimeUtc field.
  • Token refresh time is returned in the refreshAfterTimeUtc field.
export DATA_PLANE_TOKEN=$(az ml online-endpoint get-credentials -n $ENDPOINT_NAME -g $RESOURCE_GROUP -w $WORKSPACE_NAME -o tsv --query accessToken)
export EXPIRY_TIME_UTC=$(az ml online-endpoint get-credentials -n $ENDPOINT_NAME -g $RESOURCE_GROUP -w $WORKSPACE_NAME -o tsv --query expiryTimeUtc)
export REFRESH_AFTER_TIME_UTC=$(az ml online-endpoint get-credentials -n $ENDPOINT_NAME -g $RESOURCE_GROUP -w $WORKSPACE_NAME -o tsv --query refreshAfterTimeUtc)

When auth_mode of the endpoint is aad_token

  • Token is returned in the accessToken field.
  • Token expiration time is returned in the expiryTimeUtc field.
export DATA_PLANE_TOKEN=$(az ml online-endpoint get-credentials -n $ENDPOINT_NAME -g $RESOURCE_GROUP -w $WORKSPACE_NAME -o tsv --query accessToken)
export EXPIRY_TIME_UTC=$(az ml online-endpoint get-credentials -n $ENDPOINT_NAME -g $RESOURCE_GROUP -w $WORKSPACE_NAME -o tsv --query expiryTimeUtc)

Verify the resource endpoint and client ID for the Microsoft Entra token

After getting the Entra token, you can verify that the token is for the right Azure resource endpoint studio.ml.azure.cn and the right client ID by decoding the token via jwt.ms, which will return a json response with the following information:

{
    "aud": "https://studio.ml.azure.cn",
    "oid": "<your-object-id>"
}

Score data using the key or token

You can use az ml online-endpoint invoke for endpoints with a key, an Azure Machine Learning token, or a Microsoft Entra token. The CLI handles the key or token automatically so you don't need to pass it explicitly.

az ml online-endpoint invoke -n my-endpoint -r request.json

Log and monitor traffic

To enable traffic logging in the diagnostics settings for the endpoint, follow the steps in How to enable/disable logs.

If the diagnostic setting is enabled, you can check the AmlOnlineEndpointTrafficLogs table to see the auth mode and user identity.

Related content