Perform safe rollout of new deployments for real-time inference

Before following the steps in this article, make sure you have the following prerequisites:

• The Azure CLI and the ml extension to the Azure CLI. For more information, see Install, set up, and use the CLI (v2).

  Important

  The CLI examples in this article assume that you are using the Bash (or compatible) shell. For example, from a Linux system or Windows Subsystem for Linux.

• An Azure Machine Learning workspace. If you don't have one, use the steps in the Install, set up, and use the CLI (v2) to create one.

• Azure role-based access controls (Azure RBAC) are used to grant access to operations in Azure Machine Learning. To perform the steps in this article, your user account must be assigned the owner or contributor role for the Azure Machine Learning workspace, or a custom role allowing Microsoft.MachineLearningServices/workspaces/onlineEndpoints/*. For more information, see Manage access to an Azure Machine Learning workspace.

• (Optional) To deploy locally, you must install Docker Engine on your local computer. We highly recommend this option, so it's easier to debug issues.

APPLIES TO: Python SDK azure-ai-ml v2 (current)

Before following the steps in this article, make sure you have the following prerequisites:

• An Azure Machine Learning workspace. If you don't have one, use the steps in the Quickstart: Create workspace resources article to create one.

• To install the Python SDK v2, use the following command:

  pip install azure-ai-ml azure-identity
  

  To update an existing installation of the SDK to the latest version, use the following command:

  pip install --upgrade azure-ai-ml azure-identity
  

  For more information, see Install the Python SDK v2 for Azure Machine Learning.

• Azure role-based access controls (Azure RBAC) are used to grant access to operations in Azure Machine Learning. To perform the steps in this article, your user account must be assigned the owner or contributor role for the Azure Machine Learning workspace, or a custom role allowing Microsoft.MachineLearningServices/workspaces/onlineEndpoints/*. For more information, see Manage access to an Azure Machine Learning workspace.

• (Optional) To deploy locally, you must install Docker Engine on your local computer. We highly recommend this option, so it's easier to debug issues.

Before following the steps in this article, make sure you have the following prerequisites:

• An Azure subscription. If you don't have an Azure subscription, create a Trial before you begin. Try the trial subscription.

• An Azure Machine Learning workspace and a compute instance. If you don't have these, use the steps in the Quickstart: Create workspace resources article to create them.

• Azure role-based access controls (Azure RBAC) are used to grant access to operations in Azure Machine Learning. To perform the steps in this article, your user account must be assigned the owner or contributor role for the Azure Machine Learning workspace, or a custom role allowing Microsoft.MachineLearningServices/workspaces/onlineEndpoints/*. For more information, see Manage access to an Azure Machine Learning workspace.

If you haven't already set the defaults for the Azure CLI, save your default settings. To avoid passing in the values for your subscription, workspace, and resource group multiple times, run this code:

az account set --subscription <subscription id>
az configure --defaults workspace=<Azure Machine Learning workspace name> group=<resource group>

To follow along with this article, first clone the examples repository (azureml-examples). Then, go to the repository's cli/ directory:

git clone --depth 1 https://github.com/Azure/azureml-examples
cd azureml-examples
cd cli

The commands in this tutorial are in the file deploy-safe-rollout-online-endpoints.sh in the cli directory, and the YAML configuration files are in the endpoints/online/managed/sample/ subdirectory.

To run the training examples, first clone the examples repository (azureml-examples). Then, go into the azureml-examples/sdk/python/endpoints/online/managed directory:

git clone --depth 1 https://github.com/Azure/azureml-examples
cd azureml-examples/sdk/python/endpoints/online/managed

The information in this article is based on the online-endpoints-safe-rollout.ipynb notebook. It contains the same content as this article, although the order of the codes is slightly different.

The workspace is the top-level resource for Azure Machine Learning, providing a centralized place to work with all the artifacts you create when you use Azure Machine Learning. In this section, we'll connect to the workspace where you'll perform deployment tasks.

1. Import the required libraries:

   # import required libraries
   from azure.ai.ml import MLClient
   from azure.ai.ml.entities import (
       ManagedOnlineEndpoint,
       ManagedOnlineDeployment,
       Model,
       Environment,
       CodeConfiguration,
   )
   from azure.identity import DefaultAzureCredential
   

   Note

   If you're using the Kubernetes online endpoint, import the KubernetesOnlineEndpoint and KubernetesOnlineDeployment class from the azure.ai.ml.entities library.

2. Configure workspace details and get a handle to the workspace:

   To connect to a workspace, we need identifier parameters—a subscription, resource group and workspace name. We'll use these details in the MLClient from azure.ai.ml to get a handle to the required Azure Machine Learning workspace. This example uses the default Azure authentication.

    # enter details of your AML workspace
    subscription_id = "<SUBSCRIPTION_ID>"
    resource_group = "<RESOURCE_GROUP>"
    workspace = "<AML_WORKSPACE_NAME>"
   

    # get a handle to the workspace
    ml_client = MLClient(
        DefaultAzureCredential(), subscription_id, resource_group, workspace
    )
   

If you have Git installed on your local machine, you can follow the instructions to clone the examples repository. Otherwise, follow the instructions to download files from the examples repository.

To follow along with this article, first clone the examples repository (azureml-examples) and then change into the azureml-examples/cli/endpoints/online/model-1 directory.

git clone --depth 1 https://github.com/Azure/azureml-examples
cd azureml-examples/cli/endpoints/online/model-1

If you cloned the examples repo, your local machine already has copies of the files for this example, and you can skip to the next section. If you didn't clone the repo, you can download it to your local machine.

1. Go to https://github.com/Azure/azureml-examples/.
2. Go to the <> Code button on the page, and then select Download ZIP from the Local tab.
3. Locate the model folder /cli/endpoints/online/model-1/model and scoring script /cli/endpoints/online/model-1/onlinescoring/score.py for a first model model-1.
4. Locate the model folder /cli/endpoints/online/model-2/model and scoring script /cli/endpoints/online/model-2/onlinescoring/score.py for a second model model-2.

First set the endpoint's name and then configure it. In this article, you'll use the endpoints/online/managed/sample/endpoint.yml file to configure the endpoint. The following snippet shows the contents of the file:

$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineEndpoint.schema.json
name: my-endpoint
auth_mode: key

The reference for the endpoint YAML format is described in the following table. To learn how to specify these attributes, see the online endpoint YAML reference. For information about limits related to managed online endpoints, see limits for online endpoints.

To create an online endpoint:

1. Set your endpoint name:

   For Unix, run this command (replace YOUR_ENDPOINT_NAME with a unique name):

   export ENDPOINT_NAME="<YOUR_ENDPOINT_NAME>"
   

   Important

   Endpoint names must be unique within an Azure region. For example, in the Azure chinanorth2 region, there can be only one endpoint with the name my-endpoint.

2. Create the endpoint in the cloud:

   Run the following code to use the endpoint.yml file to configure the endpoint:

   az ml online-endpoint create --name $ENDPOINT_NAME -f endpoints/online/managed/sample/endpoint.yml
   

In this article, you'll use the endpoints/online/managed/sample/blue-deployment.yml file to configure the key aspects of the deployment. The following snippet shows the contents of the file:

$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineDeployment.schema.json
name: blue
endpoint_name: my-endpoint
model:
  path: ../../model-1/model/
code_configuration:
  code: ../../model-1/onlinescoring/
  scoring_script: score.py
environment: 
  conda_file: ../../model-1/environment/conda.yaml
  image: mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:latest
instance_type: Standard_DS3_v2
instance_count: 1

To create a deployment named blue for your endpoint, run the following command to use the blue-deployment.yml file to configure

az ml online-deployment create --name blue --endpoint-name $ENDPOINT_NAME -f endpoints/online/managed/sample/blue-deployment.yml --all-traffic

In the blue-deployment.yaml file, we specify the path (where to upload files from) inline. The CLI automatically uploads the files and registers the model and environment. As a best practice for production, you should register the model and environment and specify the registered name and version separately in the YAML. Use the form model: azureml:my-model:1 or environment: azureml:my-env:1.

For registration, you can extract the YAML definitions of model and environment into separate YAML files and use the commands az ml model create and az ml environment create. To learn more about these commands, run az ml model create -h and az ml environment create -h.

For more information on registering your model as an asset, see Register your model as an asset in Machine Learning by using the CLI. For more information on creating an environment, see Manage Azure Machine Learning environments with the CLI & SDK (v2).

To create a managed online endpoint, use the ManagedOnlineEndpoint class. This class allows users to configure the key aspects of the endpoint.

1. Configure the endpoint:

   # Creating a unique endpoint name with current datetime to avoid conflicts
   import datetime
   
   online_endpoint_name = "endpoint-" + datetime.datetime.now().strftime("%m%d%H%M%f")
   
   # create an online endpoint
   endpoint = ManagedOnlineEndpoint(
       name=online_endpoint_name,
       description="this is a sample online endpoint",
       auth_mode="key",
       tags={"foo": "bar"},
   )
   

   Note

   To create a Kubernetes online endpoint, use the KubernetesOnlineEndpoint class.

2. Create the endpoint:

   ml_client.begin_create_or_update(endpoint).result()
   

To create a deployment for your managed online endpoint, use the ManagedOnlineDeployment class. This class allows users to configure the key aspects of the deployment. The following table describes the attributes of a deployment:

1. Configure blue deployment:

   # create blue deployment
   model = Model(path="../model-1/model/sklearn_regression_model.pkl")
   env = Environment(
       conda_file="../model-1/environment/conda.yml",
       image="mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04",
   )
   
   blue_deployment = ManagedOnlineDeployment(
       name="blue",
       endpoint_name=online_endpoint_name,
       model=model,
       environment=env,
       code_configuration=CodeConfiguration(
           code="../model-1/onlinescoring", scoring_script="score.py"
       ),
       instance_type="Standard_DS2_v2",
       instance_count=1,
   )
   

   In this example, we specify the path (where to upload files from) inline. The SDK automatically uploads the files and registers the model and environment. As a best practice for production, you should register the model and environment and specify the registered name and version separately in the codes.

   For more information on registering your model as an asset, see Register your model as an asset in Machine Learning by using the SDK.

   For more information on creating an environment, see Manage Azure Machine Learning environments with the CLI & SDK (v2).

   Note

   To create a deployment for a Kubernetes online endpoint, use the KubernetesOnlineDeployment class.

2. Create the deployment:

   ml_client.begin_create_or_update(blue_deployment).result()
   

   # blue deployment takes 100 traffic
   endpoint.traffic = {"blue": 100}
   ml_client.begin_create_or_update(endpoint).result()
   

When you create a managed online endpoint in the Azure Machine Learning studio, you must define an initial deployment for the endpoint. Before you can define a deployment, you must have a registered model in your workspace. Let's begin by registering the model to use for the deployment.

A model registration is a logical entity in the workspace. This entity can contain a single model file or a directory of multiple files. As a best practice for production, you should register the model and environment. When creating the endpoint and deployment in this article, we'll assume that you've registered the model folder that contains the model.

To register the example model, follow these steps:

1. Go to the Azure Machine Learning studio.

2. In the left navigation bar, select the Models page.

3. Select Register, and then choose From local files.

4. Select Unspecified type for the Model type.

5. Select Browse, and choose Browse folder.

   

6. Select the \azureml-examples\cli\endpoints\online\model-1\model folder from the local copy of the repo you cloned or downloaded earlier. When prompted, select Upload and wait for the upload to complete.

7. Select Next after the folder upload is completed.

8. Enter a friendly Name for the model. The steps in this article assume the model is named model-1.

9. Select Next, and then Register to complete registration.

10. Repeat the previous steps to register a model-2 from the \azureml-examples\cli\endpoints\online\model-2\model folder in the local copy of the repo you cloned or downloaded earlier.

For more information on working with registered models, see Register and work with models.

For information on creating an environment in the studio, see Create an environment.

Use the Azure Machine Learning studio to create a managed online endpoint directly in your browser. When you create a managed online endpoint in the studio, you must define an initial deployment. You can't create an empty managed online endpoint.

One way to create a managed online endpoint in the studio is from the Models page. This method also provides an easy way to add a model to an existing managed online deployment. To deploy the model named model-1 that you registered previously in the Register your model section:

1. Go to the Azure Machine Learning studio.

2. In the left navigation bar, select the Models page.

3. Select the model named model-1 by checking the circle next to its name.

4. Select Deploy > Real-time endpoint.

   

   This action opens up a window where you can specify details about your endpoint.

   

5. Enter an Endpoint name.

6. Keep the default selections: Managed for the compute type and key-based authentication for the authentication type.

7. Select Next, until you get to the "Deployment" page. Here, perform the following tasks:

   • Name the deployment "blue".
   • Check the box for Enable Application Insights diagnostics and data collection to allow you to view graphs of your endpoint's activities in the studio later.

8. Select Next to go to the "Environment" page. Here, perform following steps:

   • In the "Select scoring file and dependencies" box, browse and select the \azureml-examples\cli\endpoints\online\model-1\onlinescoring\score.py file from the repo you cloned or downloaded earlier.
   • Start typing sklearn in the search box above the list of environments, and select the AzureML-sklearn-0.24 curated environment.

9. Select Next to go to the "Compute" page. Here, keep the default selection for the virtual machine "Standard_DS3_v2" and change the Instance count to 1.

10. Select Next, to accept the default traffic allocation (100%) to the blue deployment.

11. Review your deployment settings and select the Create button.

    

Alternatively, you can create a managed online endpoint from the Endpoints page in the studio.

1. Go to the Azure Machine Learning studio.

2. In the left navigation bar, select the Endpoints page.

3. Select + Create.

   

This action opens up a window for you to specify details about your endpoint and deployment. Enter settings for your endpoint and deployment as described in the previous steps 5-11, accepting defaults until you're prompted to Create the deployment.

You can view the status of your existing endpoint and deployment by running:

az ml online-endpoint show --name $ENDPOINT_NAME 

az ml online-deployment show --name blue --endpoint $ENDPOINT_NAME 

You should see the endpoint identified by $ENDPOINT_NAME and, a deployment called blue.

The endpoint can be invoked using the invoke command. We'll send a sample request using a json file.

az ml online-endpoint invoke --name $ENDPOINT_NAME --request-file endpoints/online/model-1/sample-request.json

Check the status to see whether the model was deployed without error:

ml_client.online_endpoints.get(name=online_endpoint_name)

Using the MLClient created earlier, we'll get a handle to the endpoint. The endpoint can be invoked using the invoke command with the following parameters:

• endpoint_name - Name of the endpoint
• request_file - File with request data
• deployment_name - Name of the specific deployment to test in an endpoint

We'll send a sample request using a json file.

# test the blue deployment with some sample data
ml_client.online_endpoints.invoke(
    endpoint_name=online_endpoint_name,
    deployment_name="blue",
    request_file="../model-1/sample-request.json",
)

You can view all your managed online endpoints in the Endpoints page. Go to the endpoint's Details page to find critical information including the endpoint URI, status, testing tools, activity monitors, deployment logs, and sample consumption code:

1. In the left navigation bar, select Endpoints. Here, you can see a list of all the endpoints in the workspace.
2. (Optional) Create a Filter on Compute type to show only Managed compute types.
3. Select an endpoint name to view the endpoint's Details page.

Use the Test tab in the endpoint's details page to test your managed online deployment. Enter sample input and view the results.

1. Select the Test tab in the endpoint's detail page. The blue deployment is already selected in the dropdown menu.
2. Copy the sample input from the json file
3. Paste the sample input in the test box.
4. Select Test.

In the deployment described in Deploy and score a machine learning model with an online endpoint, you set the instance_count to the value 1 in the deployment yaml file. You can scale out using the update command:

az ml online-deployment update --name blue --endpoint-name $ENDPOINT_NAME --set instance_count=2

Using the MLClient created earlier, we'll get a handle to the deployment. The deployment can be scaled by increasing or decreasing the instance_count.

# scale the deployment
blue_deployment = ml_client.online_deployments.get(
    name="blue", endpoint_name=online_endpoint_name
)
blue_deployment.instance_count = 2
ml_client.online_deployments.begin_create_or_update(blue_deployment).result()

# Get the details for online endpoint
endpoint = ml_client.online_endpoints.get(name=online_endpoint_name)

# existing traffic details
print(endpoint.traffic)

# Get the scoring URI
print(endpoint.scoring_uri)

Use the following instructions to scale the deployment up or down by adjusting the number of instances:

1. In the endpoint Details page. Find the card for the blue deployment.
2. Select the edit icon in the header of the blue deployment's card.
3. Change the instance count to 2.
4. Select Update.

Create a new deployment named green:

az ml online-deployment create --name green --endpoint-name $ENDPOINT_NAME -f endpoints/online/managed/sample/green-deployment.yml

Since we haven't explicitly allocated any traffic to green, it has zero traffic allocated to it. You can verify that using the command:

az ml online-endpoint show -n $ENDPOINT_NAME --query traffic

Though green has 0% of traffic allocated, you can invoke it directly by specifying the --deployment name:

az ml online-endpoint invoke --name $ENDPOINT_NAME --deployment-name green --request-file endpoints/online/model-2/sample-request.json

If you want to use a REST client to invoke the deployment directly without going through traffic rules, set the following HTTP header: azureml-model-deployment: <deployment-name>. The below code snippet uses curl to invoke the deployment directly. The code snippet should work in Unix/WSL environments:

# get the scoring uri
SCORING_URI=$(az ml online-endpoint show -n $ENDPOINT_NAME -o tsv --query scoring_uri)
# use curl to invoke the endpoint
curl --request POST "$SCORING_URI" --header "Authorization: Bearer $ENDPOINT_KEY" --header 'Content-Type: application/json' --header "azureml-model-deployment: green" --data @endpoints/online/model-2/sample-request.json

Create a new deployment for your managed online endpoint and name the deployment green:

# create green deployment
model2 = Model(path="../model-2/model/sklearn_regression_model.pkl")
env2 = Environment(
    conda_file="../model-2/environment/conda.yml",
    image="mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04",
)

green_deployment = ManagedOnlineDeployment(
    name="green",
    endpoint_name=online_endpoint_name,
    model=model2,
    environment=env2,
    code_configuration=CodeConfiguration(
        code="../model-2/onlinescoring", scoring_script="score.py"
    ),
    instance_type="Standard_DS2_v2",
    instance_count=1,
)

# use MLClient to create green deployment
ml_client.begin_create_or_update(green_deployment).result()

Though green has 0% of traffic allocated, you can still invoke the endpoint and deployment with the json file.

ml_client.online_endpoints.invoke(
    endpoint_name=online_endpoint_name,
    deployment_name="green",
    request_file="../model-2/sample-request.json",
)

Create a new deployment to add to your managed online endpoint and name the deployment green.

From the Endpoint details page

1. Select + Add Deployment button in the endpoint "Details" page.

2. Select Deploy a model.

3. Select Next to go to the "Model" page and select the model model-2.

4. Select Next to go to the "Deployment" page and perform the following tasks:

   1. Name the deployment "green".
   2. Enable application insights diagnostics and data collection.

5. Select Next to go to the "Environment" page. Here, perform following steps:

   • In the "Select scoring file and dependencies" box, browse and select the \azureml-examples\cli\endpoints\online\model-2\onlinescoring\score.py file from the repo you cloned or downloaded earlier.
   • Start typing sklearn in the search box above the list of environments, and select the AzureML-sklearn-0.24 curated environment.

6. Select Next to go to the "Compute" page. Here, keep the default selection for the virtual machine "Standard_DS3_v2" and change the Instance count to 1.

7. Select Next to go to the "Traffic" page. Here, keep the default traffic allocation to the deployments (100% traffic to "blue" and 0% traffic to "green").

8. Select Next to review your deployment settings.

   

9. Select Create to create the deployment.

Alternatively, you can use the Models page to add a deployment:

1. In the left navigation bar, select the Models page.
2. Select a model by checking the circle next to the model name.
3. Select Deploy > Real-time endpoint.
4. Choose to deploy to an existing managed online endpoint.
5. Follow the previous steps 3 to 9 to finish creating the green deployment.

Though green has 0% of traffic allocated, you can still invoke the endpoint and deployment. Use the Test tab in the endpoint's details page to test your managed online deployment. Enter sample input and view the results.

1. Select the Test tab in the endpoint's detail page.
2. Select the green deployment from the dropdown menu.
3. Copy the sample input from the json file.
4. Paste the sample input in the test box.
5. Select Test.

The following command mirrors 10% of the traffic to the green deployment:

az ml online-endpoint update --name $ENDPOINT_NAME --mirror-traffic "green=10"

You can test mirror traffic by invoking the endpoint several times without specifying a deployment to receive the incoming traffic:

for i in {1..20} ; do
    az ml online-endpoint invoke --name $ENDPOINT_NAME --request-file endpoints/online/model-1/sample-request.json
done

You can confirm that the specific percentage of the traffic was sent to the green deployment by seeing the logs from the deployment:

az ml online-deployment get-logs --name blue --endpoint $ENDPOINT_NAME

After testing, you can set the mirror traffic to zero to disable mirroring:

az ml online-endpoint update --name $ENDPOINT_NAME --mirror-traffic "green=0"

The following command mirrors 10% of the traffic to the green deployment:

endpoint.mirror_traffic = {"green": 10}
ml_client.begin_create_or_update(endpoint).result()

You can test mirror traffic by invoking the endpoint several times without specifying a deployment to receive the incoming traffic:

# You can test mirror traffic by invoking the endpoint several times
for i in range(20):
    ml_client.online_endpoints.invoke(
        endpoint_name=online_endpoint_name,
        request_file="../model-1/sample-request.json",
    )

You can confirm that the specific percentage of the traffic was sent to the green deployment by seeing the logs from the deployment:

ml_client.online_deployments.get_logs(
    name="green", endpoint_name=online_endpoint_name, lines=50
)

After testing, you can set the mirror traffic to zero to disable mirroring:

ml_client.online_deployments.get_logs(
    name="green", endpoint_name=online_endpoint_name, lines=50
)

To mirror 10% of the traffic to the green deployment:

1. From the endpoint Details page, Select Update traffic.
2. Slide the button to Enable mirrored traffic.
3. Select the green deployment in the "Deployment name" dropdown menu.
4. Keep the default traffic allocation of 10%.
5. Select Update.

The endpoint details page now shows mirrored traffic allocation of 10% to the green deployment.

To test mirrored traffic, see the Azure CLI or Python tabs to invoke the endpoint several times. Confirm that the specific percentage of the traffic was sent to the green deployment by seeing the logs from the deployment. You can access the deployment logs from the endpoint's Deployment logs tab. You can also use Metrics and Logs to monitor performance of the mirrored traffic. For more information, see Monitor online endpoints.

After testing, you can disable mirroring:

1. From the endpoint Details page, Select Update traffic.
2. Slide the button next to Enable mirrored traffic again to disable mirrored traffic.
3. Select Update.

Once you've tested your green deployment, allocate a small percentage of traffic to it:

az ml online-endpoint update --name $ENDPOINT_NAME --traffic "blue=90 green=10"

Once you've tested your green deployment, allocate a small percentage of traffic to it:

endpoint.traffic = {"blue": 90, "green": 10}
ml_client.begin_create_or_update(endpoint).result()

Once you've tested your green deployment, allocate a small percentage of traffic to it:

1. In the endpoint Details page, Select Update traffic.
2. Adjust the deployment traffic by allocating 10% to the green deployment and 90% to the blue deployment.
3. Select Update.

Once you're fully satisfied with your green deployment, switch all traffic to it.

az ml online-endpoint update --name $ENDPOINT_NAME --traffic "blue=0 green=100"

Once you're fully satisfied with your green deployment, switch all traffic to it.

endpoint.traffic = {"blue": 0, "green": 100}
ml_client.begin_create_or_update(endpoint).result()

Once you're fully satisfied with your green deployment, switch all traffic to it.

1. In the endpoint Details page, Select Update traffic.
2. Adjust the deployment traffic by allocating 100% to the green deployment and 0% to the blue deployment.
3. Select Update.

az ml online-deployment delete --name blue --endpoint $ENDPOINT_NAME --yes --no-wait

ml_client.online_deployments.begin_delete(
    name="blue", endpoint_name=online_endpoint_name
).result()

1. In the endpoint Details page, find the blue deployment.
2. Select the delete icon next to the deployment name.

If you aren't going to use the endpoint and deployment, you should delete them. By deleting the endpoint, you'll also delete all its underlying deployments.

az ml online-endpoint delete --name $ENDPOINT_NAME --yes --no-wait

If you aren't going to use the endpoint and deployment, you should delete them. By deleting the endpoint, you'll also delete all its underlying deployments.

ml_client.online_endpoints.begin_delete(name=online_endpoint_name).result()

If you aren't going to use the endpoint and deployment, you should delete them. By deleting the endpoint, you'll also delete all its underlying deployments.

1. Go to the Azure Machine Learning studio.
2. In the left navigation bar, select the Endpoints page.
3. Select an endpoint by checking the circle next to the model name.
4. Select Delete.

Alternatively, you can delete a managed online endpoint directly by selecting the Delete icon in the endpoint's details page.