How to use batch endpoints in Azure Machine Learning studio

  • Article

In this article, you learn how to use batch endpoints to do batch scoring in Azure Machine Learning studio. For more, see What are Azure Machine Learning endpoints?.

In this article, you learn about:

  • Create a batch endpoint with a no-code experience for MLflow model
  • Check batch endpoint details
  • Start a batch scoring job
  • Overview of batch endpoint features in Azure machine learning studio

Important

When working on a private link-enabled workspaces, batch enpoints can be created and managed using Azure Machine Learning studio. However, they can't be invoked from the UI. Please use the Azure ML CLI v instead for job creation. For more details about how to use it see Invoke the batch endpoint to start a batch scoring job.

Prerequisites

Create a batch endpoint

There are two ways to create Batch Endpoints in Azure Machine Learning studio:

  • From the Endpoints page, select Batch Endpoints and then select + Create.

    Screenshot of creating a batch endpoint/deployment from Endpoints page

OR

  • From the Models page, select the model you want to deploy and then select Deploy to batch endpoint.

    Screenshot of creating a batch endpoint/deployment from Models page

Tip

If you're using an MLflow model, you can use no-code batch endpoint creation. That is, you don't need to prepare a scoring script and environment, both can be auto generated. For more, see Train and track ML models with MLflow and Azure Machine Learning.

Screenshot of deploying an MLflow model

Complete all the steps in the wizard to create a batch endpoint and deployment.

Screenshot of batch endpoints/deployment review screen

Check batch endpoint details

After a batch endpoint is created, select it from the Endpoints page to view the details.

Screenshot of the check batch endpoints and deployment details

Start a batch scoring job

A batch scoring workload runs as an offline job. By default, batch scoring stores the scoring outputs in blob storage. You can also configure the outputs location and overwrite some of the settings to get the best performance.

  1. Select + Create job:

    Screenshot of the create job option to start batch scoring

  2. You can update the default deployment while submitting a job from the drop-down:

    Screenshot of using the deployment to submit a batch job

Overwrite settings

Some settings can be overwritten when you start a batch scoring job. For example, you might overwrite settings to make better use of the compute resource, or to improve performance. To override settings, select Override deployment settings and provide the settings. For more information, see Use batch endpoints.

Screenshot of the overwrite setting when starting a batch job

Start a batch scoring job with different input options

You have two options to specify the data inputs in Azure machine learning studio:

  • Use a registered dataset:

    Note

    During Preview, only FileDataset is supported.

    Screenshot of selecting registered dataset as an input option

OR

  • Use a datastore:

    You can specify AzureML registered datastore or if your data is publicly available, specify the public path.

    Screenshot of selecting datastore as an input option

Configure the output location

By default, the batch scoring results are stored in the default blob store for the workspace. Results are in a folder named after the job name (a system-generated GUID).

To change where the results are stored, providing a blob store and output path when you start a job.

Important

You must use a unique output location. If the output file exists, the batch scoring job will fail.

Screenshot of optionally configuring output location

Summary of all submitted jobs

To see a summary of all the submitted jobs for an endpoint, select the endpoint and then select the Runs tab.

Screenshot of summary of jobs submitted to a batch endpoint

Check batch scoring results

To learn how to view the scoring results, see Use batch endpoints.

Add a deployment to an existing batch endpoint

In Azure machine learning studio, there are two ways to add a deployment to an existing batch endpoint:

  • From the Endpoints page, select the batch endpoint to add a new deployment to. Select + Add deployment, and complete the wizard to add a new deployment.

    Screenshot of add new deployment option

OR

  • From the Models page, select the model you want to deploy. Then select Deploy to batch endpoint option from the drop-down. In the wizard, on the Endpoint screen, select Existing. Complete the wizard to add the new deployment.

    Screenshot of selecting an existing batch endpoint to add new deployment

Update the default deployment

If an endpoint has multiple deployments, one of the deployments is the default. The default deployment receives 100% of the traffic to the endpoint. To change the default deployment, use the following steps:

  1. Select the endpoint from the Endpoints page.
  2. Select Update default deployment. From the Details tab, select the deployment you want to set as default and then select Update. Screenshot of updating default deployment

Delete batch endpoint and deployments

To delete an endpoint, select the endpoint from the Endpoints page and then select delete.

Warning

Deleting an endpoint also deletes all deployments to that endpoint.

To delete a deployment, select the endpoint from the Endpoints page, select the deployment, and then select delete.

Next steps

In this article, you learned how to create and call batch endpoints. See these other articles to learn more about Azure Machine Learning: