Create an Azure Machine Learning compute cluster

APPLIES TO: Azure CLI ml extension v2 (current) Python SDK azure-ai-ml v2 (current)

This article explains how to create and manage a compute cluster in your Azure Machine Learning workspace.

You can use Azure Machine Learning compute cluster to distribute a training or batch inference process across a cluster of CPU or GPU compute nodes in the cloud. For more information on the VM sizes that include GPUs, see GPU-optimized virtual machine sizes.

Learn how to:

  • Create a compute cluster.
  • Lower your compute cluster cost with low priority VMs.
  • Set up a managed identity for the cluster.

Note

Instead of creating a compute cluster, use serverless compute to offload compute lifecycle management to Azure Machine Learning.

Prerequisites

Select the appropriate tab for the rest of the prerequisites based on your preferred method of creating the compute cluster.

Start at Azure Machine Learning studio.

Note

When configuring a Virtual Network (VNet) located in a different resource group from your Azure Machine Learning workspace, be aware that resources such as Network Security Groups (NSGs), Public IPs, and Load Balancers will be created in the same resource group as the VNet. This behavior ensures proper network management and isolation.

What is a compute cluster?

Azure Machine Learning compute cluster is a managed-compute infrastructure that allows you to easily create a single or multi-node compute. The compute cluster is a resource that can be shared with other users in your workspace. The compute scales up automatically when a job is submitted, and can be put in an Azure Virtual Network. Compute cluster supports no public IP deployment as well in virtual network. The compute executes in a containerized environment and packages your model dependencies in a Docker container.

Compute clusters can run jobs securely in either a managed virtual network or an Azure virtual network, without requiring enterprises to open up SSH ports. The job executes in a containerized environment and packages your model dependencies in a Docker container.

Limitations

  • Compute clusters can be created in a different region than your workspace. This functionality is only available for compute clusters, not compute instances.

    Warning

    When using a compute cluster in a different region than your workspace or datastores, you might see increased network latency and data transfer costs. The latency and costs can occur when creating the cluster, and when running jobs on it.

  • Azure Machine Learning Compute has default limits, such as the number of cores that can be allocated. For more information, see Manage and request quotas for Azure resources.

  • Azure allows you to place locks on resources, so that they can't be deleted or are read only. Do not apply resource locks to the resource group that contains your workspace. Applying a lock to the resource group that contains your workspace prevents scaling operations for Azure Machine Learning compute clusters. For more information on locking resources, see Lock resources to prevent unexpected changes.

Caution

Applying resource locks, such as "Delete" or "Read-only", to the resource group that contains your Machine Learning workspace or to a separate resource group where you've configured a virtual network can prevent operations like creation, scaling, or deletion of these clusters. Ensure that resource locks are configured appropriately to avoid unintended disruptions.

Create

Time estimate: Approximately five minutes.

Note

If you use serverless compute, you don't need to create a compute cluster.

Azure Machine Learning Compute can be reused across runs. The compute can be shared with other users in the workspace and is retained between runs, automatically scaling nodes up or down based on the number of runs submitted, and the max_nodes set on your cluster. The min_nodes setting controls the minimum nodes available.

The dedicated cores per region per VM family quota and total regional quota, which applies to compute cluster creation, is unified and shared with Azure Machine Learning training compute instance quota.

Important

To avoid charges when no jobs are running, set the minimum nodes to 0. This setting allows Azure Machine Learning to de-allocate the nodes when they aren't in use. Any value larger than 0 will keep that number of nodes running, even if they are not in use.

The compute autoscales down to zero nodes when it isn't used. Dedicated VMs are created to run your jobs as needed.

Use the following examples to create a compute cluster:

Create a single- or multi- node compute cluster for your training, batch inference or reinforcement learning workloads.

  1. Navigate to Azure Machine Learning studio.

  2. Under Manage, select Compute.

  3. If you have no compute resources, select New in the middle of the page.

    Screenshot that shows the New button to create a compute target.

  4. If you see a list of compute resources, select +New above the list.

    Screenshot that shows the New button to create the resource.

  5. In the tabs at the top, select Compute cluster.

  6. Fill out the form as follows:

    Field Description
    Location The Azure region where the compute cluster is created. By default, this is the same location as the workspace. If you don't have sufficient quota in the default region, switch to a different region for more options.
    When using a different region than your workspace or datastores, you might see increased network latency and data transfer costs. The latency and costs can occur when creating the cluster, and when running jobs on it.
    Virtual machine type Choose CPU or GPU. This type can't be changed after creation.
    Virtual machine priority Choose Dedicated or Low priority. Low priority virtual machines are cheaper but don't guarantee the compute nodes. Your job might be preempted.
    Virtual machine size Supported virtual machine sizes might be restricted in your region. Check the availability list
  7. Select Next to proceed to Advanced Settings and fill out the form as follows:

    Field Description
    Compute name * Name is required and must be between 3 to 24 characters long.

    * Valid characters are upper and lower case letters, digits, and the - character.

    * Name must start with a letter.

    * Name needs to be unique across all existing computes within an Azure region. You see an alert if the name you choose isn't unique.

    * If - character is used, then it needs to be followed by at least one letter later in the name.
    Minimum number of nodes Minimum number of nodes that you want to provision. If you want a dedicated number of nodes, set that count here. Save money by setting the minimum to 0, so you don't pay for any nodes when the cluster is idle.
    Maximum number of nodes Maximum number of nodes that you want to provision. The compute automatically scales to a maximum of this node count when a job is submitted.
    Idle seconds before scale down Idle time before scaling the cluster down to the minimum node count.
    Enable SSH access Use the same instructions as Enable SSH access for a compute instance.
    Advanced settings Optional. Configure network settings.

    * If an Azure Virtual Network, Specify the Resource group, Virtual network, and Subnet to create the compute instance inside the network. For more information, see network requirements.

    * If an Azure Machine Learning managed network, the compute cluster is automatically in the managed network. For more information, see managed computes with a managed network.

    * No public IP configures whether the compute cluster has a public IP address when in a network.

    * Assign a managed identity to grant access to resources.
  8. Select Create.

Enable SSH access

SSH access is disabled by default. SSH access can't be changed after creation. Make sure to enable access if you plan to debug interactively with VS Code Remote.

After you have selected Next: Advanced Settings:

  1. Turn on Enable SSH access.
  2. In the SSH public key source, select one of the options from the dropdown:
    • If you Generate new key pair:
      1. Enter a name for the key in Key pair name.
      2. Select Create.
      3. Select Download private key and create compute. The key is usually downloaded into the Downloads folder.
    • If you select Use existing public key stored in Azure, search for and select the key in Stored key.
    • If you select Use existing public key, provide an RSA public key in the single-line format (starting with "ssh-rsa") or the multi-line PEM format. You can generate SSH keys using ssh-keygen on Linux and OS X, or PuTTYGen on Windows.

Connect with SSH access

After you create a compute with SSH access enabled, use these steps for access.

  1. Find the compute in your workspace resources:

    1. On the left, select Compute.
    2. Use the tabs at the top to select Compute instance or Compute cluster to find your machine.
  2. Select the compute name in the list of resources.

  3. Find the connection string:

    • For a compute instance, select Connect at the top of the Details section.

      Screenshot that shows connect tool at the top of the Details page.

    • For a compute cluster, select Nodes at the top, then select the Connection string in the table for your node. Screenshot that shows connection string for a node in a compute cluster.

  4. Copy the connection string.

  5. For Windows, open PowerShell or a command prompt:

    1. Go into the directory or folder where your key is stored

    2. Add the -i flag to the connection string to locate the private key and point to where it is stored:

      ssh -i <keyname.pem> azureuser@... (rest of connection string)

  6. For Linux users, follow the steps from Create and use an SSH key pair for Linux VMs in Azure

  7. For SCP use:

    scp -i key.pem -P {port} {fileToCopyFromLocal } azureuser@yourComputeInstancePublicIP:~/{destination}

Lower your compute cluster cost with low priority VMs

You can also choose to use low-priority VMs to run some or all of your workloads. These VMs don't have guaranteed availability and might be preempted while in use. You have to restart a preempted job.

Using Azure Low Priority Virtual Machines allows you to take advantage of Azure's unused capacity at a significant cost savings. At any point in time when Azure needs the capacity back, the Azure infrastructure evicts Azure Low Priority Virtual Machines. Therefore, Azure Low Priority Virtual Machine is great for workloads that can handle interruptions. The amount of available capacity can vary based on size, region, time of day, and more. When deploying Azure Low Priority Virtual Machines, Azure allocates the VMs if there's capacity available, but there's no SLA for these VMs. An Azure Low Priority Virtual Machine offers no high availability guarantees. At any point in time when Azure needs the capacity back, the Azure infrastructure evicts Azure Low Priority Virtual Machines.

Use any of these ways to specify a low-priority VM:

In the studio, choose Low Priority when you create a VM.

Delete

While your compute cluster scales down to zero nodes when not in use, unprovisioned nodes contribute to your quota usage. Deleting the compute cluster removes the compute target from your workspace, and releases the quota.

  1. Navigate to Azure Machine Learning studio.
  2. In the left menu, under Manage, select Compute.
  3. At the top of the Compute page, select Compute cluster.
  4. Select the cluster you want to delete.
  5. At the top of the page, select Delete.

Set up managed identity

For information on how to configure a managed identity with your compute cluster, see Set up authentication between Azure Machine Learning and other services.

Troubleshooting

There's a chance that some users who created their Azure Machine Learning workspace from the Azure portal before the GA release might not be able to create AmlCompute in that workspace. You can either raise a support request against the service or create a new workspace through the portal or the SDK to unblock yourself immediately.

Important

If your compute instance or compute clusters are based on any of these series, recreate with another VM size.

These series retired on August 31, 2024:

Stuck at resizing

If your Azure Machine Learning compute cluster appears stuck at resizing (0 -> 0) for the node state, Azure resource locks might be the cause.

Azure allows you to place locks on resources, so that they cannot be deleted or are read only. Locking a resource can lead to unexpected results. Some operations that don't seem to modify the resource actually require actions that are blocked by the lock.

With Azure Machine Learning, applying a delete lock to the resource group for your workspace will prevent scaling operations for Azure ML compute clusters. To work around this problem we recommend removing the lock from resource group and instead applying it to individual items in the group.

Important

Do not apply the lock to the following resources:

Resource name Resource type
<GUID>-azurebatch-cloudservicenetworksecurityggroup Network security group
<GUID>-azurebatch-cloudservicepublicip Public IP address
<GUID>-azurebatch-cloudserviceloadbalancer Load balancer

These resources are used to communicate with, and perform operations such as scaling on, the compute cluster. Removing the resource lock from these resources should allow autoscaling for your compute clusters.

For more information on resource locking, see Lock resources to prevent unexpected changes.

Next step

Use your compute cluster to: