Use RDMA or GPU instances in Batch pools

To run certain Batch jobs, you can take advantage of Azure VM sizes designed for large-scale computation. For example:

• To run multi-instance MPI workloads, choose HB, HC, NC, or ND series or other sizes that have a network interface for Remote Direct Memory Access (RDMA). These sizes connect to an InfiniBand network for inter-node communication, which can accelerate MPI applications.

• For CUDA applications, choose N-series sizes that include NVIDIA Tesla graphics processing unit (GPU) cards.

This article provides guidance and examples to use some of Azure's specialized sizes in Batch pools. For specs and background, see:

• High performance compute VM sizes (Linux, Windows)

• GPU-enabled VM sizes (Linux, Windows)

Dependencies

The RDMA or GPU capabilities of compute-intensive sizes in Batch are supported only in certain operating systems. The supported operating systems for these VM sizes include only a subset of those available for virtual machine creation. Depending on how you create your Batch pool, you might need to install or configure extra driver or other software on the nodes. The following tables summarize these dependencies. See linked articles for details. For options to configure Batch pools, see later in this article.

Linux pools - Virtual machine configuration

Size	Capability	Operating systems	Required software	Pool settings
NC24rs_v3 *	RDMA	Ubuntu 22.04 LTS, or CentOS-based HPC (Azure Marketplace)	Intel MPI 5 Linux RDMA drivers	Enable inter-node communication, disable concurrent task execution
NCv3 series	NVIDIA Tesla GPU (varies by series)	Ubuntu 22.04 LTS, or CentOS 7.3 or 7.4 (Azure Marketplace)	NVIDIA CUDA or CUDA Toolkit drivers	N/A

^*RDMA-capable N-series sizes also include NVIDIA Tesla GPUs

Windows pools - Virtual Machine Configuration

Size	Capability	Operating systems	Required software	Pool settings
NC24rs_v3 *	RDMA	Windows Server 2016, 2012 R2, or 2012 (Azure Marketplace)	Microsoft MPI 2012 R2 or later, or Intel MPI 5 Windows RDMA drivers	Enable inter-node communication, disable concurrent task execution
NCv3 series	NVIDIA Tesla GPU (varies by series)	Windows Server 2016 or 2012 R2 (Azure Marketplace)	NVIDIA CUDA or CUDA Toolkit drivers	N/A

^*RDMA-capable N-series sizes also include NVIDIA Tesla GPUs

Pool configuration options

To configure a specialized VM size for your Batch pool, you have several options to install required software or drivers:

• For pools in the virtual machine configuration, choose a preconfigured Azure Marketplace VM image that has drivers and software preinstalled. Examples:

• Data Science Virtual Machine for Linux or Windows - includes NVIDIA CUDA drivers

• Linux images for Batch container workloads that also include GPU and RDMA drivers:

  • Ubuntu Server (with GPU and RDMA drivers) for Azure Batch container pools

• Create a custom Windows or Linux VM image with installed drivers, software, or other settings required for the VM size.

• Create a Batch application package from a zipped driver or application installer. Then, configure Batch to deploy this package to pool nodes and install once when each node is created. For example, if the application package is an installer, create a start task command line to silently install the app on all pool nodes. Consider using an application package and a pool start task if your workload depends on a particular driver version.

  Note

  The start task must run with elevated (admin) permissions, and it must wait for success. Long-running tasks will increase the time to provision a Batch pool.

Example: NVIDIA GPU drivers on Windows NC VM pool

To run CUDA applications on a pool of Windows NC nodes, you need to install NVDIA GPU drivers. The following sample steps use an application package to install the NVIDIA GPU drivers. You might choose this option if your workload depends on a specific GPU driver version.

1. Download a setup package for the GPU drivers on Windows Server 2016 from the NVIDIA website - for example, version 411.82. Save the file locally using a short name like GPUDriverSetup.exe.
2. Create a zip file of the package.
3. Upload the package to your Batch account. For steps, see the application packages guidance. Specify an application ID such as GPUDriver, and a version such as 411.82.
4. Using the Batch APIs or Azure portal, create a pool in the virtual machine configuration with the desired number of nodes and scale. The following table shows sample settings to install the NVIDIA GPU drivers silently using a start task:

Setting	Value
Image Type	Marketplace (Linux/Windows)
Publisher	MicrosoftWindowsServer
Offer	WindowsServer
Sku	2016-Datacenter
Node size	NC6 Standard
Application package references	GPUDriver, version 411.82
Start task enabled	True Command line - cmd /c "%AZ_BATCH_APP_PACKAGE_GPUDriver#411.82%\\GPUDriverSetup.exe /s" User identity - Pool autouser, admin Wait for success - True

Example: NVIDIA GPU drivers on a Linux NC VM pool

To run CUDA applications on a pool of Linux NC nodes, you need to install necessary NVIDIA Tesla GPU drivers from the CUDA Toolkit. The following sample steps create and deploy a custom Ubuntu 22.04 LTS image with the GPU drivers:

1. Deploy an Azure NC-series VM running Ubuntu 22.04 LTS. For example, create the VM in the US South Central region.

2. Add the NVIDIA GPU Drivers extension to the VM by using the Azure portal, a client computer that connects to the Azure subscription, or Azure local Shell. Alternatively, follow the steps to connect to the VM and install CUDA drivers manually.

3. Follow the steps to create an Azure Compute Gallery image for Batch.

4. Create a Batch account in a region that supports NC VMs.

5. Using the Batch APIs or Azure portal, create a pool using the custom image and with the desired number of nodes and scale. The following table shows sample pool settings for the image:

Setting	Value
Image Type	Custom Image
Custom Image	Name of the image
Node agent SKU	batch.node.ubuntu 22.04
Node size	NC6 Standard

Next steps

• To run MPI jobs on an Azure Batch pool, see the Windows or Linux examples.