Configure Maximum Transmission Unit (MTU) for virtual machines in Azure

  • Article

The Maximum Transmission Unit (MTU) is a measurement representing the largest size ethernet frame (packet) transmitted by a network device or interface. If a packet exceeds the largest size accepted by the device, the packet is fragmented into multiple smaller packets, then later reassembled at the destination.

Fragmentation and reassembly can introduce performance and ordering issues, resulting in a suboptimal experience. Optimizing MTU for your solution can provide network bandwidth performance benefits by reducing the total number of packets required to send a dataset. Configuration of larger MTU sizes can potentially improve network throughput as it reduces the number of packets and header overhead required to send a dataset.

The MTU is a configurable setting in a virtual machine's operating system. The default value MTU setting in Azure is 1500 bytes.

VMs in Azure can support larger MTU than the 1,500-byte default only for traffic that stays within the virtual network.

The following table shows the largest MTU size supported on the Azure Network Interfaces available in Azure:

Operating System Network Interface Largest MTU for inter virtual network traffic
Windows Server Mellanox Cx-3, Cx-4, Cx-5 3900
When setting the MTU value with Set-NetAdapterAdvancedProperty, use the value 4088.. To persist reboots, the value returned by Test-Connection must also be set with Set-NetIPInterface.
Windows Server (Preview) Azure Network Adapter MANA 9000
When setting the MTU value with Set-NetAdapterAdvancedProperty, use the value 9014. To persist reboots, the value returned by Test-Connection must also be set with Set-NetIPInterface.
Linux Mellanox Cx-3, Cx-4, Cx-5 3900
Linux (Preview) Azure Network Adapter 9000

Prerequisites

  • An Azure account with an active subscription. Create an account.

  • Two Linux virtual machines in the same virtual network in Azure. For more information about creating a Linux virtual machine, see Create a Linux virtual machine in the Azure portal. Remote access to the virtual machines is required for completion of the article. For more information about connecting to Azure Virtual Machines securely, see What is Azure Bastion?

    • For the purposes of this article, the virtual machines are named vm-1 and vm-2. Replace these values with your values.

Resource examples

The following resources are used as examples in this article. Replace these values with your values.

Resource Name IP Address
Virtual Machine 1 vm-1 10.0.0.4
Virtual Machine 2 vm-2 10.0.0.5

Precautions

  • Virtual machines in Azure can support a larger MTU than the 1,500-byte default only for traffic that stays within the virtual network. A larger MTU isn't supported for scenarios outside of intra-virtual network VM-to-VM traffic. Traffic traversing through gateways, peering’s, or to the internet might not be supported. Configuration of a larger MTU can result in fragmentation and reduction in performance. For traffic utilizing these scenarios, utilize the default 1,500 byte MTU for testing to ensure that a larger MTU is supported across the entire network path.

  • Optimal MTU is operating system, network, and application specific. The maximal supported MTU might not be optimal for your use case.

  • Always test MTU settings changes in a noncritical environment first before applying broadly or to critical environments.

Path MTU Discovery

It's important to understand the MTU supported across the network path your application or machines uses. Path MTU discovery is a means to find out the largest MTU supported between a source and destination address. Using a larger MTU than is supported between the source and destination address results in fragmentation, which could negatively affect performance.

In this article, the examples used test the MTU path between two virtual machines. Subsequent tests can be performed from a virtual machine to any routable destination.

Use the following steps to set a larger MTU size on a source and destination virtual machine. Verify the path MTU with a shell script for Linux or PowerShell for Windows. If the larger MTU isn't supported, the results shown in the path MTU discovery test differ from the settings configured on the source or destination virtual machine interface.

The shell script is available in the Azure samples gallery. Download the script for Linux from the following link and save to vm-1 and vm-2.

Use the following steps to change the MTU size on a Linux virtual machine:

  1. Sign-in to vm-1

  2. Use the ip command to show the current network interfaces and their MTU settings, Record the IP address for the subsequent steps. In this example, the IP address is 10.0.0.4 and the ethernet interface is eth0.

    ip address show

    azureuser@vm-1:~$ ip address show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:0d:3a:c5:f3:14 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.4/24 metric 100 brd 10.0.0.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::20d:3aff:fec5:f314/64 scope link 
       valid_lft forever preferred_lft forever
3: enP46433s1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master eth0 state UP group default qlen 1000
    link/ether 00:0d:3a:c5:f3:14 brd ff:ff:ff:ff:ff:ff
    altname enP46433p0s2
    inet6 fe80::20d:3aff:fec5:f314/64 scope link 
       valid_lft forever preferred_lft forever

  3. Set the MTU value on vm-1 to the highest value supported by the network interface. In this example, the name of the network interface is eth0. Replace this value with your value.

    • For the Mellanox adapter, use the following example to set the MTU value to 3900:
    echo '3900' | sudo tee /sys/class/net/eth0/mtu || echo "failed: $?"
    • For the Azure Network Adapter, use the following example to set the MTU value to 9000:
    echo '9000' | sudo tee /sys/class/net/eth0/mtu || echo "failed: $?"

    Important

    The MTU changes made in the previous steps don't persist during a reboot. To make the changes permanent, consult the appropriate documentation for your Linux distribution.

  4. Use the ip command to verify that the MTU settings are applied to the network interface:

    ip address show

    azureuser@vm-1:~$ ip address show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 3900 qdisc mq state UP group default qlen 1000
    link/ether 00:0d:3a:c5:f3:14 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.4/24 metric 100 brd 10.0.0.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::20d:3aff:fec5:f314/64 scope link 
       valid_lft forever preferred_lft forever
3: enP46433s1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 3900 qdisc mq master eth0 state UP group default qlen 1000
    link/ether 00:0d:3a:c5:f3:14 brd ff:ff:ff:ff:ff:ff
    altname enP46433p0s2
    inet6 fe80::20d:3aff:fec5:f314/64 scope link 
       valid_lft forever preferred_lft forever

  5. Sign-in to vm-2 to repeat the previous steps to set the MTU value to the highest value supported by the network interface.

  6. Sign-in to vm-1.

  7. Use the following example to execute the Linux shell script to test the largest MTU size that can be used for a specific network path. Replace the value of the destination host with the IP address of vm-2.

    ./GetPathMtu.sh 10.0.0.5

  8. The output is similar to the following example. If the script's output doesn't display the setting on the network interface, it indicates that the MTU size isn't set correctly. Alternatively, it could mean that a network device along the path only supports the MTU size returned by the GetPathMTU script.

    azureuser@vm-1:~/GetPathMTU$ ./GetPathMtu.sh 10.0.0.5
destination: 10.0.0.5
startSendBufferSize: 1200
interfaceName: Default interface
Test started ....................................................................................................................................................................................................
3900

  9. Verify the MTU size on the network interface using PING. For Linux, use the -M, -s, and -c flags. The -M option instructs ping to NOT fragment, -s sets the packet size, and -c sets the number of pings to send. To determine the packet size, subtract 28 from the MTU setting of 3900.

    ping 10.0.0.5 -c 10 -M do -s 3872

    azureuser@vm-1:~/GetPathMTU$ ping 10.0.0.5 -c 10 -M do -s 3872
PING 10.0.0.5 (10.0.0.5) 3872(3900) bytes of data.
3880 bytes from 10.0.0.5: icmp_seq=1 ttl=64 time=3.70 ms
3880 bytes from 10.0.0.5: icmp_seq=2 ttl=64 time=1.08 ms
3880 bytes from 10.0.0.5: icmp_seq=3 ttl=64 time=1.51 ms
3880 bytes from 10.0.0.5: icmp_seq=4 ttl=64 time=1.25 ms
3880 bytes from 10.0.0.5: icmp_seq=5 ttl=64 time=1.29 ms
3880 bytes from 10.0.0.5: icmp_seq=6 ttl=64 time=1.05 ms
3880 bytes from 10.0.0.5: icmp_seq=7 ttl=64 time=5.67 ms
3880 bytes from 10.0.0.5: icmp_seq=8 ttl=64 time=1.92 ms
3880 bytes from 10.0.0.5: icmp_seq=9 ttl=64 time=2.72 ms
3880 bytes from 10.0.0.5: icmp_seq=10 ttl=64 time=1.20 ms

--- 10.0.0.5 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9014ms
rtt min/avg/max/mdev = 1.051/2.138/5.666/1.426 ms

    An indication that there is a mismatch in settings between the source and destination displays as an error message in the output. In this case, the MTU isn't set on the source network interface.

    azureuser@vm-1:~/GetPathMTU$ ping 10.0.0.5 -c 10 -M do -s 3872
PING 10.0.0.5 (10.0.0.5) 3872(3900) bytes of data.
ping: local error: message too long, mtu=1500
ping: local error: message too long, mtu=1500
ping: local error: message too long, mtu=1500
ping: local error: message too long, mtu=1500
ping: local error: message too long, mtu=1500
ping: local error: message too long, mtu=1500
ping: local error: message too long, mtu=1500
ping: local error: message too long, mtu=1500
ping: local error: message too long, mtu=1500
ping: local error: message too long, mtu=1500

--- 10.0.0.5 ping statistics ---
10 packets transmitted, 0 received, +10 errors, 100% packet loss, time 9248ms

  10. Sign-in to vm-2.

  11. Use the following example to run the Linux shell script to test the largest MTU size that can be used for a specific network path:

    ./GetPathMtu.sh 10.0.0.4

  12. The output is similar to the following example. If the script's output doesn't display the setting on the network interface, it indicates that the MTU size isn't set correctly. Alternatively, it could mean that a network device along the path only supports the MTU size returned by the GetPathMTU script.

    azureuser@vm-1:~/GetPathMTU$ ./GetPathMtu.sh 10.0.0.4
destination: 10.0.0.4
startSendBufferSize: 1200
interfaceName: Default interface
Test started ....................................................................................................................................................................................................
3900

  13. Verify the MTU size on the network interface using PING. For Linux, use the -M, -s, and -c flags. The -M option instructs ping to NOT fragment, -s sets the packet size, and -c sets the number of pings to send. To determine the packet size, subtract 28 from the MTU setting of 3900.

    ping 10.0.0.4 -c 10 -M do -s 3872

    azureuser@vm-2:~/GetPathMTU$ ping 10.0.0.4 -c 10 -M do -s 3872
PING 10.0.0.4 (10.0.0.4) 3872(3900) bytes of data.
3880 bytes from 10.0.0.4: icmp_seq=1 ttl=64 time=3.70 ms
3880 bytes from 10.0.0.4: icmp_seq=2 ttl=64 time=1.08 ms
3880 bytes from 10.0.0.4: icmp_seq=3 ttl=64 time=1.51 ms
3880 bytes from 10.0.0.4: icmp_seq=4 ttl=64 time=1.25 ms
3880 bytes from 10.0.0.4: icmp_seq=5 ttl=64 time=1.29 ms
3880 bytes from 10.0.0.4: icmp_seq=6 ttl=64 time=1.05 ms
3880 bytes from 10.0.0.4: icmp_seq=7 ttl=64 time=5.67 ms
3880 bytes from 10.0.0.4: icmp_seq=8 ttl=64 time=1.92 ms
3880 bytes from 10.0.0.4: icmp_seq=9 ttl=64 time=2.72 ms
3880 bytes from 10.0.0.4: icmp_seq=10 ttl=64 time=1.20 ms

--- 10.0.0.4 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9014ms
rtt min/avg/max/mdev = 1.051/2.138/5.666/1.426 ms

    An indication that there is a mismatch in settings between the source and destination displays as an error message in the output. In this case, the MTU isn't set on the source network interface.

    azureuser@vm-2:~/GetPathMTU$ ping 10.0.0.4 -c 10 -M do -s 3872
PING 10.0.0.4 (10.0.0.4) 3872(3900) bytes of data.
ping: local error: message too long, mtu=1500
ping: local error: message too long, mtu=1500
ping: local error: message too long, mtu=1500
ping: local error: message too long, mtu=1500
ping: local error: message too long, mtu=1500
ping: local error: message too long, mtu=1500
ping: local error: message too long, mtu=1500
ping: local error: message too long, mtu=1500
ping: local error: message too long, mtu=1500
ping: local error: message too long, mtu=1500

--- 10.0.0.4 ping statistics ---
10 packets transmitted, 0 received, +10 errors, 100% packet loss, time 9248ms

Revert changes

To revert the changes made in this article, use the following steps:

  1. Sign-in to vm-1.

  2. Use the following example to set the MTU value to the default value of 1500:

    echo '1500' | sudo tee /sys/class/net/eth0/mtu || echo "failed: $?"

    Important

    The MTU changes made in the previous steps don't persist during a reboot. To make the changes permanent, consult the appropriate documentation for your Linux distribution.

  3. Use the ip command to verify that the MTU settings are applied to the network interface:

    ip address show

    azureuser@vm-1:~$ ip address show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:0d:3a:c5:f3:14 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.4/24 metric 100 brd 10.0.0.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::20d:3aff:fec5:f314/64 scope link 
       valid_lft forever preferred_lft forever
3: enP46433s1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master eth0 state UP group default qlen 1000
    link/ether 00:0d:3a:c5:f3:14 brd ff:ff:ff:ff:ff:ff
    altname enP46433p0s2
    inet6 fe80::20d:3aff:fec5:f314/64 scope link 
       valid_lft forever preferred_lft forever

  4. Sign-in to vm-2 to repeat the previous steps to set the MTU value to the default value of 1500.

Related content