Configure `kube-proxy` in Azure Kubernetes Service (AKS) (Preview)

Article
11/02/2023

kube-proxy is a component of Kubernetes that handles routing traffic for services within the cluster. There are two backends available for Layer 3/4 load balancing in upstream kube-proxy: iptables and IPVS.

iptables is the default backend utilized in the majority of Kubernetes clusters. It's simple and well-supported, but not as efficient or intelligent as IPVS.
IPVS uses the Linux Virtual Server, a layer 3/4 load balancer built into the Linux kernel. IPVS provides a number of advantages over the default iptables configuration, including state awareness, connection tracking, and more intelligent load balancing. IPVS doesn't support Azure Network Policy.

For more information, see the Kubernetes documentation on kube-proxy.

Note

If you want, you can disable the AKS-managed kube-proxy DaemonSet to support bring-your-own CNI.

Important

AKS preview features are available on a self-service, opt-in basis. Previews are provided "as is" and "as available," and they're excluded from the service-level agreements and limited warranty. AKS previews are partially covered by customer support on a best-effort basis. As such, these features aren't meant for production use. For more information, see the following support articles:

Before you begin

If using the Azure CLI, you need the aks-preview extension. See Install the aks-preview Azure CLI extension.
If using ARM or the REST API, the AKS API version must be 2022-08-02-preview or later.
You need to register the KubeProxyConfigurationPreview feature flag. See Register the KubeProxyConfigurationPreview feature flag.

Install the `aks-preview` Azure CLI extension

Install the aks-preview extension using the az extension add command.
```
az extension add --name aks-preview
```
Update to the latest version of the extension using the az extension update command.
```
az extension update --name aks-preview
```

Register the `KubeProxyConfigurationPreview` feature flag

Register the KubeProxyConfigurationPreview feature flag using the az feature register command.
```
az feature register --namespace "Microsoft.ContainerService" --name "KubeProxyConfigurationPreview"
```
It takes a few minutes for the status to show Registered.

Verify the registration status using the az feature show command.

az feature show --namespace "Microsoft.ContainerService" --name "KubeProxyConfigurationPreview"

When the status reflects Registered, refresh the registration of the Microsoft.ContainerService resource provider using the az provider register command.
```
az provider register --namespace Microsoft.ContainerService
```

`kube-proxy` configuration options

You can view the full kube-proxy configuration structure in the AKS Cluster Schema.

enabled: Determines deployment of the kube-proxy DaemonSet. Defaults to true.
mode: You can set to either IPTABLES or IPVS. Defaults to IPTABLES.
ipvsConfig: If mode is IPVS, this object contains IPVS-specific configuration properties.
- scheduler: Determines which connection scheduler to use. Supported values include:
  - LeastConnection: Sends connections to the backend pod with the fewest connections.
  - RoundRobin: Evenly distributes connections between backend pods.
- tcpFinTimeoutSeconds: Sets the timeout length value after a TCP session receives a FIN.
- tcpTimeoutSeconds: Sets the timeout length value for idle TCP sessions.
- udpTimeoutSeconds: Sets the timeout length value for idle UDP sessions.

Note

IPVS load balancing operates in each node independently and is only aware of connections flowing through the local node. This means that while LeastConnection results in a more even load under a higher number of connections, when a low amount of connections (# connects < 2 * node count) occur, traffic may be relatively unbalanced

Use `kube-proxy` in a new or existing AKS cluster

kube-proxy configuration is a cluster-wide setting. You don't need to update your services.

Warning

Changing the kube-proxy configuration may cause a slight interruption in cluster service traffic flow.

Create a configuration file with the desired kube-proxy configuration. For example, the following configuration enables IPVS with the LeastConnection scheduler and sets the TCP timeout to 900 seconds.
```
{
  "enabled": true,
  "mode": "IPVS",
  "ipvsConfig": {
    "scheduler": "LeastConnection",
    "TCPTimeoutSeconds": 900,
    "TCPFINTimeoutSeconds": 120,
    "UDPTimeoutSeconds": 300
  }
}
```

Create a new cluster or update an existing cluster with the configuration file using the az aks create or az aks update command with the --kube-proxy-config parameter set to the configuration file.

# Create a new cluster
az aks create -g <resourceGroup> -n <clusterName> --kube-proxy-config kube-proxy.json

# Update an existing cluster
az aks update -g <resourceGroup> -n <clusterName> --kube-proxy-config kube-proxy.json

Next steps

This article covered how to configure kube-proxy in Azure Kubernetes Service (AKS). To learn more about load balancing in AKS, see the following articles:

Configure kube-proxy in Azure Kubernetes Service (AKS) (Preview)

Before you begin

Install the aks-preview Azure CLI extension

Register the KubeProxyConfigurationPreview feature flag

kube-proxy configuration options

Use kube-proxy in a new or existing AKS cluster