LocalDNS is a feature in Azure Kubernetes Service (AKS) designed to enhance the Domain Name System (DNS) resolution performance and resiliency for workloads running in your cluster. When you deploy a DNS proxy on each node, LocalDNS reduces DNS query latency, improves reliability during network disruptions, and provides advanced configuration options for DNS caching and forwarding. This article explains how LocalDNS works, its configuration options, and how to enable, verify, and troubleshoot LocalDNS in your AKS clusters.
To learn about what LocalDNS is, including architecture details, and key capabilities, refer to DNS Resolution in Azure Kubernetes Service (AKS).
Best practices for LocalDNS configuration
When implementing LocalDNS in your AKS clusters, consider the following best practices:
Start with a minimal configuration: Begin with a simple configuration that uses the
Preferredmode before moving toRequiredmode. This setup allows you to validate that LocalDNS works as expected without breaking your cluster.Implement proper caching strategies: Configure cache settings based on your workload characteristics:
- For frequently changing records, use shorter
cacheDurationInSecondsvalues. When doing so, it's important to note that cacheDurationInSeconds acts as a cap on the DNS record TTL but doesn't increase it. The resulting TTL is the smaller of what is returned from upstream or what is set in the cache plugin. - For stable records, use longer cache durations to reduce DNS queries.
- Enable
serveStalewith appropriate settings to maintain service during DNS outages. - Caching with LocalDNS operates on a best effort basis and doesn't guarantee stale responses. The cache is divided into 256 shards and with a default maximum of 10,000 entries, allowing each shard to hold about 39 entries. When a shard is full and a new entry needs to be added, one of the existing entries is chosen at random to be evicted. There's no preference for older or expires entries. As a result, a stale record might not always be available, especially under high query volume.
- For frequently changing records, use shorter
Monitor DNS performance: After enabling LocalDNS, monitor your application's DNS performance using:
- Application performance metrics.
- Node metrics to detect reduced network pressure.
- Log entries when
queryLoggingis set toLog.
Follow least privilege principle: When configuring DNS forwarding rules, only allow access to the required DNS servers and domains.
Test before production deployment: Always test LocalDNS configuration in a nonproduction environment before rolling it out to production clusters.
Use Infrastructure as Code (IaC): Store your localdnsconfig.json file in your infrastructure repository and include it in your AKS deployment templates.
Network configuration for TCP forwarding: When using TCP for DNS forwarding to VnetDNS, ensure that your Network Security Groups (NSGs), firewalls, or Network Virtual Appliances (NVAs) don't block TCP traffic between CoreDNS/LocalDNS and VnetDNS servers.
Avoid enabling both NodeLocal DNSCache and LocalDNS: It isn't recommended to enable both the upstream Kubernetes NodeLocal DNSCache and LocalDNS in your node pool. While AKS doesn't block this configuration, all DNS traffic is routed through LocalDNS, which might lead to unexpected behavior or reduced benefits from NodeLocal DNSCache.
Prerequisites
- You must have an existing AKS cluster with Kubernetes versions 1.31 or later to use LocalDNS. If you need an AKS cluster, you can create one using Azure CLI, Azure PowerShell, or the Azure portal.
- This article requires version 2.80.0 or later of the Azure CLI. If you're using Azure Cloud Shell, the latest version is already installed.
- Your AKS cluster can't have node autoprovisioning enabled to use LocalDNS.
- LocalDNS is only supported on node pools running Azure Linux or Ubuntu 22.04 or newer.
- LocalDNS only supports Virtual Machine Scale Set node pools.
- The Virtual Machine (VM) SKU used for your node pool must have at least 4 vCPUs (cores) to support LocalDNS.
Manage LocalDNS on an AKS cluster
LocalDNS is configured at the node pool level in AKS, meaning you can enable or disable LocalDNS independently for each node pool in your cluster. This tailors DNS resolution behavior based on the specific requirements of different workloads or environments. To enable LocalDNS on a node pool, you need to provide a configuration file: localdnsconfig.json that defines how LocalDNS should operate for that node pool.
If you don't specify a custom configuration file, AKS automatically applies a default LocalDNS configuration.
To enable LocalDNS during node pool creation, use the following command with your custom configuration file:
az aks nodepool add --name mynodepool1 --cluster-name myAKSCluster --resource-group myResourceGroup --localdns-config ./localdnsconfig.json
To enable LocalDNS on an existing node pool, use the following command with your custom configuration file:
az aks nodepool update --name mynodepool1 --cluster-name myAKSCluster --resource-group myResourceGroup --localdns-config ./localdnsconfig.json
Important
Enabling LocalDNS on a node pool initiates a reimage operation on all nodes within that pool. This process can cause temporary disruption to running workloads and might lead to application downtime if not properly managed. You should plan for potential service interruptions and ensure that the applications are configured for high availability or have appropriate disruption budgets in place before enabling this setting.
Create a custom server block in LocalDNS
CoreDNS matches queries to a specific server block based on an exact match for domain being queried and not on partial matches. If you have the need for custom server blocks, you can add them to your LocalDNS configuration by creating a file called localdnsconfig.json with the added configurations.
For example, if you have specific DNS needs when accessing microsoft.com, you could use the following server block:
"microsoft.com": {
"queryLogging": "Error",
"protocol": "ForceTCP",
"forwardDestination": "ClusterCoreDNS",
"forwardPolicy": "Sequential",
"maxConcurrent": 1000,
"cacheDurationInSeconds": 3600,
"serveStaleDurationInSeconds": 3600,
"serveStale": "Immediate"
}
Monitor LocalDNS
LocalDNS exposes Prometheus metrics that you can use for monitoring and alerting. These metrics are exposed on port 9253 of the Node IP and can be scraped from there.
The following example YAML shows a scrape configuration you can use with the Azure Managed Prometheus add on as a DaemonSet:
kind: ConfigMap
apiVersion: v1
metadata:
name: ama-metrics-prometheus-config-node
namespace: kube-system
data:
prometheus-config: |-
global:
scrape_interval: 1m
scrape_configs:
- job_name: localdns-metrics
scrape_interval: 1m
scheme: http
metrics_path: /metrics
relabel_configs:
- source_labels: [__metrics_path__]
regex: (.*)
target_label: metrics_path
- source_labels: [__address__]
replacement: '$NODE_NAME'
target_label: instance
static_configs:
- targets: ['$NODE_IP:9253']
Troubleshoot LocalDNS
DNS queries to specific domains are failing
If DNS queries to specific domains are failing after enabling LocalDNS:
- Check if you have domain-specific overrides in your localdnsconfig.json that might be misconfigured.
- Temporarily try removing domain-specific overrides and using only the default
.configuration. - Check if the issue occurs with both User Datagram Protocol (UDP) and Transmission Control Protocol (TCP) by adjusting the
protocolsetting.
Update VNet DNS servers for LocalDNS
When you update custom DNS servers directly in the VNet configuration (using the Azure portal or CLI), these changes aren't automatically applied to your AKS cluster nodes. This happens because updating DNS settings at the VNet level only informs the Network Resource Provider (NRP), but doesn't notify the AKS Resource Provider. As a result, AKS nodes continue to use the previous DNS server settings until further action is taken.
To ensure AKS nodes pick up the new VNet DNS server settings:
Update the VNet DNS configuration using the Azure portal or APIs as needed.
Reimage the node pool through the AKS Resource Provider to apply and persist the DNS changes:
az aks nodepool upgrade --resource-group myResourceGroup --cluster-name myAKSCluster --name mynodepool --node-image-only
This process ensures the AKS Resource Provider is aware of the DNS changes and applies them to all nodes in the node pool.
Next steps
For information on LocalDNS in AKS, see LocalDNS in Azure Kubernetes Service (conceptual).
For comprehensive troubleshooting guidance on DNS issues when using LocalDNS, see Troubleshoot LocalDNS issues in AKS.
For details on how to customize CoreDNS in AKS, refer to the CoreDNS customization guide.
For information on the CoreDNS project, see the CoreDNS upstream project page.
To learn more about core network concepts, see Network concepts for applications in AKS.