Troubleshoot network issues with registry

  • Article

This article helps you troubleshoot problems you might encounter when accessing an Azure container registry in a virtual network or behind a firewall or proxy server.

Symptoms

May include one or more of the following:

  • Unable to push or pull images and you receive error dial tcp: lookup myregistry.azurecr.cn
  • Unable to push or pull images and you receive error Client.Timeout exceeded while awaiting headers
  • Unable to push or pull images and you receive Azure CLI error Could not connect to the registry login server
  • Unable to pull images from registry to Azure Kubernetes Service or another Azure service
  • Unable to access a registry behind an HTTPS proxy and you receive error Error response from daemon: login attempt failed with status: 403 Forbidden or Error response from daemon: Get <registry>: proxyconnect tcp: EOF Login failed
  • Unable to configure virtual network settings and you receive error Failed to save firewall and virtual network settings for container registry
  • Unable to access or view registry settings in Azure portal or manage registry using the Azure CLI
  • Unable to add or modify virtual network settings or public access rules
  • ACR Tasks is unable to push or pull images
  • Microsoft Defender for Cloud can't scan images in registry, or scan results don't appear in Microsoft Defender for Cloud
  • You receive error host is not reachable when attempting to access a registry configured with a private endpoint.

Causes

  • A client firewall or proxy prevents access - solution
  • Public network access rules on the registry prevent access - solution
  • Virtual network or private endpoint configuration prevents access - solution
  • You attempt to integrate Microsoft Defender for Cloud or certain other Azure services with a registry that has a private endpoint, service endpoint, or public IP access rules - solution

Further diagnosis

Run the az acr check-health command to get more information about the health of the registry environment and optionally access to a target registry. For example, diagnose certain network connectivity or configuration problems.

See Check the health of an Azure container registry for command examples. If errors are reported, review the error reference and the following sections for recommended solutions.

If you're experiencing problems using an Azure Kubernetes Service with an integrated registry, run the az aks check-acr command to validate that the AKS cluster can reach the registry.

Note

Some network connectivity symptoms can also occur when there are issues with registry authentication or authorization. See Troubleshoot registry login.

Potential solutions

Configure client firewall access

To access a registry from behind a client firewall or proxy server, configure firewall rules to access the registry's public REST and data endpoints. If dedicated data endpoints are enabled, you need rules to access:

  • REST endpoint: <registryname>.azurecr.cn
  • Data endpoint(s): <registry-name>.<region>.data.azurecr.cn

For a geo-replicated registry, configure access to the data endpoint for each regional replica.

Behind an HTTPS proxy, ensure that both your Docker client and Docker daemon are configured for proxy behavior. If you change your proxy settings for the Docker daemon, be sure to restart the daemon.

Registry resource logs in the ContainerRegistryLoginEvents table may help diagnose an attempted connection that is blocked.

Related links:

Configure public access to registry

If accessing a registry over the internet, confirm the registry allows public network access from your client. By default, an Azure container registry allows access to the public registry endpoints from all networks. A registry can limit access to selected networks, or selected IP addresses.

If the registry is configured for a virtual network with a service endpoint, disabling public network access also disables access over the service endpoint. If your registry is configured for a virtual network with Private Link, IP network rules don't apply to the registry's private endpoints.

Related links:

Configure VNet access

Confirm that the virtual network is configured with either a private endpoint for Private Link or a service endpoint (preview). Currently an Azure Bastion endpoint isn't supported.

If a private endpoint is configured, confirm that DNS resolves the registry's public FQDN such as myregistry.azurecr.cn to the registry's private IP address.

  • Run the az acr check-health command with the --vnet parameter to confirm the DNS routing to the private endpoint in the virtual network.
  • Use a network utility such as dig or nslookup for DNS lookup.
  • Ensure that DNS records are configured for the registry FQDN and for each of the data endpoint FQDNs.

Review NSG rules and service tags used to limit traffic from other resources in the network to the registry.

If a service endpoint to the registry is configured, confirm that a network rule is added to the registry that allows access from that network subnet. The service endpoint only supports access from virtual machines and AKS clusters in the network.

If you want to restrict registry access using a virtual network in a different Azure subscription, ensure that you register the Microsoft.ContainerRegistry resource provider in that subscription. Register the resource provider for Azure Container Registry using the Azure portal, Azure CLI, or other Azure tools.

If Azure Firewall or a similar solution is configured in the network, check that egress traffic from other resources such as an AKS cluster is enabled to reach the registry endpoints.

Related links:

Configure service access

Currently, access to a container registry with network restrictions isn't allowed from several Azure services:

  • Microsoft Defender for Cloud can't perform image vulnerability scanning in a registry that restricts access to private endpoints, selected subnets, or IP addresses.
  • Resources of certain Azure services are unable to access a container registry with network restrictions, including Azure App Service and Azure Container Instances.

If access or integration of these Azure services with your container registry is required, remove the network restriction. For example, remove the registry's private endpoints, or remove or modify the registry's public access rules.

Starting January 2021, you can configure a network-restricted registry to allow access from select trusted services.

Related links:

Advanced troubleshooting

If collection of resource logs is enabled in the registry, review the ContainterRegistryLoginEvents log. This log stores authentication events and status, including the incoming identity and IP address. Query the log for registry authentication failures.

Related links:

Next steps

If you don't resolve your problem here, see the following options.