Azure 中的出站连接Outbound connections in Azure

Azure 通过多种不同的机制为客户部署提供出站连接。Azure provides outbound connectivity for customer deployments through several different mechanisms. 本文介绍具体的方案、其应用方式、工作原理以及管理方式。This article describes what the scenarios are, when they apply, how they work, and how to manage them.

Note

本文仅涵盖了资源管理器部署。This article covers Resource Manager deployments only. 有关 Azure 中的所有经典部署方案,请查看出站连接(经典)Review Outbound connections (Classic) for all Classic deployment scenarios in Azure.

Azure 中的部署可与 Azure 外部的公用 IP 地址空间中的终结点进行通信。A deployment in Azure can communicate with endpoints outside Azure in the public IP address space. 当实例启动到公共 IP 地址空间中的目标的出站流时,Azure 会动态将专用 IP 地址映射到公共 IP 地址。When an instance initiates an outbound flow to a destination in the public IP address space, Azure dynamically maps the private IP address to a public IP address. 创建此映射后,此出站发起流的返回流量还可以抵达发起流的专用 IP 地址。After this mapping is created, return traffic for this outbound originated flow can also reach the private IP address where the flow originated.

Azure 使用源网络地址转换 (SNAT) 来执行此功能。Azure uses source network address translation (SNAT) to perform this function. 当多个专用 IP 地址伪装成单个公共 IP 地址时,Azure 将使用端口地址转换 (PAT) 来伪装专用 IP 地址。When multiple private IP addresses are masquerading behind a single public IP address, Azure uses port address translation (PAT) to masquerade private IP addresses. 临时端口用于 PAT,是基于池大小预先分配的。Ephemeral ports are used for PAT and are preallocated based on pool size.

有多种出站方案There are multiple outbound scenarios. 可根据需要结合这些方案。You can combine these scenarios as needed. 请认真分析这些方案,以了解在部署模型和应用方案中应用这些方案时的功能、约束和模式。Review them carefully to understand the capabilities, constraints, and patterns as they apply to your deployment model and application scenario. 查看有关管理这些方案的指导。Review guidance for managing these scenarios.

Important

标准负载均衡器和标准公共 IP 为出站连接引入了新功能和不同的行为。Standard Load Balancer and Standard Public IP introduce new abilities and different behaviors to outbound connectivity. 它们不同于基本 SKU。They are not the same as Basic SKUs. 如果在使用标准 SKU 时需要出站连接,则必须使用标准公共 IP 地址或标准公共负载均衡器显式定义它。If you want outbound connectivity when working with Standard SKUs, you must explicitly define it either with Standard Public IP addresses or Standard public Load Balancer. 这包括在使用内部标准负载均衡器时创建出站连接。This includes creating outbound connectivity when using an internal Standard Load Balancer. 建议始终使用标准公共负载均衡器上的出站规则。We recommend you always use outbound rules on a Standard public Load Balancer. 方案 3不适用于标准 SKU。Scenario 3 is not available with Standard SKU. 这意味着使用内部标准负载均衡器时,如果需要出站连接,则需要采取步骤为后端池中的 VM 创建出站连接。That means when an internal Standard Load Balancer is used, you need to take steps to create outbound connectivity for the VMs in the backend pool if outbound connectivity is desired. 在出站连接的上下文中,单独的 VM、可用性集中的所有 VM、VMSS 中的所有实例都是一个组。In the context of outbound connectivity, a single standalone VM, all the VM's in an Availability Set, all the instances in a VMSS behave as a group. 这意味着,如果可用性集中的单个 VM 与标准 SKU 关联,则该可用性集中的所有 VM 实例现在都遵循相同的规则,就好像这些 VM 实例与标准 SKU 相关联一样,即使单个实例与标准 SKU 没有直接关联。This means, if a single VM in an Availability Set is associated with a Standard SKU, all VM instances within this Availability Set now behave by the same rules as if they are associated with Standard SKU, even if an individual instance is not directly associated with it. 请仔细查看整个文档以了解整体概念,查看标准负载均衡器了解 SKU 之间的差异,并查看出站规则Carefully review this entire document to understand the overall concepts, review Standard Load Balancer for differences between SKUs, and review outbound rules. 使用出站规则可以对出站连接的所有方面进行细化管理控制。Using outbound rules allows you fine grained control over all aspects of outbound connectivity.

方案概述Scenario overview

Azure 负载均衡器和相关资源是使用 Azure 资源管理器时显式定义的。Azure Load Balancer and related resources are explicitly defined when you're using Azure Resource Manager. Azure 目前提供三种不同的方法实现 Azure 资源管理器资源的出站连接。Azure currently provides three different methods to achieve outbound connectivity for Azure Resource Manager resources.

SKUSKUs 方案Scenario 方法Method IP 协议IP protocols 说明Description
标准、基本Standard, Basic 1.具有公共 IP 地址的 VM(有或没有负载均衡器)1. VM with Public IP address (with or without Load Balancer) SNAT,不使用端口伪装SNAT, port masquerading not used TCP、UDP、ICMP、ESPTCP, UDP, ICMP, ESP Azure 使用分配实例 NIC 的 IP 配置的公共 IP。Azure uses the public IP assigned to the IP configuration of the instance's NIC. 此实例具有所有可用的临时端口。The instance has all ephemeral ports available. 在使用标准负载均衡器时,应使用出站规则显式定义出站连接When using Standard Load Balancer, you should use outbound rules to explicitly define outbound connectivity
标准、基本Standard, Basic 2.与 VM 关联的公共负载均衡器(实例上没有公共 IP 地址)2. Public Load Balancer associated with a VM (no Public IP address on the instance) 使用负载均衡器前端进行端口伪装 (PAT) 的 SNATSNAT with port masquerading (PAT) using the Load Balancer frontends TCP、UDPTCP, UDP Azure 与多个专用 IP 地址共享公共负载均衡器前端的公共 IP 地址。Azure shares the public IP address of the public Load Balancer frontends with multiple private IP addresses. Azure 使用前端的临时端口进行 PAT。Azure uses ephemeral ports of the frontends to PAT.
无或基本none or Basic 3.独立 VM(无负载均衡器,无公共 IP 地址)3. Standalone VM (no Load Balancer, no Public IP address) 使用端口伪装 (PAT) 的 SNATSNAT with port masquerading (PAT) TCP、UDPTCP, UDP Azure 自动指定用于 SNAT 的公共 IP 地址,与可用性集的多个专用 IP 地址共享此公共 IP 地址,并使用此公共 IP 地址的临时端口。Azure automatically designates a public IP address for SNAT, shares this public IP address with multiple private IP addresses of the availability set, and uses ephemeral ports of this public IP address. 此方案是前述方案的回退方案。This scenario is a fallback for the preceding scenarios. 如果需要可见性和控制,则我们不建议采用。We don't recommend it if you need visibility and control.

如果不希望 VM 与 Azure 外部的公共 IP 地址空间中的终结点通信,则可以根据需要使用网络安全组 (NSG) 来阻止访问。If you don't want a VM to communicate with endpoints outside Azure in public IP address space, you can use network security groups (NSGs) to block access as needed. 阻止出站连接部分详细介绍了 NSG。The section Preventing outbound connectivity discusses NSGs in more detail. 本文不会介绍有关在无任何出站访问权限的情况下,如何设计和管理虚拟网络的设计和实施指导。Guidance on designing, implementing, and managing a virtual network without any outbound access is outside the scope of this article.

场景 1:使用公共 IP 地址的 VMScenario 1: VM with Public IP address

在此场景中,向 VM 分配了公共 IP。In this scenario, the VM has a Public IP assigned to it. 就出站连接而言,VM 是否经过负载均衡并不重要。As far as outbound connections are concerned, it doesn't matter whether the VM is load balanced or not. 此方案优先于其他方案。This scenario takes precedence over the others. 使用公共 IP 地址时,VM 将该公共 IP 地址用于所有出站流。When a Public IP address is used, the VM uses the Public IP address for all outbound flows.

分配到 VM 的公共 IP 属于 1 对 1 关系(而不是 1 对多关系),并实现为无状态的 1 对 1 NAT。A public IP assigned to a VM is a 1:1 relationship (rather than 1: many) and implemented as a stateless 1:1 NAT. 不使用端口伪装 (PAT),并且 VM 具有所有可供使用的临时端口。Port masquerading (PAT) is not used, and the VM has all ephemeral ports available for use.

如果应用程序启动很多出站流,并且遇到 SNAT 端口耗尽的情况,可以考虑分配公共 IP 地址以缓解 SNAT 约束If your application initiates many outbound flows and you experience SNAT port exhaustion, consider assigning a Public IP address to mitigate SNAT constraints. 请查看管理 SNAT 耗尽Review Managing SNAT exhaustion in its entirety.

场景 2:无公共 IP 地址的负载均衡 VMScenario 2: Load-balanced VM without a Public IP address

在此方案中,VM 是公共负载均衡器池的一部分。In this scenario, the VM is part of a public Load Balancer backend pool. 没有分配给 VM 的公共 IP 地址。The VM does not have a public IP address assigned to it. 必须为负载均衡器资源配置一个负载均衡器规则,以在公共 IP 前端与后端池之间创建链接。The Load Balancer resource must be configured with a load balancer rule to create a link between the public IP frontend with the backend pool.

如果没有完成此规则配置,则行为将如适用于没有公共 IP 的独立 VM 方案中所述。If you do not complete this rule configuration, the behavior is as described in the scenario for Standalone VM with no Public IP. 不需要在规则中添加后端池的正常运行的侦听器或者运行状况探测就能成功实现。It is not necessary for the rule to have a working listener in the backend pool for the health probe to succeed.

当负载均衡的 VM 创建出站流时,Azure 将此出站流的专用源 IP 地址转换为公共负载均衡器前端的公共 IP 地址。When the load-balanced VM creates an outbound flow, Azure translates the private source IP address of the outbound flow to the public IP address of the public Load Balancer frontend. Azure 使用 (SNAT) 来执行此功能。Azure uses SNAT to perform this function. Azure 还使用 PAT 来伪装公共 IP 地址后面的多个专用 IP 地址。Azure also uses PAT to masquerade multiple private IP addresses behind a public IP address.

使用负载均衡器的公共 IP 地址前端的临时端口区分由 VM 产生的各个流。Ephemeral ports of the load balancer's public IP address frontend are used to distinguish individual flows originated by the VM. 创建出站流后,SNAT 动态使用预先分配的临时端口SNAT dynamically uses preallocated ephemeral ports when outbound flows are created. 在此情况下,用于 SNAT 的临时端口被称为 SNAT 端口。In this context, the ephemeral ports used for SNAT are called SNAT ports.

SNAT 端口是按照了解 SNAT 和 PAT 部分中所述预先分配的。SNAT ports are pre-allocated as described in the Understanding SNAT and PAT section. 它们是可能会耗尽的有限资源。They're a finite resource that can be exhausted. 因此了解它们的使用方式很重要。It's important to understand how they are consumed. 请查看管理 SNAT 耗尽,了解如何根据需要进行设计和缓解。To understand how to design for this consumption and mitigate as necessary, review Managing SNAT exhaustion.

如果多个公共 IP 地址与负载均衡器基本版相关联,则所有这些公共 IP 地址都是出站流的候选项,并且会随机选择其中一个。When multiple public IP addresses are associated with Load Balancer Basic, any of these public IP addresses are a candidate for outbound flows, and one is selected at random.

场景 3:无公共 IP 地址的独立 VMScenario 3: Standalone VM without a Public IP address

在此场景中,VM 不是公共负载均衡器池的一部分(也不是内部标准负载均衡器池的一部分),并且没有分配给它的公共 IP 地址。In this scenario, the VM is not part of a public Load Balancer pool (and not part of an internal Standard Load Balancer pool) and does not have a Public IP address assigned to it. 当 VM 创建出站流时,Azure 将此出站流的专用源 IP 地址转换为公共源 IP 地址。When the VM creates an outbound flow, Azure translates the private source IP address of the outbound flow to a public source IP address. 用于此出站流的公共 IP 地址是不可配置的,并且不会影响订阅的公共 IP 资源限制。The public IP address used for this outbound flow is not configurable and does not count against the subscription's public IP resource limit. 此公共 IP 地址不属于你,不能保留。This public IP address does not belong to you and cannot be reserved. 如果重新部署 VM、可用性集或虚拟机规模集,则将释放此公共 IP 地址并请求新的公共 IP 地址。If you redeploy the VM or Availability Set or virtual machine scale set, this public IP address will be released and a new public IP address requested. 请不要使用此方案将 IP 地址加入允许列表。Do not use this scenario for whitelisting IP addresses. 而是使用其他两个方案之一,其中你显式声明出站方案和要用于出站连接的公共 IP 地址。Instead, use one of the other two scenarios where you explicitly declare the outbound scenario and the public IP address to be used for outbound connectivity.

Important

仅当附加了内部基本负载均衡器时,此场景才适用。 This scenario also applies when only an internal Basic Load Balancer is attached. 如果已将内部标准负载均衡器附加到 VM,则场景 3 不适用。 Scenario 3 is not available when an internal Standard Load Balancer is attached to a VM. 除了使用内部标准负载均衡器以外,还必须显式创建场景 1场景 2You must explicitly create scenario 1 or scenario 2 in addition to using an internal Standard Load Balancer.

Azure 结合端口伪装 (PAT) 使用 SNAT 来执行此功能。Azure uses SNAT with port masquerading (PAT) to perform this function. 此方案类似于方案 2,但无法控制使用的 IP 地址。This scenario is similar to scenario 2, except there is no control over the IP address used. 这是方案 1 和方案 2 不存在时的回退方案。This is a fallback scenario for when scenarios 1 and 2 do not exist. 如果需要控制出站地址,则我们不建议使用此方案。We don't recommend this scenario if you want control over the outbound address. 如果出站连接是应用程序的关键部分,应该选择另一种方案。If outbound connections are a critical part of your application, you should choose another scenario.

SNAT 端口是根据了解 SNAT 和 PAT 部分中所述预先分配的。SNAT ports are preallocated as described in the Understanding SNAT and PAT section. 共享可用性集的 VM 数目决定了适用的预分配层。The number of VMs sharing an Availability Set determines which preallocation tier applies. 没有可用性集的独立 VM 实际上是用于确定预分配(1024 SNAT 端口)的、包含 1 个 VM 的池。A standalone VM without an Availability Set is effectively a pool of 1 for the purposes of determining preallocation (1024 SNAT ports). SNAT 端口是可能会被耗尽的有限资源。SNAT ports are a finite resource that can be exhausted. 因此了解它们的使用方式很重要。It's important to understand how they are consumed. 请查看管理 SNAT 耗尽,了解如何根据需要进行设计和缓解。To understand how to design for this consumption and mitigate as necessary, review Managing SNAT exhaustion.

多个组合方案Multiple, combined scenarios

可以结合前面部分中所述的方案来实现特定的效果。You can combine the scenarios described in the preceding sections to achieve a particular outcome. 存在多个方案时,优先顺序如下:方案 1 优先于方案 23When multiple scenarios are present, an order of precedence applies: scenario 1 takes precedence over scenario 2 and 3. 方案 2 优先于方案 3Scenario 2 overrides scenario 3.

例如,在 Azure 资源管理器部署中,应用程序严重依赖于与有限数量的目标建立出站连接,但也通过负载均衡器前端接收入站流。An example is an Azure Resource Manager deployment where the application relies heavily on outbound connections to a limited number of destinations but also receives inbound flows over a Load Balancer frontend. 在此情况下,可以结合方案 1 和 2 来缓解问题。In this case, you can combine scenarios 1 and 2 for relief. 有关其他模式,请查看管理 SNAT 耗尽For additional patterns, review Managing SNAT exhaustion.

对出站流使用多个前端Multiple frontends for outbound flows

标准负载均衡器Standard Load Balancer

存在多个(公共)IP 前端时,标准负载均衡器同时使用出站流的所有候选项。Standard Load Balancer uses all candidates for outbound flows at the same time when multiple (public) IP frontends is present. 如果对出站连接启用了负载均衡规则,则每个前端的可用预分配 SNAT 端口数将会倍增。Each frontend multiplies the number of available preallocated SNAT ports if a load balancing rule is enabled for outbound connections.

可以使用一个新的负载均衡规则选项,来禁止对出站连接使用某个前端 IP 地址:You can choose to suppress a frontend IP address from being used for outbound connections with a new load balancing rule option:

      "loadBalancingRules": [
        {
          "disableOutboundSnat": false
        }
      ]

通常,disableOutboundSnat 选项默认为 false,表示此规则将为负载均衡规则的后端池中关联的 VM 编制出站 SNAT。Normally, the disableOutboundSnat option defaults to false and signifies that this rule programs outbound SNAT for the associated VMs in the backend pool of the load balancing rule. 可将 disableOutboundSnat 更改为 true,防止负载均衡器对此负载均衡规则的后端池中 VM 的出站连接使用关联的前端 IP 地址。The disableOutboundSnat can be changed to true to prevent Load Balancer from using the associated frontend IP address for outbound connections for the VMs in the backend pool of this load balancing rule. 此外,仍可以根据多个组合方案中所述,为出站流指定特定的 IP 地址。And you can also still designate a specific IP address for outbound flows as described in Multiple, combined scenarios as well.

负载均衡器基本版Load Balancer Basic

多个(公共)IP 前端适用于出站流时,负载均衡器基本版将选择单个前端用于出站流。Load Balancer Basic chooses a single frontend to be used for outbound flows when multiple (public) IP frontends are candidates for outbound flows. 此项选择不可配置,应将选择算法视为随机。This selection is not configurable, and you should consider the selection algorithm to be random. 可以根据多个组合方案中所述,为出站流指定特定的 IP 地址。You can designate a specific IP address for outbound flows as described in Multiple, combined scenarios.

了解 SNAT 和 PATUnderstanding SNAT and PAT

端口伪装 SNAT (PAT)Port masquerading SNAT (PAT)

公共负载均衡器资源与 VM 实例相关联时,将重写每个出站连接源。When a public Load Balancer resource is associated with VM instances, each outbound connection source is rewritten. 出站连接源从虚拟网络专用 IP 地址空间重新写入负载均衡器的前端公共 IP 地址。The source is rewritten from the virtual network private IP address space to the frontend Public IP address of the load balancer. 在公共 IP 地址空间中,流的 5 元组(源 IP 地址、源端口、IP 转换协议、目标 IP 地址、目标端口)必须唯一。In the public IP address space, the 5-tuple of the flow (source IP address, source port, IP transport protocol, destination IP address, destination port) must be unique. 端口伪装 SNAT 可与 TCP 或 UDP IP 协议一起使用。Port masquerading SNAT can be used with either TCP or UDP IP protocols.

重写专用源 IP 地址后,临时端口(SNAT 端口)用于实现此目的,因为多个流源自单个公共 IP 地址。Ephemeral ports (SNAT ports) are used to achieve this after rewriting the private source IP address, because multiple flows originate from a single public IP address. 伪装 SNAT 算法的端口为 UDP 与 TCP 分配不同的 SNAT 端口。The port masquerading SNAT algorithm allocates SNAT ports differently for UDP versus TCP.

TCP SNAT 端口TCP SNAT Ports

每个到单个目标 IP 地址、端口的流使用一个 SNAT 端口。One SNAT port is consumed per flow to a single destination IP address, port. 对于到相同的目标 IP 地址、端口和协议的多个 TCP 流,每个 TCP 流使用一个 SNAT 端口。For multiple TCP flows to the same destination IP address, port, and protocol, each TCP flow consumes a single SNAT port. 这可以确保源自相同的公共 IP 地址,并到相同的目标 IP 地址、端口和协议的流的唯一性。This ensures that the flows are unique when they originate from the same public IP address and go to the same destination IP address, port, and protocol.

每个流均流到不同目标 IP 地址、端口和协议的多个流共用一个 SNAT 端口。Multiple flows, each to a different destination IP address, port, and protocol, share a single SNAT port. 目标 IP 地址、端口和协议使流保持唯一,无需使用其他源端口来区分公共 IP 地址空间中的流。The destination IP address, port, and protocol make flows unique without the need for additional source ports to distinguish flows in the public IP address space.

UDP SNAT 端口UDP SNAT Ports

UDP SNAT 端口由与 TCP SNAT 端口不同的算法管理。UDP SNAT ports are managed by a different algorithm than TCP SNAT ports. 负载均衡器对 UDP 使用称为“端口受限锥形 NAT”的算法。Load Balancer uses an algorithm known as "port-restricted cone NAT" for UDP. 无论目标 IP 地址、端口如何,每个流都会使用一个 SNAT 端口。One SNAT port is consumed for each flow, irrespective of destination IP address, port.

SNAT 端口重用SNAT port reuse

释放某个端口以后,即可根据需要重复使用该端口。Once a port has been released, the port is available for reuse as needed. 可以将 SNAT 端口视为一个适用于给定场景的从低到高的序列,第一个可用 SNAT 端口用于新的连接。You can think of SNAT ports as a sequence from lowest to highest available for a given scenario, and the first available SNAT port is used for new connections.

耗尽Exhaustion

如果 SNAT 端口资源已经耗尽,那么在现有流释放 SNAT 端口之前出站流会失败。When SNAT port resources are exhausted, outbound flows fail until existing flows release SNAT ports. 当流关闭时,负载均衡器将回收 SNAT 端口,并使用 4 分钟空闲超时回收空闲流中的 SNAT 端口。Load Balancer reclaims SNAT ports when the flow closes and uses a 4-minute idle timeout for reclaiming SNAT ports from idle flows.

由于使用的算法不同,UDP SNAT端口的耗尽速度通常比 TCP SNAT 端口快得多。UDP SNAT ports generally exhaust much faster than TCP SNAT ports due to the difference in algorithm used. 在进行设计和规模测试时必须考虑到这种差异。You must design and scale test with this difference in mind.

有关可以使用哪些模式来缓解通常导致 SNAT 端口耗尽的状态,请查看管理 SNAT 部分。For patterns to mitigate conditions that commonly lead to SNAT port exhaustion, review the Managing SNAT section.

端口伪装 SNAT (PAT) 的临时端口预先分配Ephemeral port preallocation for port masquerading SNAT (PAT)

使用端口伪装 SNAT (PAT) 时,Azure 使用某种算法根据后端池的大小来确定可用的预先分配 SNAT 端口数目。Azure uses an algorithm to determine the number of preallocated SNAT ports available based on the size of the backend pool when using port masquerading SNAT (PAT). SNAT 端口是可用于特定公共 IP 源地址的临时端口。SNAT ports are ephemeral ports available for a particular public IP source address.

将分别为 UDP 和 TCP 预分配相同数量的 SNAT 端口,并根据 IP 传输协议独立地使用这些端口。The same number of SNAT ports are preallocated for UDP and TCP respectively and consumed independently per IP transport protocol. 但是,SNAT 端口使用情况会因流是 UDP 还是 TCP 而有所不同。However, the SNAT port usage is different depending on whether the flow is UDP or TCP.

Important

标准 SKU SNAT 编程依据 IP 传输协议并且派生自负载均衡规则。Standard SKU SNAT programming is per IP transport protocol and derived from the load balancing rule. 如果只存在一个 TCP 负载均衡规则,则 SNAT 仅可用于 TCP。If only a TCP load balancing rule exists, SNAT is only available for TCP. 如果只有一个 TCP 负载均衡规则并且 UDP 需要出站 SNAT,请创建从同一个前端到同一个后端池的 UDP 负载均衡规则。If you have only a TCP load balancing rule and need outbound SNAT for UDP, create a UDP load balancing rule from the same frontend to the same backend pool. 这将触发针对 UDP 的 SNAT 编程。This will trigger SNAT programming for UDP. 不需要采用工作规则或运行状况探测。A working rule or health probe is not required. 无论在负载均衡规则中指定了什么传输协议,基本 SKU SNAT 都始终针对 IP 传输协议编写 SNAT 程序。Basic SKU SNAT always programs SNAT for both IP transport protocol, irrespective of the transport protocol specified in the load balancing rule.

Azure 向每个 VM 的 NIC IP 配置预先分配 SNAT 端口。Azure preallocates SNAT ports to the IP configuration of the NIC of each VM. 将 IP 配置添加到池后,将会根据后端池的大小预先分配此 IP 配置的 SNAT 端口。When an IP configuration is added to the pool, the SNAT ports are preallocated for this IP configuration based on the backend pool size. 创建出站流后,当流关闭或空闲超时时,PAT 动态使用(不超过预先分配的限制)和释放这些端口。When outbound flows are created, PAT dynamically consumes (up to the preallocated limit) and releases these ports when the flow closes or idle timeouts happen.

下表显示了针对后端池大小层的 SNAT 端口预分配:The following table shows the SNAT port preallocations for tiers of backend pool sizes:

池大小(VM 实例)Pool size (VM instances) 每个 IP 配置的预先分配 SNAT 端口Preallocated SNAT ports per IP configuration
1-501-50 1,0241,024
51-10051-100 512512
101-200101-200 256256
201-400201-400 128128
401-800401-800 6464
801-1,000801-1,000 3232

Note

结合多个前端使用标准负载均衡器时,上表中每个前端 IP 地址的可用 SNAT 端口数目将会倍增。When using Standard Load Balancer with multiple frontends, each frontend IP address multiplies the number of available SNAT ports in the previous table. 例如,如果某个后端池包含 50 个 VM 和 2 个负载均衡规则,并且每个 VM 使用独立的前端 IP 地址,则该后端池将为每个 IP 配置使用 2048 个 (2 x 1024) SNAT 端口。For example, a backend pool of 50 VM's with 2 load balancing rules, each with a separate frontend IP address, will use 2048 (2x 1024) SNAT ports per IP configuration. 参阅有关多个前端的详细信息。See details for multiple frontends.

请记住,可用的 SNAT 端口数不会直接转换为流数。Remember that the number of SNAT ports available does not translate directly to number of flows. 可以针对多个唯一目标重用单个 SNAT 端口。A single SNAT port can be reused for multiple unique destinations. 仅当需要使流保持唯一时,才使用端口。Ports are consumed only if it's necessary to make flows unique. 有关设计和缓解指导,请参阅如何管理这项可耗尽的资源;另请参阅介绍 PAT 的部分。For design and mitigation guidance, refer to the section about how to manage this exhaustible resource and the section that describes PAT.

更改后端池大小可能会影响建立的某些流。Changing the size of your backend pool might affect some of your established flows. 如果后端池大小递增并转换为下一层,则在转换为下一个更大的后端池层期间,一半的预先分配 SNAT 端口将被回收。If the backend pool size increases and transitions into the next tier, half of your preallocated SNAT ports are reclaimed during the transition to the next larger backend pool tier. 与回收的 SNAT 端口关联的流会超时,必须重新建立连接。Flows that are associated with a reclaimed SNAT port will time out and must be reestablished. 如果尝试新流,则只要预先分配的端口可用,则该流就能立即成功。If a new flow is attempted, the flow will succeed immediately as long as preallocated ports are available.

如果后端池减小并转换到更低层级,可用的 SNAT 端口数会增多。If the backend pool size decreases and transitions into a lower tier, the number of available SNAT ports increases. 在这种情况下,现有的分配 SNAT 端口及其相应的流不会受到影响。In this case, existing allocated SNAT ports and their respective flows are not affected.

SNAT 端口分配特定于 IP 传输协议(TCP 和 UDP 是分别维护的),并在以下条件下释放:SNAT ports allocations are IP transport protocol specific (TCP and UDP are maintained separately) and are released under the following conditions:

TCP SNAT 端口释放TCP SNAT port release

  • 如果服务器/客户端均发送 FINACK,则 SNAT 端口在 240 秒后释放。If either server/client sends FINACK, SNAT port will be released after 240 seconds.
  • 如果出现 RST,则 SNAT 端口在 15 秒后释放。If a RST is seen, SNAT port will be released after 15 seconds.
  • 如果已达到空闲超时,则会释放端口。If idle timeout has been reached, port is released.

UDP SNAT 端口释放UDP SNAT port release

  • 如果已达到空闲超时,则会释放端口。If idle timeout has been reached, port is released.

解决问题Problem solving

本部分旨在帮助解决 SNAT 耗尽的问题,以及 Azure 中的出站连接可能出现的其他情况。This section is intended to help mitigate SNAT exhaustion and that can occur with outbound connections in Azure.

应对 SNAT (PAT) 端口耗尽问题Managing SNAT (PAT) port exhaustion

用于 PAT临时端口是可用尽的资源,如无公共 IP 地址的独立 VM无公共 IP 地址的负载均衡 VM 中所述。Ephemeral ports used for PAT are an exhaustible resource, as described in Standalone VM without a Public IP address and Load-balanced VM without a Public IP address.

如果知道正在启动与同一目标 IP 地址和端口的多个出站 TCP 或 UDP 连接,观察失败的出站连接,或者支持人员通知已耗尽 SNAT 端口(PAT 使用的预先分配临时端口),则有几个常见缓解选项可供选择。If you know that you're initiating many outbound TCP or UDP connections to the same destination IP address and port, and you observe failing outbound connections or are advised by support that you're exhausting SNAT ports (preallocated ephemeral ports used by PAT), you have several general mitigation options. 查看这些选项,确定可用且最适合自己的方案的选项。Review these options and decide what is available and best for your scenario. 一个或多个选项可能有助于管理此方案。It's possible that one or more can help manage this scenario.

如果难以理解出站连接行为,可以使用 IP 堆栈统计 (netstat)。If you are having trouble understanding the outbound connection behavior, you can use IP stack statistics (netstat). 或者使用数据包捕获来观察连接行为。Or it can be helpful to observe connection behaviors by using packet captures. 可以在实例的来宾 OS 中执行这些数据包捕获,或使用网络观察程序来捕获数据包You can perform these packet captures in the guest OS of your instance or use Network Watcher for packet capture.

修改应用程序以重复使用连接Modify the application to reuse connections

在应用程序中重复使用连接,可以降低对用于 SNAT 的临时端口的需求。You can reduce demand for ephemeral ports that are used for SNAT by reusing connections in your application. 这尤其适用于 HTTP/1.1 这样的协议,在这些协议中,默认重复使用连接。This is especially true for protocols like HTTP/1.1, where connection reuse is the default. 其他使用 HTTP 作为其传输(如 REST)的协议也可以因此受益。And other protocols that use HTTP as their transport (for example, REST) can benefit in turn.

对每个请求来说,重复使用总是优于单独的原子 TCP 连接。Reuse is always better than individual, atomic TCP connections for each request. 重复使用可以带来更高的性能和非常高效的 TCP 事务。Reuse results in more performant, very efficient TCP transactions.

修改应用程序以使用连接池Modify the application to use connection pooling

可以在应用程序中使用连接池方案,其中在一组固定的连接(在可能的情况下,重复使用每一个)上,内部分布请求。You can employ a connection pooling scheme in your application, where requests are internally distributed across a fixed set of connections (each reusing where possible). 此方案会限制正在使用的临时端口的数量,并创建更加可预测的环境。This scheme constrains the number of ephemeral ports in use and creates a more predictable environment. 此方案还可以通过在操作答复上单个连接阻塞时允许多个同时操作,增加请求的吞吐量。This scheme can also increase the throughput of requests by allowing multiple simultaneous operations when a single connection is blocking on the reply of an operation.

连接池可能已经存在于正在使用的框架中,以开发应用程序或应用程序的配置设置。Connection pooling might already exist within the framework that you're using to develop your application or the configuration settings for your application. 可将连接池与连接重复使用相结合。You can combine connection pooling with connection reuse. 多个请求使用指向相同目标 IP 地址和端口的可预测固定数量的端口。Your multiple requests then consume a fixed, predictable number of ports to the same destination IP address and port. 请求还可以从 TCP 事务的高效使用中受益,从而减少延迟和资源利用。The requests also benefit from efficient use of TCP transactions reducing latency and resource utilization. UDP 事务也能从中受益,因为管理 UDP 流数又能避免耗尽情况和管理 SNAT 端口利用率。UDP transactions can also benefit, because managing the number of UDP flows can in turn avoid exhaust conditions and manage the SNAT port utilization.

修改应用程序以使用主动性较低的重试逻辑Modify the application to use less aggressive retry logic

当用于 PAT预先分配临时端口耗尽或应用程序故障发生时,无衰减或回退逻辑的积极重试或暴力重试会使耗尽状况再次发生或一直持续。When preallocated ephemeral ports used for PAT are exhausted or application failures occur, aggressive or brute force retries without decay and backoff logic cause exhaustion to occur or persist. 使用主动性较低的重试逻辑,可以降低对临时端口的需求。You can reduce demand for ephemeral ports by using a less aggressive retry logic.

临时端口有 4 分钟的空闲超时(不可调整)。Ephemeral ports have a 4-minute idle timeout (not adjustable). 如果重试太过积极,则消耗没有机会进行自行清除。If the retries are too aggressive, the exhaustion has no opportunity to clear up on its own. 因此,应用程序停用事务的方式和频率对于设计至关重要。Therefore, considering how--and how often--your application retries transactions is a critical part of the design.

向每个 VM 分配公共 IPAssign a Public IP to each VM

分配公共 IP 地址会将方案更改为 VM 的公共 IPAssigning a Public IP address changes your scenario to Public IP to a VM. 用于各 VM 的公共 IP 的所有临时端口都可供 VM 使用。All ephemeral ports of the public IP that are used for each VM are available to the VM. (与以下方案相反:公共 IP 的临时端口与同相应后端池关联的 VM 的所有临时端口共享)。需要作出一些权衡,比如公共 IP 地址的额外成本和将大量个人 IP 地址列入允许列表所产生的潜在影响。(As opposed to scenarios where ephemeral ports of a public IP are shared with all the VMs associated with the respective backend pool.) There are trade-offs to consider, such as the additional cost of public IP addresses and the potential impact of whitelisting a large number of individual IP addresses.

Note

此选项不适用于 Web 辅助角色。This option is not available for web worker roles.

使用多个前端Use multiple frontends

使用公共标准负载均衡器时,可为出站连接分配多个前端 IP 地址,并将可用的 SNAT 端口数倍增When using public Standard Load Balancer, you assign multiple frontend IP addresses for outbound connections and multiply the number of SNAT ports available. 创建前端 IP 配置、规则和后端池,以触发将 SNAT 编程到前端公共 IP。Create a frontend IP configuration, rule, and backend pool to trigger the programming of SNAT to the public IP of the frontend. 该规则不需要运行,并且运行状况探测不需要成功。The rule does not need to function and a health probe does not need to succeed. 如果对入站连接(而不仅仅是出站连接)也使用多个前端,应使用自定义的运行状况探测来确保可靠性。If you do use multiple frontends for inbound as well (rather than just for outbound), you should use custom health probes well to ensure reliability.

Note

在大多数情况下,SNAT 端口耗尽是设计不当的征兆。In most cases, exhaustion of SNAT ports is a sign of bad design. 在使用更多前端来添加 SNAT 端口之前,请确保已了解端口耗尽的原因。Make sure you understand why you are exhausting ports before using more frontends to add SNAT ports. 否则可能会忽视问题,导致以后出现故障。You may be masking a problem which can lead to failure later.

横向扩展Scale out

预分配端口是根据后端池大小分配的并且分组到各个层中,这样,当某些端口必须重新分配以适应后端池更大的下一个层时,可以最大限度地减少中断。Preallocated ports are assigned based on the backend pool size and grouped into tiers to minimize disruption when some of the ports have to be reallocated to accommodate the next larger backend pool size tier. 有可能可以通过将后端池扩展到给定层的最大大小来提高给定前端的 SNAT 端口利用率。You may have an option to increase the intensity of SNAT port utilization for a given frontend by scaling your backend pool to the maximum size for a given tier. 这需要应用程序高效地进行横向扩展。This requires for the application to scale out efficiently.

例如,后端池中的两个虚拟机对于每个 IP 配置将有 1024 个 SNAT 端口可用,整个部署总共有 2048 个 SNAT 端口。For example, two virtual machines in the backend pool would have 1024 SNAT ports available per IP configuration, allowing a total of 2048 SNAT ports for the deployment. 如果部署已增加到 50 台虚拟机,虽然每台虚拟机的预分配端口数保持不变,但整个部署可以使用 51,200 (50 x 1024) 个 SNAT 端口。If the deployment were to be increased to 50 virtual machines, even though the number of preallocated ports remains constant per virtual machine, a total of 51,200 (50 x 1024) SNAT ports can be used by the deployment. 如果希望横向扩展部署,请检查每层的预分配端口的数量以确保将横向扩展规划为各自层的最大容量。If you wish to scale out your deployment, check the number of preallocated ports per tier to make sure you shape your scale out to the maximum for the respective tier. 在上述示例中,如果选择了横向扩展到 51 个而非 50 个实例,则你将提升到下一个层,最终,每台 VM 的 SNAT 端口数以及端口总数会更少。In the preceding example, if you had chosen to scale out to 51 instead of 50 instances, you would progress to the next tier and end up with fewer SNAT ports per VM as well as in total.

如果横向扩展到后端池较大的下一层,并且需要重新分配已分配端口,则部分出站连接可能会超时。If you scale out to the next larger backend pool size tier, there is potential for some of your outbound connections to time out if allocated ports have to be reallocated. 如果仅使用部分 SNAT 端口,则在后端池较大的下一层中横向扩展无意义。If you are only using some of your SNAT ports, scaling out across the next larger backend pool size is inconsequential. 每次移动到后端池的下一层时,半数现有端口将重新分配。Half the existing ports will be reallocated each time you move to the next backend pool tier. 如果不希望发生此行为,则需要将部署规划为层大小。If you don't want this to take place, you need to shape your deployment to the tier size. 或者确保应用程序可以根据需要进行检测和重试。Or make sure your application can detect and retry as necessary. TCP keepalive 可以帮助检测 SNAT 端口何时由于被重新分配而不再工作。TCP keepalives can assist in detect when SNAT ports no longer function due to being reallocated.

保持 keepalive 重置出站空闲超时Use keepalives to reset the outbound idle timeout

出站连接有 4 分钟的空闲超时。Outbound connections have a 4-minute idle timeout. 此超时不可调整。This timeout is not adjustable. 但是,可以根据需要使用传输(例如 TCP keepalive)或应用层 keepalive 来刷新空闲流和重置此空闲超时。However, you can use transport (for example, TCP keepalives) or application-layer keepalives to refresh an idle flow and reset this idle timeout if necessary.

使用 TCP keepalive 时,在连接的一端启用它们就足够了。When using TCP keepalives, it is sufficient to enable them on one side of the connection. 例如,若要重置流的空闲计时器,在服务器端启用它们就足够了,没有必要在两端都启动 TCP keepalive。For example, it is sufficient to enable them on the server side only to reset the idle timer of the flow and it is not necessary for both sides to initiated TCP keepalives. 应用程序层(包括数据库客户端-服务器配置)也存在类似的概念。Similar concepts exist for application layer, including database client-server configurations. 检查服务器端对于特定于应用程序的 keepalive 存在哪些选项。Check the server side for what options exist for application specific keepalives.

发现 VM 使用的公共 IPDiscovering the public IP that a VM uses

有多种方法来确定出站连接的公共源 IP 地址。There are many ways to determine the public source IP address of an outbound connection. OpenDNS 提供了一种服务可以向你显示 VM 的公共 IP 地址。OpenDNS provides a service that can show you the public IP address of your VM.

使用 nslookup 命令,可以将名称 myip.opendns.com 的 DNS 查询发送到 OpenDNS 解析程序。By using the nslookup command, you can send a DNS query for the name myip.opendns.com to the OpenDNS resolver. 该服务返回用于发送此查询的源 IP 地址。The service returns the source IP address that was used to send the query. 在 VM 中运行以下查询时,返回的是用于该 VM 的公共 IP。When you run the following query from your VM, the response is the public IP used for that VM:

nslookup myip.opendns.com resolver1.opendns.com

阻止出站连接Preventing outbound connectivity

有时允许 VM 创建出站流是不可取的。Sometimes it's undesirable for a VM to be allowed to create an outbound flow. 或者,可能需要管理哪些目标可以通过出站流访问或哪些目标可以启动入站流。Or there might be a requirement to manage which destinations can be reached with outbound flows, or which destinations can begin inbound flows. 在此情况下,可以使用网络安全组管理 VM 可访问的目标。In this case, you can use network security groups to manage the destinations that the VM can reach. 还可以使用 NSG 来管理可启动入站流的公共目标。You can also use NSGs to manage which public destination can initiate inbound flows.

将 NSG 应用于负载均衡的 VM 时,需要注意服务标记默认安全规则When you apply an NSG to a load-balanced VM, pay attention to the service tags and default security rules. 必须确保 VM 可以接收来自 Azure 负载均衡器的运行状况探测请求。You must ensure that the VM can receive health probe requests from Azure Load Balancer.

如果 NSG 阻止来自 AZURE_LOADBALANCER 默认标记的运行状况探测请求,那么 VM 的运行状况探测程序将失败,并且 VM 被标记为停机。If an NSG blocks health probe requests from the AZURE_LOADBALANCER default tag, your VM health probe fails and the VM is marked down. 负载均衡器停止向此 VM 发送新流。Load Balancer stops sending new flows to that VM.

限制Limitations

  • 在门户中配置负载均衡规则时,不能将 DisableOutboundSnat 用作选项。DisableOutboundSnat is not available as an option when configuring a load balancing rule in the portal. 请改用 REST、模板或客户端工具。Use REST, template, or client tools instead.
  • 考虑到 VNet 出现之前的服务和其他平台服务的运行方式带来的副作用,只有在使用内部标准负载均衡器的情况下,才可以访问没有 VNet 和其他 Azure 平台服务的 Web 辅助角色。Web Worker Roles without a VNet and other Azure platform services can be accessible when only an internal Standard Load Balancer is used due to a side effect from how pre-VNet services and other platform services function. 请勿依赖此副作用,因为相应的服务本身或底层平台可能会在不通知的情况下进行更改。Do not rely on this side effect as the respective service itself or the underlying platform may change without notice. 在仅使用内部标准负载均衡器时,必须始终假定需要明确创建出站连接。You must always assume you need to create outbound connectivity explicitly if desired when using an internal Standard Load Balancer only. 本文中所述的默认 SNAT 方案 3 不可用。The default SNAT scenario 3 described in this article is not available.

后续步骤Next steps