非对称路由与多个网络路径Asymmetric routing with multiple network paths

本文说明当网络源与目标之间有多个路径时,正向和返回网络流量如何选择不同的路由。This article explains how forward and return network traffic might take different routes when multiple paths are available between network source and destination.

只有在理解两个概念之后,才能了解非对称路由。It's important to understand two concepts to understand asymmetric routing. 一个概念是多个网络路径的影响。One is the effect of multiple network paths. 另一个概念是设备(如防火墙)如何保持状态。The other is how devices, like a firewall, keep state. 这种类型的设备称为有状态设备。These types of devices are called stateful devices. 这两个因素的组合会造成有状态设备丢弃网络流量的情况,因为有状态设备未检测到源自设备本身的流量。A combination of these two factors creates scenarios in which network traffic is dropped by a stateful device because the stateful device didn't detect that traffic originated with the device itself.

多个网络路径Multiple network paths

当企业网络只通过其 Internet 服务提供商与 Internet 建立了一条链路时,往返 Internet 的所有流量都会经过同一条路径。When an enterprise network has only one link to the Internet through their Internet service provider, all traffic to and from the Internet travels the same path. 公司往往会购买多条线路作为冗余路径,以改善网络运行时间。Often, companies purchase multiple circuits, as redundant paths, to improve network uptime. 发生这种情况时,离开网络前向 Internet 的流量可以通过一条链路,返回流量可以通过另一条链路。When this happens, it's possible that traffic that goes outside of the network, to the Internet, goes through one link, and the return traffic goes through a different link. 这通常称为非对称路由。This is commonly known as asymmetric routing. 在非对称路由中,反向网络流量采取与原始流程不同的路径。In asymmetric routing, reverse network traffic takes a different path from the original flow.

具有多个路径的网络

尽管这主要发生在 Internet 上,但非对称路由也适用于其他多个路径的组合。Although it primarily occurs on the Internet, asymmetric routing also applies to other combinations of multiple paths. 举例来说,它适用于前往相同目标的 Internet 路径和专用路径,以及前往相同目标的多个专用路径。It applies, for example, both to an Internet path and a private path that go to the same destination, and to multiple private paths that go to the same destination.

从源到目标的每个路由器将计算抵达目标的最佳路径。Each router along the way, from source to destination, computes the best path to reach a destination. 路由器根据两个主要因素确定可能最佳的路径:The router's determination of best possible path is based on two main factors:

  • 外部网络之间的路由基于边界网关协议 (BGP) 路由协议。Routing between external networks is based on a routing protocol, Border Gateway Protocol (BGP). BGP 接受邻居的播发并通过一连串的步骤来运行这些播发,确定抵达预定目标的最佳路径。BGP takes advertisements from neighbors and runs them through a series of steps to determine the best path to the intended destination. 它在其路由表中存储最佳路径。It stores the best path in its routing table.
  • 与路由关联的子网掩码的长度会影响路由路径。The length of a subnet mask associated with a route influences routing paths. 如果路由器收到 IP 地址相同但子网掩码不同的多个播发,路由器将首选具有较长子网掩码的播发,因为这被视为更明确的路由。If a router receives multiple advertisements for the same IP address but with different subnet masks, the router prefers the advertisement with a longer subnet mask because it's considered a more specific route.

有状态设备Stateful devices

为了路由,路由器将查看数据包的 IP 标头。Routers look at the IP header of a packet for routing purposes. 某些设备看起来在数据包的更深处。Some devices look even deeper inside the packet. 这些设备通常会查看第 4 层(传输控制协议 (TCP) 或用户数据报协议 (UDP)),甚至第 7 层(应用程序层)标头。Typically, these devices look at Layer4 (Transmission Control Protocol, or TCP; or User Datagram Protocol, or UDP), or even Layer7 (Application Layer) headers. 这几种设备要么是安全设备,要么是带宽优化设备。These kinds of devices are either security devices or bandwidth-optimization devices.

防火墙是有状态设备的常见示例。A firewall is a common example of a stateful device. 防火墙根据各种字段(例如协议、TCP/UDP 端口、URL 标头),允许或拒绝数据包通过其接口。A firewall allows or denies a packet to pass through its interfaces based on various fields such as protocol, TCP/UDP port, and URL headers. 此级别的数据包检查在设备上造成沉重的处理负载。This level of packet inspection puts a heavy processing load on the device. 为了改善性能,防火墙会检查流程的第一个数据包。To improve performance, the firewall inspects the first packet of a flow. 如果允许数据包继续传递,防火墙会将流程信息保存在其状态表中。If it allows the packet to proceed, it keeps the flow information in its state table. 根据最初的决定,允许与此流程相关的所有后续数据包。All subsequent packets related to this flow are allowed based on the initial determination. 属于现有流程的数据包可以抵达防火墙。A packet that is part of an existing flow might arrive at the firewall. 如果防火墙没有以前的状态信息,则丢弃数据包。If the firewall has no prior state information about it, the firewall drops the packet.

非对称路由与 ExpressRouteAsymmetric routing with ExpressRoute

通过 Azure ExpressRoute 连接到 Microsoft 时,网络会发生以下变化:When you connect to Microsoft through Azure ExpressRoute, your network changes like this:

  • 与 Microsoft 建立多个链接。You have multiple links to Microsoft. 一个链接是现有的 Internet 连接,另一个是通过 ExpressRoute 的连接。One link is your existing Internet connection, and the other is via ExpressRoute. 发往 Microsoft 一些流量可能通过 Internet 传输,但通过 ExpressRoute 返回,反之亦然。Some traffic to Microsoft might go through the Internet but come back via ExpressRoute, or vice versa.
  • 可以通过 ExpressRoute 接收更明确的 IP 地址。You receive more specific IP addresses via ExpressRoute. 因此,对于通过 ExpressRoute 提供的服务,路由器始终首选 ExpressRoute 将网络中的流量传输到 Microsoft。So, for traffic from your network to Microsoft for services offered via ExpressRoute, routers always prefer ExpressRoute.

为了了解这两项更改对网络的影响,让我们设想一些场景。To understand the effect these two changes have on a network, let’s consider some scenarios. 例如,只有一条线路通往 Internet,并且所有 Microsoft 服务都是通过 Internet 使用的。As an example, you have only one circuit to the Internet and you consume all Microsoft services via the Internet. 在用户网络与 Microsoft 之间来回传输的流量遍历相同的 Internet 链接并通过防火墙。The traffic from your network to Microsoft and back traverses the same Internet link and passes through the firewall. 防火墙在看到第一个数据包时记录流程,并允许返回数据包,因为此流程在于状态表中存在。The firewall records the flow as it sees the first packet and return packets are allowed because the flow exists in the state table.

非对称路由与 ExpressRoute

然后,启用 ExpressRoute,通过 ExpressRoute 使用 Microsoft 提供的服务。Then, you turn on ExpressRoute and consume services offered by Microsoft over ExpressRoute. Microsoft 提供的所有其他服务都通过 Internet 使用。All other services from Microsoft are consumed over the Internet. 在连接到 ExpressRoute 的边缘服务器上部署不同的防火墙。You deploy a separate firewall at your edge that is connected to ExpressRoute. Microsoft 通过 ExpressRoute,针对特定服务向网络播发更明确的前缀。Microsoft advertises more specific prefixes to your network over ExpressRoute for specific services. 路由基础结构选择 ExpressRoute 作为这些前缀的首选路径。Your routing infrastructure chooses ExpressRoute as the preferred path for those prefixes. 如果不是通过 ExpressRoute 向 Microsoft 播发公共 IP 地址,Microsoft 将通过 Internet 来与公共 IP 地址通信。If you are not advertising your public IP addresses to Microsoft over ExpressRoute, Microsoft communicates with your public IP addresses via the Internet. 从网络到 Microsoft 的正向流量使用 ExpressRoute,来自 Microsoft 的反向流量使用 Internet。Forward traffic from your network to Microsoft uses ExpressRoute, and reverse traffic from Microsoft uses the Internet. 当边缘服务器上的防火墙看到了在状态表中找不到的流程的响应数据包时,将丢弃返回流量。When the firewall at the edge sees a response packet for a flow that it does not find in the state table, it drops the return traffic.

如果选择为 ExpressRoute 和 Internet 播发同一网络地址转换 (NAT) 池,会发现网络中专用 IP 地址上的客户端有类似的问题。If you choose to advertise the same network address translation (NAT) pool for ExpressRoute and for the Internet, you'll see similar issues with the clients in your network on private IP addresses. Windows Update 等服务的请求通过 Internet 传递,因为这些服务的 IP 地址不通过 ExpressRoute 播发。Requests for services like Windows Update go via the Internet because IP addresses for these services are not advertised via ExpressRoute. 但是,返回流量通过 ExpressRoute 返回。However, the return traffic comes back via ExpressRoute. 如果 Microsoft 从 Internet 和 ExpressRoute 收到具有相同子网掩码的 IP 地址,则首选基于 Internet 的 ExpressRoute。If Microsoft receives an IP address with the same subnet mask from the Internet and ExpressRoute, it prefers ExpressRoute over the Internet. 如果在网络边缘上面向 ExpressRoute 的防火墙或其他有状态设备没有任何有关流程的先前信息,将丢弃属于该流程的数据包。If a firewall or another stateful device that is on your network edge and facing ExpressRoute has no prior information about the flow, it drops the packets that belong to that flow.

非对称路由解决方案Asymmetric routing solutions

有两个主要选项可以解决非对称路由问题。You have two main options to solve the problem of asymmetric routing. 一个是通过路由,另一个是使用基于源的 NAT (SNAT)。One is through routing, and the other is by using source-based NAT (SNAT).

路由Routing

确保已向相应的广域网 (WAN) 链接播发公共 IP 地址。Ensure that your public IP addresses are advertised to appropriate wide area network (WAN) links. 例如,如果要将 Internet 用于身份验证流量,将 ExpressRoute 用于邮件流量,则不应通过 ExpressRoute 播发 Active Directory 联合身份验证服务 (AD FS) 公共 IP 地址。For example, if you want to use the Internet for authentication traffic and ExpressRoute for your mail traffic, you should not advertise your Active Directory Federation Services (AD FS) public IP addresses over ExpressRoute. 同样地,请确保不要对路由器通过 ExpressRoute 接收的 IP 地址公开本地 AD FS 服务器。Similarly, be sure not to expose an on-premises AD FS server to IP addresses that the router receives over ExpressRoute. 通过 ExpressRoute 收到的路由更加明确,因此可让 ExpressRoute 成为 Microsoft 身份验证流量的首选路径。Routes received over ExpressRoute are more specific so they make ExpressRoute the preferred path for authentication traffic to Microsoft. 这会导致非对称路由。This causes asymmetric routing.

如果想要使用 ExpressRoute 进行身份验证,请确保通过 ExpressRoute(不使用 NAT)播发 ADFS 公共 IP 地址。If you want to use ExpressRoute for authentication, make sure that you are advertising AD FS public IP addresses over ExpressRoute without NAT. 这样,源自 Microsoft 前往本地 AD FS 服务器的流量将经过 ExpressRoute。This way, traffic that originates from Microsoft and goes to an on-premises AD FS server goes over ExpressRoute. 从客户到 Microsoft 的返回流量使用 ExpressRoute,因为这是基于 Internet 的首选路由。Return traffic from customer to Microsoft uses ExpressRoute because it's the preferred route over the Internet.

基于源的 NATSource-based NAT

解决非对称路由问题的另一种方法是使用 SNAT。Another way of solving asymmetric routing issues is by using SNAT. 例如,某个用户尚未通过 ExpressRoute 播发本地简单邮件传输协议 (SMTP) 服务器的公共 IP 地址,因为该用户倾向于使用 Internet 进行此类通信。For example, you have not advertised the public IP address of an on-premises Simple Mail Transfer Protocol (SMTP) server over ExpressRoute because you intend to use the Internet for this type of communication. 源自 Microsoft 前往本地 SMTP 服务器的请求将遍历 Internet。A request that originates with Microsoft and then goes to your on-premises SMTP server traverses the Internet. 通过 SNAT 将传入请求传递到内部 IP 地址。You SNAT the incoming request to an internal IP address. 来自 SMTP 服务器的反向流量将抵达边缘防火墙(用于 NAT),而不通过 ExpressRoute。Reverse traffic from the SMTP server goes to the edge firewall (which you use for NAT) instead of through ExpressRoute. 返回流量将通过 Internet 返回。The return traffic goes back via the Internet.

基于源的 NAT 网络配置

非对称路由检测Asymmetric routing detection

跟踪路由是确保网络流量遍历预期路径的最佳方式。Traceroute is the best way to make sure that your network traffic is traversing the expected path. 如果预期从本地 SMTP 服务器到 Microsoft 的流量采用 Internet 路径,则预期跟踪路由是从 SMTP 服务器到 Office 365。If you expect traffic from your on-premises SMTP server to Microsoft to take the Internet path, the expected traceroute is from the SMTP server to Office 365. 结果证明,流量确实离开了网络前往 Internet,而不是前往 ExpressRoute。The result validates that traffic is indeed leaving your network toward the Internet and not toward ExpressRoute.