使用用户定义的路由自定义群集出口Customize cluster egress with a User-Defined Route

可以根据具体的方案自定义 AKS 群集的出口。Egress from an AKS cluster can be customized to fit specific scenarios. 默认情况下,AKS 将预配一个可设置并用于出口的标准 SKU 负载均衡器。By default, AKS will provision a Standard SKU Load Balancer to be set up and used for egress. 但是,如果禁用了公共 IP 或者出口需要额外的跃点,则默认设置可能不能满足所有方案的要求。However, the default setup may not meet the requirements of all scenarios if public IPs are disallowed or additional hops are required for egress.

本文介绍了如何自定义群集的出口路由以支持自定义网络方案,例如,禁用公共 IP 并要求群集位于网络虚拟设备 (NVA) 后面。This article walks through how to customize a cluster's egress route to support custom network scenarios, such as those which disallows public IPs and requires the cluster to sit behind a network virtual appliance (NVA).

先决条件Prerequisites

  • Azure CLI 2.0.81 或更高版本Azure CLI version 2.0.81 or greater
  • API 2020-01-01 或更高版本API version of 2020-01-01 or greater

限制Limitations

  • OutboundType 只能在创建群集时定义,以后无法对其进行更新。OutboundType can only be defined at cluster create time and can't be updated afterwards.
  • 设置 outboundType 需要将 AKS 群集的 vm-set-type 指定为 VirtualMachineScaleSets,将 load-balancer-sku 指定为 StandardSetting outboundType requires AKS clusters with a vm-set-type of VirtualMachineScaleSets and load-balancer-sku of Standard.
  • outboundType 设置为 UDR 值需要用户定义的路由以及群集的有效出站连接。Setting outboundType to a value of UDR requires a user-defined route with valid outbound connectivity for the cluster.
  • 如果将 outboundType 设置为 UDR 的值,则意味着路由到负载均衡器的流入量源 IP 可能与群集的流出量目标地址不匹配。Setting outboundType to a value of UDR implies the ingress source IP routed to the load-balancer may not match the cluster's outgoing egress destination address.

AKS 中的出站类型概述Overview of outbound types in AKS

可以使用 loadBalanceruserDefinedRouting 类型的唯一 outboundType 来自定义 AKS 群集。An AKS cluster can be customized with a unique outboundType of type loadBalancer or userDefinedRouting.

重要

出站类型仅影响群集的出口流量。Outbound type impacts only the egress traffic of your cluster. 有关详细信息,请参阅设置入口控制器For more information, see setting up ingress controllers.

备注

可以将自己的路由表与 UDR 和 kubenet 网络一起使用。You can use your own route table with UDR and kubenet networking. 确保群集标识(服务主体或托管标识)具有对自定义路由表的“参与者”权限。Make sure you cluster identity (service principal or managed identity) has Contributor permissions to the custom route table.

loadBalancer 的出站类型Outbound type of loadBalancer

如果设置了 loadBalancer,AKS 将自动完成以下配置。If loadBalancer is set, AKS completes the following configuration automatically. 对于经过 AKS 分配的公共 IP 的传出流量,将使用负载均衡器。The load balancer is used for egress through an AKS assigned public IP. 出站类型 loadBalancer 支持 loadBalancer 类型的 Kubernetes 服务,此类型的服务应收到由 AKS 资源提供程序创建的负载均衡器传出的流量。An outbound type of loadBalancer supports Kubernetes services of type loadBalancer, which expect egress out of the load balancer created by the AKS resource provider.

以下配置由 AKS 完成。The following configuration is done by AKS.

  • 为群集出口预配一个公共 IP 地址。A public IP address is provisioned for cluster egress.
  • 将该公共 IP 地址分配给负载均衡器资源。The public IP address is assigned to the load balancer resource.
  • 为群集中的代理节点设置负载均衡器的后端池。Backend pools for the load balancer are set up for agent nodes in the cluster.

下面是默认情况下在 AKS 群集中部署的网络拓扑,该拓扑使用类型为 loadBalanceroutboundTypeBelow is a network topology deployed in AKS clusters by default, which use an outboundType of loadBalancer.

此图显示了入口 IP 和出口 IP,其中的入口 IP 将流量定向到负载均衡器,负载均衡器将往返于内部群集的流量和其他流量定向到出口 IP,出口 IP 将流量定向到 Internet、MCR、Azure 所需服务和 AKS 控制平面。

userDefinedRouting 的出站类型Outbound type of userDefinedRouting

备注

使用出站类型是一种高级网络方案,需要正确配置网络。Using outbound type is an advanced networking scenario and requires proper network configuration.

如果设置了 userDefinedRouting,则 AKS 不会自动配置出口路径。If userDefinedRouting is set, AKS won't automatically configure egress paths. 必须由你完成出口设置。The egress setup must be done by you.

AKS 群集必须部署到具有之前已配置的子网的现有虚拟网络中,因为在未使用标准负载均衡器 (SLB) 体系结构时,必须建立显式出口。The AKS cluster must be deployed into an existing virtual network with a subnet that has been previously configured because when not using standard load balancer (SLB) architecture, you must establish explicit egress. 因此,此体系结构需要向防火墙、网关、代理等设备显式发送出口流量,或需要允许网络地址转换 (NAT) 由分配给标准负载均衡器或设备的公共 IP 完成。As such, this architecture requires explicitly sending egress traffic to an appliance like a firewall, gateway, proxy or to allow the Network Address Translation (NAT) to be done by a public IP assigned to the standard load balancer or appliance.

使用 userDefinedRouting 创建负载均衡器Load balancer creation with userDefinedRouting

只有在部署了“loadBalancer”类型的第一个 Kubernetes 服务时,出站类型为 UDR 的 AKS 群集才会接收标准负载均衡器 (SLB)。AKS clusters with an outbound type of UDR receive a standard load balancer (SLB) only when the first Kubernetes service of type 'loadBalancer' is deployed. 负载均衡器配置了用于入站请求的公共 IP 地址以及用于入站请求的后端池 。The load balancer is configured with a public IP address for inbound requests and a backend pool for inbound requests. 入站规则由 Azure 云提供商配置,但由于采用 UDR 出站类型,因此未配置出站公共 IP 地址或出站规则。Inbound rules are configured by the Azure cloud provider, but no outbound public IP address or outbound rules are configured as a result of having an outbound type of UDR. 你的 UDR 仍将是出口流量的唯一来源。Your UDR will still be the only source for egress traffic.

Azure 负载均衡器在设置规则之后才会产生费用Azure load balancers don't incur a charge until a rule is placed.

部署出站类型为 UDR 且具有 Azure 防火墙的群集Deploy a cluster with outbound type of UDR and Azure Firewall

若要演示如何应用出站类型为用户定义的路由的群集,可以通过其自己子网上的 Azure 防火墙的虚拟网络中配置一个群集。To illustrate the application of a cluster with outbound type using a user-defined route, a cluster can be configured on a virtual network with an Azure Firewall on its own subnet. 请参阅使用防火墙限制出口流量示例上的相应示例。See this example on the restrict egress traffic with Azure firewall example.

重要

UDR 的出站类型要求路由表中有 0.0.0.0/0 的路由和 NVA(网络虚拟设备)的下一个跃点目标。Outbound type of UDR requires there is a route for 0.0.0.0/0 and next hop destination of NVA (Network Virtual Appliance) in the route table. 路由表已具有默认的 0.0.0.0/0 到 Internet 的路由,但没有连接 SNAT 的公共 IP,光是添加此路由不会为你提供出口。The route table already has a default 0.0.0.0/0 to Internet, without a Public IP to SNAT just adding this route will not provide you egress. AKS 将验证你没有创建指向 Internet 的 0.0.0.0/0 路由,而是创建了指向 NVA 或网关等的路由。使用 UDR 出站类型时,不会创建用于入站请求的负载均衡器公共 IP 地址,除非配置了 loadbalancer 类型的服务。AKS will validate that you don't create a 0.0.0.0/0 route pointing to the Internet but instead to NVA or gateway, etc. When using an outbound type of UDR, a load balancer public IP address for inbound requests is not created unless a service of type loadbalancer is configured. 如果设置了 UDR 出站类型,AKS 永远不会创建用于出站请求的公共 IP 地址。A public IP address for outbound requests is never created by AKS if an outbound type of UDR is set.

后续步骤Next steps

参阅 Azure 网络 UDR 概述See Azure networking UDR overview.

参阅如何创建、更改或删除路由表See how to create, change, or delete a route table.