在 Azure Kubernetes 服务 (AKS) 中结合自己的 IP 地址范围使用 kubenet 网络Use kubenet networking with your own IP address ranges in Azure Kubernetes Service (AKS)

默认情况下,AKS 群集使用 kubenet,系统会为你创建 Azure 虚拟网络和子网。By default, AKS clusters use kubenet, and an Azure virtual network and subnet are created for you. 节点使用 kubenet 从 Azure 虚拟网络子网获取 IP 地址。With kubenet , nodes get an IP address from the Azure virtual network subnet. Pod 接收从逻辑上不同的地址空间到节点的 Azure 虚拟网络子网的 IP 地址。Pods receive an IP address from a logically different address space to the Azure virtual network subnet of the nodes. 然后配置网络地址转换 (NAT),以便 Pod 可以访问 Azure 虚拟网络上的资源。Network address translation (NAT) is then configured so that the pods can reach resources on the Azure virtual network. 流量的源 IP 地址通过 NAT 转换为节点的主 IP 地址。The source IP address of the traffic is NAT'd to the node's primary IP address. 这种方法大大减少了需要在网络空间中保留供 Pod 使用的 IP 地址数量。This approach greatly reduces the number of IP addresses that you need to reserve in your network space for pods to use.

借助 Azure 容器网络接口 (CNI),每个 Pod 都可以从子网获得 IP 地址,并且可供直接访问。With Azure Container Networking Interface (CNI), every pod gets an IP address from the subnet and can be accessed directly. 这些 IP 地址在网络空间中必须唯一,并且必须事先计划。These IP addresses must be unique across your network space, and must be planned in advance. 每个节点都有一个配置参数来表示它支持的最大 Pod 数。Each node has a configuration parameter for the maximum number of pods that it supports. 这样,就会为每个节点预留相应的 IP 地址数。The equivalent number of IP addresses per node are then reserved up front for that node. 使用此方法需要经过更详细的规划,并且经常会耗尽 IP 地址,或者在应用程序需求增长时需要在更大的子网中重建群集。This approach requires more planning, and often leads to IP address exhaustion or the need to rebuild clusters in a larger subnet as your application demands grow. 可以在创建群集时或新建节点池时,配置可部署到节点的最大 Pod 数。You can configure the maximum pods deployable to a node at cluster create time or when creating new node pools. 如果在创建新节点池时未指定 maxPod,则会收到 kubenet 的默认值 110。If you don't specify maxPods when creating new node pools, you receive a default value of 110 for kubenet.

本文介绍如何使用 kubenet 网络来创建和使用 AKS 群集的虚拟网络子网。This article shows you how to use kubenet networking to create and use a virtual network subnet for an AKS cluster. 有关网络选项和注意事项的详细信息,请参阅 Kubernetes 和 AKS 的网络概念For more information on network options and considerations, see Network concepts for Kubernetes and AKS.

先决条件Prerequisites

  • AKS 群集的虚拟网络必须允许出站 Internet 连接。The virtual network for the AKS cluster must allow outbound internet connectivity.
  • 不要在同一子网中创建多个 AKS 群集。Don't create more than one AKS cluster in the same subnet.
  • AKS 群集不得将 169.254.0.0/16172.30.0.0/16172.31.0.0/16192.0.2.0/24 用于 Kubernetes 服务地址范围、Pod 地址范围或群集虚拟网络地址范围。AKS clusters may not use 169.254.0.0/16, 172.30.0.0/16, 172.31.0.0/16, or 192.0.2.0/24 for the Kubernetes service address range, pod address range or cluster virtual network address range.
  • AKS 群集使用的服务主体在虚拟网络中的子网上必须至少具有网络参与者角色。The service principal used by the AKS cluster must have at least Network Contributor role on the subnet within your virtual network. 你还必须具有相应的权限(如订阅所有者),才能创建服务主体并向其分配权限。You must also have the appropriate permissions, such as the subscription owner, to create a service principal and assign it permissions. 如果希望定义自定义角色而不是使用内置的网络参与者角色,则需要以下权限:If you wish to define a custom role instead of using the built-in Network Contributor role, the following permissions are required:
    • Microsoft.Network/virtualNetworks/subnets/join/action
    • Microsoft.Network/virtualNetworks/subnets/read

警告

若要使用 Windows Server 节点池,必须使用 Azure CNI。To use Windows Server node pools, you must use Azure CNI. 对于 Windows Server 容器,无法将 kubenet 用作网络模型。The use of kubenet as the network model is not available for Windows Server containers.

准备阶段Before you begin

需要安装并配置 Azure CLI 2.0.65 或更高版本。You need the Azure CLI version 2.0.65 or later installed and configured. 运行 az --version 即可查找版本。Run az --version to find the version. 如果需要进行安装或升级,请参阅安装 Azure CLIIf you need to install or upgrade, see Install Azure CLI.

使用自有子网的 kubenet 网络概述Overview of kubenet networking with your own subnet

在许多环境中,你已定义了具有分配的 IP 地址范围的虚拟网络和子网。In many environments, you have defined virtual networks and subnets with allocated IP address ranges. 这些虚拟网络资源用于支持多个服务和应用程序。These virtual network resources are used to support multiple services and applications. 若要提供网络连接,AKS 群集可以使用 kubenet (基本网络)或 Azure CNI(高级网络)。To provide network connectivity, AKS clusters can use kubenet (basic networking) or Azure CNI ( advanced networking ).

使用 kubenet 时,只有节点接收虚拟网络子网中的 IP 地址。With kubenet , only the nodes receive an IP address in the virtual network subnet. Pod 无法直接相互通信。Pods can't communicate directly with each other. 用户定义的路由 (UDR) 和 IP 转发用于不同节点中 Pod 之间的连接。Instead, User Defined Routing (UDR) and IP forwarding is used for connectivity between pods across nodes. 默认情况下,UDR 和 IP 转发配置由 AKS 服务进行创建和维护,但你可以选择自带路由表以进行自定义路由管理By default, UDRs and IP forwarding configuration is created and maintained by the AKS service, but you have to the option to bring your own route table for custom route management. 此外,可以在接收分配的 IP 地址的服务后面部署 Pod,并对应用程序的流量进行负载均衡。You could also deploy pods behind a service that receives an assigned IP address and load balances traffic for the application. 下图显示了 AKS 节点(不是 Pod)如何接收虚拟网络子网中的 IP 地址:The following diagram shows how the AKS nodes receive an IP address in the virtual network subnet, but not the pods:

使用 AKS 群集的 Kubenet 网络模型

Azure 在一个 UDR 中最多支持 400 个路由,因此,AKS 群集中的节点数不能超过 400 个。Azure supports a maximum of 400 routes in a UDR, so you can't have an AKS cluster larger than 400 nodes. kubenet 不支持 Azure 网络策略。Azure Network Policies aren't supported with kubenet . 可以使用 Calico 网络策略,因为 kubenet 支持这些策略。You can use Calico Network Policies, as they are supported with kubenet.

使用 Azure CNI 时,每个 Pod 将接收 IP 子网中的 IP 地址,并且可以直接与其他 Pod 和服务通信。With Azure CNI , each pod receives an IP address in the IP subnet, and can directly communicate with other pods and services. 群集的最大大小可为指定的 IP 地址范围上限。Your clusters can be as large as the IP address range you specify. 但是,必须提前规划 IP 地址范围,AKS 节点根据它们支持的最大 Pod 数消耗所有 IP 地址。However, the IP address range must be planned in advance, and all of the IP addresses are consumed by the AKS nodes based on the maximum number of pods that they can support. Azure CNI 支持网络策略(Azure 或 Calico)。Network Policies (either Azure or Calico) are supported with Azure CNI .

Kubenet 的限制和注意事项Limitations & considerations for kubenet

  • 在 Kubenet 的设计中需要额外的跃点,这会导致 Pod 通信出现轻微延迟。An additional hop is required in the design of kubenet, which adds minor latency to pod communication.

  • 需要路由表和用户定义的路由才能使用 Kubenet,这会增加操作的复杂性。Route tables and user-defined routes are required for using kubenet, which adds complexity to operations.

  • 由于 Kubenet 设计,Kubenet 不支持直接 Pod 寻址。Direct pod addressing isn't supported for kubenet due to kubenet design.

  • 与 Azure CNI 群集不同,多个 Kubenet 群集无法共享一个子网。Unlike Azure CNI clusters, multiple kubenet clusters can't share a subnet.

  • Kubenet 不支持的功能包括:Features not supported on kubenet include:

IP 地址可用性和耗尽IP address availability and exhaustion

使用 Azure CNI 时,一个常见问题是分配的 IP 地址范围太小,以致在扩展或升级群集时需要添加更多的节点。With Azure CNI , a common issue is the assigned IP address range is too small to then add additional nodes when you scale or upgrade a cluster. 网络团队可能无法提供足够大的 IP 地址范围来支持预期的应用程序需求。The network team may also not be able to issue a large enough IP address range to support your expected application demands.

作为一种折衷方案,可以创建使用 kubenet 的 AKS 群集并连接到现有虚拟网络子网。As a compromise, you can create an AKS cluster that uses kubenet and connect to an existing virtual network subnet. 这种方法可让节点接收定义的 IP 地址,而无需提前为群集中可能运行的所有潜在 Pod 节点预留大量的 IP 地址。This approach lets the nodes receive defined IP addresses, without the need to reserve a large number of IP addresses up front for all of the potential pods that could run in the cluster.

使用 kubenet 时,可以大幅减小要使用的 IP 地址范围,并且可以支持大型群集和应用程序的需求。With kubenet , you can use a much smaller IP address range and be able to support large clusters and application demands. 例如,即使在子网上使用 /27 IP 地址范围,也可运行包括 20-25 个节点的群集,空间足以进行缩放或升级。For example, even with a /27 IP address range on your subnet, you could run a 20-25 node cluster with enough room to scale or upgrade. 此群集大小最多支持 2,200-2,750 个 Pod(每个节点的最大 Pod 数默认为 110 个)。This cluster size would support up to 2,200-2,750 pods (with a default maximum of 110 pods per node). 可以在 AKS 中使用 kubenet 配置的每个节点的最大 Pod 数为 110。The maximum number of pods per node that you can configure with kubenet in AKS is 110.

以下基本计算方法对网络模型的差异做了比较:The following basic calculations compare the difference in network models:

  • kubenet -一个简单的 /24 IP 地址范围最多可以支持群集中的 251 个节点(每个 Azure 虚拟网络子网预留前三个 IP 地址用于管理操作)kubenet - a simple /24 IP address range can support up to 251 nodes in the cluster (each Azure virtual network subnet reserves the first three IP addresses for management operations)
    • 此节点计数最多可以支持 27,610 个 Pod(使用 kubenet 的每个节点的最大 Pod 数默认为 110 个)This node count could support up to 27,610 pods (with a default maximum of 110 pods per node with kubenet )
  • Azure CNI - 相同的基本 /24 子网范围最多只能支持群集中的 8 个节点Azure CNI - that same basic /24 subnet range could only support a maximum of 8 nodes in the cluster
    • 此节点计数最多只能支持 240 个 Pod(使用 Azure CNI 的每个节点的最大 Pod 数默认为 30 个)This node count could only support up to 240 pods (with a default maximum of 30 pods per node with Azure CNI )

备注

这些最大值未考虑到帐户升级或扩展操作。These maximums don't take into account upgrade or scale operations. 在实践中,不可能会运行子网 IP 地址范围支持的节点数上限。In practice, you can't run the maximum number of nodes that the subnet IP address range supports. 必须留出一些 IP 地址,供扩展或升级操作期间使用。You must leave some IP addresses available for use during scale of upgrade operations.

虚拟网络对等互连和 ExpressRoute 连接Virtual network peering and ExpressRoute connections

若要提供本地连接, kubenetAzure CNI 网络方法都可以使用 Azure 虚拟网络对等互连ExpressRoute 连接To provide on-premises connectivity, both kubenet and Azure-CNI network approaches can use Azure virtual network peering or ExpressRoute connections. 精心规划 IP 地址范围,以防止地址重叠和流量路由错误。Plan your IP address ranges carefully to prevent overlap and incorrect traffic routing. 例如,许多本地网络使用通过 ExpressRoute 连接播发的 10.0.0.0/8 地址范围。For example, many on-premises networks use a 10.0.0.0/8 address range that is advertised over the ExpressRoute connection. 建议在此地址范围(例如 172.16.0.0/16 )外部的 Azure 虚拟网络子网中创建 AKS 群集。It's recommended to create your AKS clusters into Azure virtual network subnets outside of this address range, such as 172.16.0.0/16 .

选择要使用的网络模型Choose a network model to use

选择用于 AKS 群集的网络插件通常需要在灵活性与高级配置需求之间进行平衡。The choice of which network plugin to use for your AKS cluster is usually a balance between flexibility and advanced configuration needs. 如果每种网络模型似乎都很合适,以下考虑因素可帮助你做出决策:The following considerations help outline when each network model may be the most appropriate.

对于以下情况,可使用 kubenetUse kubenet when:

  • IP 地址空间有限。You have limited IP address space.
  • 大部分 Pod 通信在群集中进行。Most of the pod communication is within the cluster.
  • 不需要虚拟节点或 Azure 网络策略等高级 AKS 功能。You don't need advanced AKS features such as virtual nodes or Azure Network Policy. 使用 Calico 网络策略Use Calico network policies.

对于以下情况,可使用 Azure CNIUse Azure CNI when:

  • 有可用的 IP 地址空间。You have available IP address space.
  • 大部分 Pod 通信是与群集外部的资源进行的。Most of the pod communication is to resources outside of the cluster.
  • 不想管理用户定义的 Pod 连接路由。You don't want to manage user defined routes for pod connectivity.
  • 需要虚拟节点或 Azure 网络策略等 AKS 高级功能。You need AKS advanced features such as virtual nodes or Azure Network Policy. 使用 Calico 网络策略Use Calico network policies.

有关帮助你决定使用哪个网络模型的详细信息,请参阅比较网络模型及其支持范围For more information to help you decide which network model to use, see Compare network models and their support scope.

创建虚拟网络和子网Create a virtual network and subnet

若要开始使用 kubenet 和自己的虚拟网络子网,请先使用 az group create 命令创建一个资源组。To get started with using kubenet and your own virtual network subnet, first create a resource group using the az group create command. 以下示例在“chinaeast2”位置创建名为“myResourceGroup”的资源组:The following example creates a resource group named myResourceGroup in the chinaeast2 location:

az group create --name myResourceGroup --location chinaeast2

如果没有可用的现有虚拟网络和子网,请使用 az network vnet create 命令创建这些网络资源。If you don't have an existing virtual network and subnet to use, create these network resources using the az network vnet create command. 在以下示例中,虚拟网络名为 myVnet ,其地址前缀为 192.168.0.0/16In the following example, the virtual network is named myVnet with the address prefix of 192.168.0.0/16 . 创建了名为 myAKSSubnet 、地址前缀为 192.168.1.0/24 的子网。A subnet is created named myAKSSubnet with the address prefix 192.168.1.0/24 .

az network vnet create \
    --resource-group myResourceGroup \
    --name myAKSVnet \
    --address-prefixes 192.168.0.0/16 \
    --subnet-name myAKSSubnet \
    --subnet-prefix 192.168.1.0/24

创建服务主体并分配权限Create a service principal and assign permissions

若要允许 AKS 群集与其他 Azure 资源交互,请使用 Azure Active Directory 服务主体。To allow an AKS cluster to interact with other Azure resources, an Azure Active Directory service principal is used. 服务主体需要有权管理 AKS 节点使用的虚拟网络和子网。The service principal needs to have permissions to manage the virtual network and subnet that the AKS nodes use. 若要创建服务主体,请使用 az ad sp create-for-rbac 命令:To create a service principal, use the az ad sp create-for-rbac command:

az ad sp create-for-rbac --skip-assignment

以下示例输出显示了服务主体的应用程序 ID 和密码。The following example output shows the application ID and password for your service principal. 其他步骤中使用了这些值向服务主体分配角色,然后创建 AKS 群集:These values are used in additional steps to assign a role to the service principal and then create the AKS cluster:

az ad sp create-for-rbac --skip-assignment
{
  "appId": "476b3636-5eda-4c0e-9751-849e70b5cfad",
  "displayName": "azure-cli-2019-01-09-22-29-24",
  "name": "http://azure-cli-2019-01-09-22-29-24",
  "password": "a1024cd7-af7b-469f-8fd7-b293ecbb174e",
  "tenant": "72f998bf-85f1-41cf-92ab-2e7cd014db46"
}

若要在剩余步骤中分配正确的委托,请使用 az network vnet showaz network vnet subnet show 命令获取所需的资源 ID。To assign the correct delegations in the remaining steps, use the az network vnet show and az network vnet subnet show commands to get the required resource IDs. 这些资源 ID 存储为变量,并在剩余的步骤中引用:These resource IDs are stored as variables and referenced in the remaining steps:

VNET_ID=$(az network vnet show --resource-group myResourceGroup --name myAKSVnet --query id -o tsv)
SUBNET_ID=$(az network vnet subnet show --resource-group myResourceGroup --vnet-name myAKSVnet --name myAKSSubnet --query id -o tsv)

现在,使用 az role assignment create 命令为 AKS 群集的服务主体分配虚拟网络中的“网络参与者”权限。Now assign the service principal for your AKS cluster Network Contributor permissions on the virtual network using the az role assignment create command. 根据上一命令的输出中所示,提供自己的 <appId> 来创建服务主体:Provide your own <appId> as shown in the output from the previous command to create the service principal:

az role assignment create --assignee <appId> --scope $VNET_ID --role "Network Contributor"

在虚拟网络中创建 AKS 群集Create an AKS cluster in the virtual network

现已创建虚拟网络和子网、已创建服务主体并为其分配了这些网络资源的使用权限。You've now created a virtual network and subnet, and created and assigned permissions for a service principal to use those network resources. 现在,请使用 az aks create 命令在虚拟网络和子网中创建 AKS 群集。Now create an AKS cluster in your virtual network and subnet using the az aks create command. 根据上一命令的输出中所示,定义自己的服务主体 <appId> 和 <password> 来创建服务主体 。Define your own service principal <appId> and <password> , as shown in the output from the previous command to create the service principal.

在创建群集的过程中还定义了以下 IP 地址范围:The following IP address ranges are also defined as part of the cluster create process:

  • --service-cidr 用于为 AKS 群集中的内部服务分配 IP 地址。The --service-cidr is used to assign internal services in the AKS cluster an IP address. 此 IP 地址范围应为未在网络环境中的其他位置使用的地址空间,包括任何本地网络范围(如果你使用 Express Route 或站点到站点 VPN 连接来连接或计划连接到 Azure 虚拟网络)。This IP address range should be an address space that isn't in use elsewhere in your network environment, including any on-premises network ranges if you connect, or plan to connect, your Azure virtual networks using Express Route or a Site-to-Site VPN connection.

  • --dns-service-ip 地址应该是服务 IP 地址范围的 .10 地址。The --dns-service-ip address should be the .10 address of your service IP address range.

  • --pod-cidr 应该是未在网络环境中的其他位置使用的较大地址空间。The --pod-cidr should be a large address space that isn't in use elsewhere in your network environment. 如果你需要或者打算使用 Express Route 或站点到站点 VPN 连接来连接 Azure 虚拟网络,则此范围可包括任何本地网络范围。This range includes any on-premises network ranges if you connect, or plan to connect, your Azure virtual networks using Express Route or a Site-to-Site VPN connection.

    • 此地址范围必须足够大,可以容纳预期要扩展到的节点数。This address range must be large enough to accommodate the number of nodes that you expect to scale up to. 部署群集后,如果需要为更多的节点提供更多的地址,你无法更改此地址范围。You can't change this address range once the cluster is deployed if you need more addresses for additional nodes.
    • Pod IP 地址范围用于将 /24 地址空间分配到群集中的每个节点。The pod IP address range is used to assign a /24 address space to each node in the cluster. 在以下示例中, --pod cidr 10.244.0.0/16 为第一个节点分配 10.244.0.0/24 ,为第二个节点分配 10.244.1.0/24 ,为第三节点分配 10.244.2.0/24In the following example, the --pod-cidr of 10.244.0.0/16 assigns the first node 10.244.0.0/24 , the second node 10.244.1.0/24 , and the third node 10.244.2.0/24 .
    • 群集扩展或升级时,Azure 平台会继续向每个新节点分配 Pod IP 地址范围。As the cluster scales or upgrades, the Azure platform continues to assign a pod IP address range to each new node.
  • --docker-bridge-address 允许 AKS 节点与基础管理平台进行通信。The --docker-bridge-address lets the AKS nodes communicate with the underlying management platform. 此 IP 地址不能在群集的虚拟网络 IP 地址范围内,并且不应当与网络上使用的其他地址范围重叠。This IP address must not be within the virtual network IP address range of your cluster, and shouldn't overlap with other address ranges in use on your network.

az aks create \
    --resource-group myResourceGroup \
    --name myAKSCluster \
    --node-count 3 \
    --network-plugin kubenet \
    --service-cidr 10.0.0.0/16 \
    --dns-service-ip 10.0.0.10 \
    --pod-cidr 10.244.0.0/16 \
    --docker-bridge-address 172.17.0.1/16 \
    --vnet-subnet-id $SUBNET_ID \
    --service-principal <appId> \
    --client-secret <password>

备注

如果希望启用 AKS 群集以包括 Calico 网络策略,可以使用以下命令。If you wish to enable an AKS cluster to include a Calico network policy you can use the following command.

az aks create \
    --resource-group myResourceGroup \
    --name myAKSCluster \
    --node-count 3 \
    --network-plugin kubenet --network-policy calico \
    --service-cidr 10.0.0.0/16 \
    --dns-service-ip 10.0.0.10 \
    --pod-cidr 10.244.0.0/16 \
    --docker-bridge-address 172.17.0.1/16 \
    --vnet-subnet-id $SUBNET_ID \
    --service-principal <appId> \
    --client-secret <password>

创建 AKS 群集时,将自动创建网络安全组和路由表。When you create an AKS cluster, a network security group and route table are automatically created. 这些网络资源可以通过 AKS 控制平面进行管理。These network resources are managed by the AKS control plane. 网络安全组自动与节点上的虚拟 NIC 相关联。The network security group is automatically associated with the virtual NICs on your nodes. 路由表自动与虚拟网络子网相关联。The route table is automatically associated with the virtual network subnet. 在你创建和公开服务时,系统会自动更新网络安全组规则和路由表。Network security group rules and route tables are automatically updated as you create and expose services.

在 kubenet 中自带子网和路由表Bring your own subnet and route table with kubenet

使用 kubenet 时,群集子网上必须存在路由表。With kubenet, a route table must exist on your cluster subnet(s). AKS 支持自带现有的子网和路由表。AKS supports bringing your own existing subnet and route table.

如果自定义子网不包含路由表,AKS 会为你创建一个路由表,并在整个群集生命周期中向其添加规则。If your custom subnet does not contain a route table, AKS creates one for you and adds rules to it throughout the cluster lifecycle. 如果在创建群集时自定义子网包含路由表,AKS 将在群集操作期间确认现有路由表,并相应地为云提供程序操作添加/更新规则。If your custom subnet contains a route table when you create your cluster, AKS acknowledges the existing route table during cluster operations and adds/updates rules accordingly for cloud provider operations.

警告

可以将自定义规则添加到自定义路由表中并进行更新。Custom rules can be added to the custom route table and updated. 但是,规则由 Kubernetes 云提供商添加,不能更新或删除。However, rules are added by the Kubernetes cloud provider which must not be updated or removed. 诸如 0.0.0.0/0 的规则必须始终存在于给定的路由表中,并映射到 internet 网关的目标,例如 NVA 或其他出口网关。Rules such as 0.0.0.0/0 must always exist on a given route table and map to the target of your internet gateway, such as an NVA or other egress gateway. 在更新规则时,请注意只修改自定义规则。Take caution when updating rules that only your custom rules are being modified.

了解有关设置自定义路由表的详细信息。Learn more about setting up a custom route table.

Kubenet 网络需要使用经过规划和组织的路由表规则才能成功路由请求。Kubenet networking requires organized route table rules to successfully route requests. 由于此原因,需要为依赖路由表的每个群集精心维护路由表。Due to this design, route tables must be carefully maintained for each cluster which relies on it. 多个群集无法共享一个路由表,因为不同群集的 Pod CIDR 可能会相互重叠,从而导致意外路由和路由中断。Multiple clusters cannot share a route table because pod CIDRs from different clusters may overlap which causes unexpected and broken routing. 在同一虚拟网络上配置多个群集或为每个群集设置专用虚拟网络时,请确保考虑以下限制。When configuring multiple clusters on the same virtual network or dedicating a virtual network to each cluster, ensure the following limitations are considered.

的限制:Limitations:

  • 必须在创建群集之前分配权限,请确保使用的服务主体具有对自定义子网和自定义路由表的写入权限。Permissions must be assigned before cluster creation, ensure you are using a service principal with write permissions to your custom subnet and custom route table.
  • kubenet 中的自定义路由表当前不支持托管标识。Managed identities are not currently supported with custom route tables in kubenet.
  • 在创建 AKS 群集之前,需要将自定义路由表与子网关联。A custom route table must be associated to the subnet before you create the AKS cluster.
  • 创建群集后,无法更新关联的路由表资源。The associated route table resource cannot be updated after cluster creation. 虽然无法更新路由表资源,但可以在路由表上修改自定义规则。While the route table resource cannot be updated, custom rules can be modified on the route table.
  • 每个 AKS 群集必须为与群集关联的所有子网使用同一个唯一的路由表。Each AKS cluster must use a single, unique route table for all subnets associated with the cluster. 由于可能存在重叠的 Pod CIDR 和发生路由规则冲突,无法对多个群集重复使用同一个路由表。You cannot reuse a route table with multiple clusters due to the potential for overlapping pod CIDRs and conflicting routing rules.

创建自定义路由表并将其与虚拟网络中的子网关联后,可以创建使用路由表的新 AKS 群集。After you create a custom route table and associate it to your subnet in your virtual network, you can create a new AKS cluster that uses your route table. 需要使用计划将 AKS 群集部署到的子网 ID。You need to use the subnet ID for where you plan to deploy your AKS cluster. 此子网还必须与自定义路由表关联。This subnet also must be associated with your custom route table.

# Find your subnet ID
az network vnet subnet list --resource-group
                            --vnet-name
                            [--subscription]
# Create a kubernetes cluster with with a custom subnet preconfigured with a route table
az aks create -g MyResourceGroup -n MyManagedCluster --vnet-subnet-id MySubnetID

后续步骤Next steps

在现有虚拟网络子网中部署 AKS 群集后,现在可以像平时一样使用该群集。With an AKS cluster deployed into your existing virtual network subnet, you can now use the cluster as normal. 开始使用 Helm 部署现有应用,或使用 Helm 创建新应用Get started with deploy existing apps using Helm, or creating new apps using Helm.