将 Kubernetes 群集部署到 Azure Stack Hub 上的自定义虚拟网络Deploy a Kubernetes cluster to a custom virtual network on Azure Stack Hub

可使用 Azure Kubernetes 服务 (AKS) 引擎在自定义虚拟网络上部署 Kubernetes 群集。You can deploy a Kubernetes cluster using the Azure Kubernetes Service (AKS) engine on a custom virtual network. 本文介绍如何在虚拟网络中查找所需的信息。This article looks at finding the information you need in your virtual network. 你可以在本文中找到计算群集使用的 IP 地址、在 API 模型中设置值以及设置路由表和网络安全组的步骤。You can find steps for calculating the IP addresses used by your cluster, setting the vales in the API Model, and setting the route table and network security group.

使用 AKS 引擎的 Azure Stack Hub 中的 Kubernetes 群集使用 kubenet 网络插件。The Kubernetes cluster in Azure Stack Hub using the AKS engine uses the kubenet network plugin. 有关 Azure 中的 kubenet 网络插件联网的讨论,请参阅在 Azure Kubernetes 服务 (AKS) 中结合自己的 IP 地址范围使用 kubenet 网络For a discussion of he kubenet network plugin networking in Azure, see Use kubenet networking with your own IP address ranges in Azure Kubernetes Service (AKS).

创建自定义虚拟网络时的约束Constraints when creating a custom virtual network

  • 自定义 VNET 必须与 Kubernetes 群集的所有其他组件位于同一订阅中。The custom VNET must be in the same subscription as all of the other components of the Kubernetes cluster.
  • 主节点池和代理节点池必须位于同一虚拟网络中。The pool of master nodes and the pool of agent nodes must be in the same virtual network. 你可以将节点部署到同一虚拟网络的不同子网中。You can deploy your nodes into different subnets within the same virtual network.
  • Kubernetes 群集子网必须使用自定义虚拟网络 IP 范围空间内的 IP 范围,请参阅获取 IP 地址块The Kubernetes cluster subnet must use an IP range within the space of the custom virtual network IP range, see Get the IP address block.

创建自定义虚拟网络Create custom virtual network

Azure Stack Hub 实例中必须有一个自定义虚拟网络。You must have a custom virtual network in your Azure Stack Hub instance. 有关详细信息,请参阅快速入门:使用 Azure 门户创建虚拟网络For more information, see Quickstart: Create a virtual network using the Azure portal.

在虚拟网络中创建新的子网。Create a new subnet in your virtual network. 需要获取子网资源 ID 和 IP 地址范围。You will need to the get the subnet Resource ID and IP address range. 部署群集时,将在 API 模型中使用该资源 ID 和范围。You will use the Resource ID and range in your API model when you deploy your cluster.

  1. 在 Azure Stack Hub 实例中打开 Azure Stack Hub 用户门户。Open the Azure Stack Hub user portal in your Azure Stack Hub instance.

  2. 选择“所有资源”,Select All resources.

  3. 在搜索框中输入虚拟网络的名称。Enter the name of your virtual network in the search box.

  4. 选择“子网” > “+ 子网”,添加子网 。Select Subnets > + Subnets to add a subnet.

  5. 使用 CIDR 表示法添加“名称”和“地址范围” 。Add a Name and an Address range using CIDR notation. 选择“确定” 。Select OK.

  6. 在“虚拟网络”边栏选项卡中,选择“属性” 。Select Properties in the Virtual networks blade. 复制“资源 ID”,然后添加 /subnets/<nameofyoursubnect>Copy the Resource ID, and then add /subnets/<nameofyoursubnect>. 将使用此值作为群集的 API 模型中 vnetSubnetId 密钥的值。You will use this value as your value for the vnetSubnetId key in the API model for your cluster. 子网的资源 ID 使用以下格式:The Resource ID for the subnet uses the following format:
    /subscriptions/SUB_ID/resourceGroups/RG_NAME/providers/Microsoft.Network/virtualNetworks/VNET_NAME/subnets/SUBNET_NAME

    虚拟网络资源 ID

  7. 在“虚拟网络”边栏选项卡中,选择“子网” 。Select Subnets in the Virtual networks blade. 选择一个子网名称,例如默认。Select the subnet name, for example default.

    虚拟网络 CIDR 块

  8. 在“子网”边栏选项卡中,记下地址范围和虚拟网络 CIDR 块,例如:10.1.0.0 - 10.1.0.255 (256 addresses)10.1.0.0/24In the subnet blade, make a note of the address range and the virtual network CIDR Block, for example: 10.1.0.0 - 10.1.0.255 (256 addresses) and 10.1.0.0/24.

获取 IP 地址块Get the IP address block

AKS 引擎支持部署到现有虚拟网络中。The AKS engine supports deploying into an existing virtual network. 部署到现有子网中时,群集将为代理使用一个连续 IP 的块,为主机使用另一个 IP 块。When deploying into an existing subnet, your cluster will use a block of consecutive IPs for agents and another for masters.

需要设置两个值。You will need to set two values. 需了解要为群集保留的 IP 地址数,以及子网 IP 空间中的第一个连续静态 IP。You will need to know the number of IP addresses you will need to reserve for your cluster, and the first consecutive static IP within the subnet IP space.

使用多个主节点时,AKS 引擎需要最多 16 个未使用的 IP 地址。The AKS engine requires a range of up to 16 unused IP addresses when you use multiple master nodes. 群集将为每个主节点使用一个 IP 地址(最多五个主节点)。The cluster will use one IP address for each master up to five masters. 在最后一个主节点之后,AKS 引擎还需要 10 个 IP 地址,用于为 IP 地址预留所准备的空余空间。The AKS engine will also require the next 10 IP address after the last master for headroom IP address reservation. 最后,为主机和空余空间预留使用了总共 16 个地址之后,还要为负载均衡器使用一个额外的 IP 地址。Finally, an additional IP address will be used by the load balancer after the masters and headroom reservation for a total of 16.

放置 IP 地址块时,需要为子网分配以下现有的 IP 地址:When placing your block of IP addresses, the subnet requires the following allocations of the existing IP addresses:

  • 前四个 IP 地址和最后一个 IP 地址为预留地址,不能在任何 Azure 子网中使用The first four IP addresses and the last IP address are reserved and can't be used in any Azure subnet
  • 应开启为 16 个 IP 地址配置的缓冲区并保持开启状态。A buffer of 16 IP addresses should be left open.
  • 群集的第一个 IP 的值应接近地址空间的末端值,以免发生 IP 冲突。The value of your cluster's first IP should be toward the end of the address space to avoid IP conflicts. 如果可能,为 firstConsecutiveStaticIP 属性分配一个接近子网中可用 IP 地址空间末端值的 IP 地址。If possible, assign to the firstConsecutiveStaticIP property to an IP address near the end of the available IP address space in the subnet.

在下面的示例中,可以看到基于上述各注意事项如何填充子网中的 IP 地址。In the following example, you can see how these various considerations fill out the IP range in a subnet. 上述内容适用于三个主节点。This is for three masters. 如果使用的子网有 256 个地址,例如 10.1.0.0/24,则需要将第一个连续静态 IP 地址设置为 207。If you are using a subnet with 256 addresses, for example 10.1.0.0/24, you will need to set your first consecutive static IP address at 207. 下表显示了地址和注意事项:The following table shows the addresses and considerations:

/24 子网的范围Range for /24 subnet NumberNumber 注意Note
10.1.0.0 - 10.1.0310.1.0.0 - 10.1.03 44 预留在 Azure 子网中。Reserved in Azure subnet.
10.1.0.224 - 10.1.0.23810.1.0.224-10.1.0.238 1414 AKS 引擎定义的群集的 IP 地址计数。IP address count for an AKS engine defined cluster.

3 个主机的 3 个 IP 地址3 IP addresses for 3 masters
空余空间的 10 个 IP 地址10 IP addresses for headroom
负载均衡器的 1 个 IP 地址1 IP address for the load balancer
10.1.0.239 - 10.1.0.25510.1.0.239 - 10.1.0.255 1616 16 个 IP 地址缓冲区。16 IP address buffer.
10.1.0.25610.1.0.256 11 预留在 Azure 子网中。Reserved in Azure subnet.

在此示例中,firstConsecutiveStaticIP 属性将为 10.1.0.224In this example, then firstConsecutiveStaticIP property would be 10.1.0.224.

对于较大的子网(例如超过 6 万个地址的 /16),可能会发现将静态 IP 地址设置为网络空间的末端值是不切实际的。For larger subnets, for example /16 with more than 60 thousand addresses, you may not find it to be practical to set your static IP assignments to the end of the network space. 在设置群集静态 IP 地址范围时,应选择与 IP 空间中前 24 个地址距离较远的值,以便在声明地址时可以复原群集。Set your cluster static IP address range away from the first 24 addresses in your IP space so that the cluster can be resilient when claiming addresses.

更新 API 模型Update the API model

更新用于将群集从 AKS 引擎部署到自定义虚拟网络的 API 模型。Update the API model used to deploy the cluster from the AKS engine to your custom virtual network.

在 masterProfile 中设置以下值:In masterProfile set the following values:

字段Field 示例Example 说明Description
vnetSubnetIdvnetSubnetId /subscriptions/77e28b6a-582f-42b0-94d2-93b9eca60845/resourceGroups/MDBN-K8S/providers/Microsoft.Network/virtualNetworks/MDBN-K8S/subnets/default 指定子网的资源 ID。Specify the Resource ID the subnet.
firstConsecutiveStaticIPfirstConsecutiveStaticIP 10.1.0.22410.1.0.224 firstConsecutiveStaticIP 配置属性分配一个 IP 地址,该地址接近所需子网中可用 IP 地址空间的末端值。Assign to the firstConsecutiveStaticIP configuration property an IP address that is near the end of the available IP address space in the desired subnet. firstConsecutiveStaticIP 仅适用于主池。firstConsecutiveStaticIP only applies to the master pool.

在 agentPoolProfiles 中设置以下值:In agentPoolProfiles set the following values:

字段Field 示例Example 说明Description
vnetSubnetIdvnetSubnetId /subscriptions/77e28b6a-582f-42b0-94d2-93b9eca60845/resourceGroups/MDBN-K8S/providers/Microsoft.Network/virtualNetworks/MDBN-K8S/subnets/default 指定子网的 Azure 资源管理器路径 ID。Specify the Azure Resource Manager path ID the subnet.

在“orchestratorProfile”中,找到“kubernetesConfig”并设置以下值 :In orchestratorProfile, find kubernetesConfig and set the following value:

字段Field 示例Example 描述Description
clusterSubnetclusterSubnet 172.16.244.0/24 群集子网(POD 网络)的 IP 范围必须是你定义的自定义 VNET IP 范围空间内的 IP 范围。IP range of the cluster subnet (POD network) must use an IP range within the space of the custom VNET IP range you defined.

例如:For example:

"masterProfile": {
  ...
  "vnetSubnetId": "/subscriptions/77e28b6a-582f-42b0-94d2-93b9eca60845/resourceGroups/MDBN-K8S/providers/Microsoft.Network/virtualNetworks/MDBN-K8S/subnets/default",
  "firstConsecutiveStaticIP": "10.1.0.224",
  ...
},
...
"agentPoolProfiles": [
  {
    ...
    "vnetSubnetId": "/subscriptions/77e28b6a-582f-42b0-94d2-93b9eca60845/resourceGroups/MDBN-K8S/providers/Microsoft.Network/virtualNetworks/MDBN-K8S/subnets/default",
    ...
  },
    ...
"kubernetesConfig": [
  {
    ...
    "clusterSubnet": "172.16.244.0/24",
    ...
  },

部署群集Deploy your cluster

将值添加到 API 模型后,可以通过使用 AKS 引擎的 deploy 命令从客户端计算机上部署群集。After adding the values to your API model, you can deploy your cluster from your client machine using the deploy command using the AKS engine. 有关说明,请参阅部署 Kubernetes 群集For instructions, see Deploy a Kubernetes cluster.

设置路由表Set the route table

部署群集后,返回 Azure Stack 用户门户中的虚拟网络。After you deploy your cluster, return to your virtual network in the Azure Stack user portal. 在子网边栏选项卡中同时设置路由表和网络安全组 (NSG)。Set both the route table and the network security group (NSG) in the subnet blade. 如果不使用 Azure CNI,例如,networkPluginkubernetesConfig API 模型配置对象中的 kubenetIf you're not using Azure CNI, for example, networkPlugin: kubenet in the kubernetesConfig API model configuration object. 成功将群集部署到自定义虚拟网络后,请从群集资源组中的网络边栏选项卡获取路由表资源的 ID。After you have successfully deployed a cluster to your custom virtual network, get the ID of the Route Table resource from Network blade in your cluster's resource group.

  1. 在 Azure Stack Hub 实例中打开 Azure Stack Hub 用户门户。Open the Azure Stack Hub user portal in your Azure Stack Hub instance.

  2. 选择“所有资源”,Select All resources.

  3. 在搜索框中输入虚拟网络的名称。Enter the name of your virtual network in the search box.

  4. 选择“子网”,然后选择包含群集的子网的名称。Select Subnets and then select the name of the subnet that contains your cluster.

    路由表和网络安全组

  5. 选择“路由表”,然后为群集选择路由表。Select Route table and then select the route table for your cluster.

备注

用于 Kubernetes Windows 群集的自定义虚拟网络存在一个已知问题Custom virtual network for Kubernetes Windows cluster has a known issue.

后续步骤Next steps