安装使用现有应用程序网关的应用程序网关入口控制器 (AGIC)Install an Application Gateway Ingress Controller (AGIC) using an existing Application Gateway

应用程序网关入口控制器 (AGIC) 是 Kubernetes 群集中的一个 pod。The Application Gateway Ingress Controller (AGIC) is a pod within your Kubernetes cluster. AGIC 监视 Kubernetes 入口资源，并根据 Kubernetes 群集状态创建和应用应用程序网关配置。AGIC monitors the Kubernetes Ingress resources, and creates and applies Application Gateway config based on the status of the Kubernetes cluster.

大纲：Outline:

先决条件Prerequisites

本文档假设已安装以下工具和基础结构：This document assumes you already have the following tools and infrastructure installed:

在安装 AGIC 之前，请 备份应用程序网关的配置：Please backup your Application Gateway's configuration before installing AGIC:

  1. 使用 Azure 门户导航到 Application Gateway 实例using Azure portal navigate to your Application Gateway instance
  2. 在 Export template 中单击 Downloadfrom Export template click Download

下载的 zip 文件包含 JSON 模板、bash 和 PowerShell 脚本，如果需要，可使用它们来还原应用程序网关The zip file you downloaded will have JSON templates, bash, and PowerShell scripts you could use to restore App Gateway should that become necessary

安装 HelmInstall Helm

Helm 是 Kubernetes 的包管理器。Helm is a package manager for Kubernetes. 我们将利用它来安装 application-gateway-kubernetes-ingress 包。We will leverage it to install the application-gateway-kubernetes-ingress package.

  1. 安装 Helm 并运行以下命令来添加 application-gateway-kubernetes-ingress Helm 包：Install Helm and run the following to add application-gateway-kubernetes-ingress helm package:

    • 已启用 Kubernetes RBAC 的 AKS 群集Kubernetes RBAC enabled AKS cluster
    kubectl create serviceaccount --namespace kube-system tiller-sa
kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=kube-system:tiller-sa
helm init --tiller-namespace kube-system --service-account tiller-sa
    • 已禁用 Kubernetes RBAC 的 AKS 群集Kubernetes RBAC disabled AKS cluster
    helm init

  2. 添加 AGIC Helm 存储库：Add the AGIC Helm repository:

    helm repo add application-gateway-kubernetes-ingress https://appgwingress.blob.core.windows.net/ingress-azure-helm-package/
helm repo update

Azure 资源管理器身份验证Azure Resource Manager Authentication

AGIC 与 Kubernetes API 服务器和 Azure 资源管理器通信。AGIC communicates with the Kubernetes API server and the Azure Resource Manager. 它需要一个标识来访问这些 API。It requires an identity to access these APIs.

设置 AAD Pod IdentitySet up AAD Pod Identity

AAD Pod Identity 是一个类似于 AGIC 的控制器，它也在 AKS 上运行。AAD Pod Identity is a controller, similar to AGIC, which also runs on your AKS. 它将 Azure Active Directory 标识绑定到 Kubernetes pod。It binds Azure Active Directory identities to your Kubernetes pods. Kubernetes pod 中的应用程序需有标识才能与其他 Azure 组件通信。Identity is required for an application in a Kubernetes pod to be able to communicate with other Azure components. 在这种特定的情况下，我们需要授权 AGIC pod 向 ARM 发出 HTTP 请求。In the particular case here, we need authorization for the AGIC pod to make HTTP requests to ARM.

请根据 AAD Pod Identity 安装说明将此组件添加到 AKS。Follow the AAD Pod Identity installation instructions to add this component to your AKS.

接下来，需要创建一个 Azure 标识并向其授予对 ARM 的权限。Next we need to create an Azure identity and give it permissions ARM. 运行以下所有命令并创建标识：Run all of the following commands and create an identity:

  1. 在 AKS 节点所在的同一个资源组 中创建 Azure 标识。Create an Azure identity in the same resource group as the AKS nodes. 选取正确的资源组十分重要。Picking the correct resource group is important. 以下命令中所需的资源组不是 AKS 门户窗格中提到的资源组，The resource group required in the command below is not the one referenced on the AKS portal pane. 而是 aks-agentpool 虚拟机的资源组。This is the resource group of the aks-agentpool virtual machines. 通常，该资源组以 MC_ 开头并包含 AKS 的名称。Typically that resource group starts with MC_ and contains the name of your AKS. 例如：MC_resourceGroup_aksABCD_chinanorth2For instance: MC_resourceGroup_aksABCD_chinanorth2

    az identity create -g <agent-pool-resource-group> -n <identity-name>

  2. 对于以下角色分配命令，需要获取新建标识的 principalId：For the role assignment commands below we need to obtain principalId for the newly created identity:

    az identity show -g <resourcegroup> -n <identity-name>

  3. 授予该标识对应用程序网关的 Contributor 访问权限。Give the identity Contributor access to your Application Gateway. 为此，需要获取应用程序网关的 ID，如下所示：/subscriptions/A/resourceGroups/B/providers/Microsoft.Network/applicationGateways/CFor this you need the ID of the Application Gateway, which will look something like this: /subscriptions/A/resourceGroups/B/providers/Microsoft.Network/applicationGateways/C

    使用以下命令获取订阅中的应用程序网关 ID 列表：az network application-gateway list --query '[].id'Get the list of Application Gateway IDs in your subscription with: az network application-gateway list --query '[].id'

    az role assignment create \
    --role Contributor \
    --assignee <principalId> \
    --scope <App-Gateway-ID>

  4. 授予标识对应用程序网关资源组的 Reader 访问权限。Give the identity Reader access to the Application Gateway resource group. 资源组 ID 如下所示：/subscriptions/A/resourceGroups/B。The resource group ID would look like: /subscriptions/A/resourceGroups/B. 可使用以下命令获取所有资源组：az group list --query '[].id'You can get all resource groups with: az group list --query '[].id'

    az role assignment create \
    --role Reader \
    --assignee <principalId> \
    --scope <App-Gateway-Resource-Group-ID>

使用服务主体Using a Service Principal

还可以通过 Kubernetes 机密为 AGIC 提供对 ARM 的访问权限。It is also possible to provide AGIC access to ARM via a Kubernetes secret.

  1. 创建 Active Directory 服务主体并使用 base64 编码。Create an Active Directory Service Principal and encode with base64. JSON Blob 需要使用 base64 编码才能保存到 Kubernetes 中。The base64 encoding is required for the JSON blob to be saved to Kubernetes.
az ad sp create-for-rbac --sdk-auth | base64 -w0
  1. 将 base64 编码的 JSON Blob 添加到 helm-config.yaml 文件中。Add the base64 encoded JSON blob to the helm-config.yaml file. 下一部分提供了有关 helm-config.yaml 的详细信息。More information on helm-config.yaml is in the next section.
armAuth:
    type: servicePrincipal
    secretJSON: <Base64-Encoded-Credentials>

以 Helm 图表的形式安装入口控制器Install Ingress Controller as a Helm Chart

前几个步骤将在 Kubernetes 群集上安装 Helm 的 Tiller。In the first few steps, we install Helm's Tiller on your Kubernetes cluster. 安装 AGIC Helm 包：Install the AGIC Helm package:

  1. 添加 application-gateway-kubernetes-ingress Helm 存储库并执行 Helm 更新Add the application-gateway-kubernetes-ingress helm repo and perform a helm update

    helm repo add application-gateway-kubernetes-ingress https://appgwingress.blob.core.windows.net/ingress-azure-helm-package/
helm repo update

  2. 下载 helm-config.yaml 用于配置 AGIC：Download helm-config.yaml, which will configure AGIC:

    wget https://raw.githubusercontent.com/Azure/application-gateway-kubernetes-ingress/master/docs/examples/sample-helm-config.yaml -O helm-config.yaml

    或复制以下 YAML 文件：Or copy the YAML file below:

    # This file contains the essential configs for the ingress controller helm chart

# Verbosity level of the App Gateway Ingress Controller
verbosityLevel: 3

################################################################################
# Specify which application gateway the ingress controller will manage
#
appgw:
    subscriptionId: <subscriptionId>
    resourceGroup: <resourceGroupName>
    name: <applicationGatewayName>
    environment: AzureChinaCloud

    # Setting appgw.shared to "true" will create an AzureIngressProhibitedTarget CRD.
    # This prohibits AGIC from applying config for any host/path.
    # Use "kubectl get AzureIngressProhibitedTargets" to view and change this.
    shared: false

################################################################################
# Specify which kubernetes namespace the ingress controller will watch
# Default value is "default"
# Leaving this variable out or setting it to blank or empty string would
# result in Ingress Controller observing all acessible namespaces.
#
# kubernetes:
#   watchNamespace: <namespace>

################################################################################
# Specify the authentication with Azure Resource Manager
#
# Two authentication methods are available:
# - Option 1: AAD-Pod-Identity (https://github.com/Azure/aad-pod-identity)
armAuth:
    type: aadPodIdentity
    identityResourceID: <identityResourceId>
    identityClientID:  <identityClientId>

## Alternatively you can use Service Principal credentials
# armAuth:
#    type: servicePrincipal
#    secretJSON: <<Generate this value with: "az ad sp create-for-rbac --sdk-auth | base64 -w0" >>

################################################################################
# Specify if the cluster is Kubernetes RBAC enabled or not
rbac:
    enabled: false # true/false

# Specify aks cluster related information. THIS IS BEING DEPRECATED.
aksClusterConfiguration:
    apiServerAddress: <aks-api-server-address>

  3. 编辑 helm-config.yaml 并填写 appgw 和 armAuth 的值。Edit helm-config.yaml and fill in the values for appgw and armAuth.

    nano helm-config.yaml

    备注

    <identity-resource-id> 和 <identity-client-id> 是在上一部分设置的 Azure AD 标识的属性。The <identity-resource-id> and <identity-client-id> are the properties of the Azure AD Identity you setup in the previous section. 可过运行以下命令检索此信息：az identity show -g <resourcegroup> -n <identity-name>，其中，<resourcegroup> 是部署顶级 AKS 群集对象、应用程序网关和托管标识的资源组。You can retrieve this information by running the following command: az identity show -g <resourcegroup> -n <identity-name>, where <resourcegroup> is the resource group in which the top level AKS cluster object, Application Gateway and Managed Identify are deployed.

  4. 使用上一步骤中的 helm-config.yaml 配置安装 Helm 图表 application-gateway-kubernetes-ingressInstall Helm chart application-gateway-kubernetes-ingress with the helm-config.yaml configuration from the previous step

    helm install -f <helm-config.yaml> application-gateway-kubernetes-ingress/ingress-azure

    或者，可以在一个步骤中结合使用 helm-config.yaml 和 Helm 命令：Alternatively you can combine the helm-config.yaml and the Helm command in one step:

    helm install ./helm/ingress-azure \
     --name ingress-azure \
     --namespace default \
     --debug \
     --set appgw.name=applicationgatewayABCD \
     --set appgw.resourceGroup=your-resource-group \
     --set appgw.subscriptionId=subscription-uuid \
     --set appgw.environment=AzureChinaCloud \
     --set appgw.shared=false \
     --set armAuth.type=servicePrincipal \
     --set armAuth.secretJSON=$(az ad sp create-for-rbac --sdk-auth | base64 -w0) \
     --set rbac.enabled=true \
     --set verbosityLevel=3 \
     --set kubernetes.watchNamespace=default \
     --set aksClusterConfiguration.apiServerAddress=aks-abcdefg.hcp.chinanorth2.azmk8s.io

  5. 检查新建 pod 的日志，以确认它是否已正确启动Check the log of the newly created pod to verify if it started properly

请参阅此操作指南，了解如何使用 Azure 应用程序网关通过 HTTP 或 HTTPS 向 Internet 公开 AKS 服务。Refer to this how-to guide to understand how you can expose an AKS service over HTTP or HTTPS, to the internet, using an Azure Application Gateway.

多群集/共享应用程序网关Multi-cluster / Shared Application Gateway

默认情况下，AGIC 对它所链接到的应用程序网关拥有完全所有权。By default AGIC assumes full ownership of the Application Gateway it is linked to. AGIC 0.8.0 和更高版本可与其他 Azure 组件共享单个应用程序网关。AGIC version 0.8.0 and later can share a single Application Gateway with other Azure components. 例如，我们可以对虚拟机规模集上托管的某个应用以及某个 AKS 群集使用同一个应用程序网关。For instance, we could use the same Application Gateway for an app hosted on Virtual Machine Scale Set as well as an AKS cluster.

在启用此设置之前，请 备份应用程序网关的配置：Please backup your Application Gateway's configuration before enabling this setting:

  1. 使用 Azure 门户导航到 Application Gateway 实例using Azure portal navigate to your Application Gateway instance
  2. 在 Export template 中单击 Downloadfrom Export template click Download

下载的 zip 文件包含可用于还原应用程序网关的 JSON 模板、bash 和 PowerShell 脚本The zip file you downloaded will have JSON templates, bash, and PowerShell scripts you could use to restore Application Gateway

示例方案Example Scenario

让我们探讨一个虚构的应用程序网关，它将管理两个网站的流量：Let's look at an imaginary Application Gateway, which manages traffic for two web sites:

使用默认设置时，AGIC 拥有它所指向的应用程序网关的完全所有权。With default settings, AGIC assumes 100% ownership of the Application Gateway it is pointed to. AGIC 将覆盖应用程序网关的所有配置。AGIC overwrites all of App Gateway's configuration. 如果我们手动为 prod.contoso.com 创建侦听器（在应用程序网关上），但未在 Kubernetes 入口中定义该侦听器，则 AGIC 很快就会删除 prod.contoso.com 配置。If we were to manually create a listener for prod.contoso.com (on Application Gateway), without defining it in the Kubernetes Ingress, AGIC will delete the prod.contoso.com config within seconds.

若要安装 AGIC 并从虚拟机规模集计算机为 prod.contoso.com 提供服务，必须将 AGIC 约束为仅配置 dev.contoso.com。To install AGIC and also serve prod.contoso.com from our Virtual Machine Scale Set machines, we must constrain AGIC to configuring dev.contoso.com only. 实例化以下 CRD 可化简化此过程：This is facilitated by instantiating the following CRD:

cat <<EOF | kubectl apply -f -
apiVersion: "appgw.ingress.k8s.io/v1"
kind: AzureIngressProhibitedTarget
metadata:
  name: prod-contoso-com
spec:
  hostname: prod.contoso.com
EOF

以上命令创建一个 AzureIngressProhibitedTarget 对象。The command above creates an AzureIngressProhibitedTarget object. 这会使 AGIC（0.8.0 和更高版本）意识到 prod.contoso.com 的应用程序网关配置的存在，并显式指示应用程序网关避免更改与该主机名相关的任何配置。This makes AGIC (version 0.8.0 and later) aware of the existence of Application Gateway config for prod.contoso.com and explicitly instructs it to avoid changing any configuration related to that hostname.

在新的 AGIC 安装中启用Enable with new AGIC installation

若要将 AGIC（0.8.0 和更高版本）限制为一部分应用程序网关配置，请修改 helm-config.yaml 模板。To limit AGIC (version 0.8.0 and later) to a subset of the Application Gateway configuration modify the helm-config.yaml template. 在 appgw: 节下，添加 shared 键并将其设置为 true。Under the appgw: section, add shared key and set it to to true.

appgw:
    subscriptionId: <subscriptionId>    # existing field
    resourceGroup: <resourceGroupName>  # existing field
    name: <applicationGatewayName>      # existing field
    environment: AzureChinaCloud        # existing field
    shared: true                        # <<<<< Add this field to enable shared Application Gateway >>>>>

应用 Helm 更改：Apply the Helm changes:

  1. 使用以下命令确保安装 AzureIngressProhibitedTarget CRD：Ensure the AzureIngressProhibitedTarget CRD is installed with: 
    kubectl apply -f https://raw.githubusercontent.com/Azure/application-gateway-kubernetes-ingress/ae695ef9bd05c8b708cedf6ff545595d0b7022dc/crds/AzureIngressProhibitedTarget.yaml
  2. 更新 Helm：Update Helm: 
    helm upgrade \
    --recreate-pods \
    -f helm-config.yaml \
    ingress-azure application-gateway-kubernetes-ingress/ingress-azure

因此，AKS 将包含名为 prohibit-all-targets 的 AzureIngressProhibitedTarget 新实例：As a result your AKS will have a new instance of AzureIngressProhibitedTarget called prohibit-all-targets:

kubectl get AzureIngressProhibitedTargets prohibit-all-targets -o yaml

顾名思义，prohibit-all-targets 对象将禁止 AGIC 更改任何主机和路径的配置。 The object prohibit-all-targets, as the name implies, prohibits AGIC from changing config for any host and path. 使用 appgw.shared=true 的 Helm 安装将部署 AGIC，但不会对应用程序网关进行任何更改。Helm install with appgw.shared=true will deploy AGIC, but will not make any changes to Application Gateway.

放宽权限Broaden permissions

使用 appgw.shared=true 的 Helm 和默认的 prohibit-all-targets 会阻止 AGIC 应用任何配置。Since Helm with appgw.shared=true and the default prohibit-all-targets blocks AGIC from applying any config.

使用以下命令放宽 AGIC 的权限：Broaden AGIC permissions with:

  1. 创建使用特定设置的新 AzureIngressProhibitedTarget：Create a new AzureIngressProhibitedTarget with your specific setup:

    cat <<EOF | kubectl apply -f -
apiVersion: "appgw.ingress.k8s.io/v1"
kind: AzureIngressProhibitedTarget
metadata:
  name: your-custom-prohibitions
spec:
  hostname: your.own-hostname.com
EOF

  2. 只有在创建自己的自定义禁止规则之后，才能删除默认的禁止规则（该规则过于宽泛）：Only after you have created your own custom prohibition, you can delete the default one, which is too broad:

    kubectl delete AzureIngressProhibitedTarget prohibit-all-targets

为现有的 AGIC 安装启用Enable for an existing AGIC installation

假设我们的群集中已有一个正常运行的 AKS、应用程序网关且已配置 AGIC。Let's assume that we already have a working AKS, Application Gateway, and configured AGIC in our cluster. 我们有 prod.contoso.com 的入口，并且能够成功地从 AKS 为它提供流量。We have an Ingress for prod.contoso.com and are successfully serving traffic for it from AKS. 我们想要将 staging.contoso.com 添加到现有的应用程序网关，但需要将此网关托管在 VM 上。We want to add staging.contoso.com to our existing Application Gateway, but need to host it on a VM. 我们将重复使用现有的应用程序网关，并为 staging.contoso.com 手动配置侦听器和后端池。We are going to reuse the existing Application Gateway and manually configure a listener and backend pools for staging.contoso.com. 但是，手动调整应用程序网关配置（通过门户、ARM API 或 Terraform）将与 AGIC 拥有完全所有权的事实相冲突。But manually tweaking Application Gateway config (via portal, ARM APIs or Terraform) would conflict with AGIC's assumptions of full ownership. 应用更改后不久，AGIC 将会覆盖或删除这些更改。Shortly after we apply changes, AGIC will overwrite or delete them.

我们可以禁止 AGIC 对一部分配置进行更改。We can prohibit AGIC from making changes to a subset of configuration.

  1. 创建 AzureIngressProhibitedTarget 对象：Create an AzureIngressProhibitedTarget object:

    cat <<EOF | kubectl apply -f -
apiVersion: "appgw.ingress.k8s.io/v1"
kind: AzureIngressProhibitedTarget
metadata:
  name: manually-configured-staging-environment
spec:
  hostname: staging.contoso.com
EOF

  2. 查看新建的对象：View the newly created object:

    kubectl get AzureIngressProhibitedTargets

  3. 通过门户修改应用程序网关配置 - 添加侦听器、路由规则、后端等。创建的新对象 (manually-configured-staging-environment) 将禁止 AGIC 覆盖与 staging.contoso.com 相关的应用程序网关配置。Modify Application Gateway config via portal - add listeners, routing rules, backends etc. The new object we created (manually-configured-staging-environment) will prohibit AGIC from overwriting Application Gateway configuration related to staging.contoso.com.