安装使用现有应用程序网关的应用程序网关入口控制器 (AGIC)Install an Application Gateway Ingress Controller (AGIC) using an existing Application Gateway
应用程序网关入口控制器 (AGIC) 是 Kubernetes 群集中的一个 pod。The Application Gateway Ingress Controller (AGIC) is a pod within your Kubernetes cluster. AGIC 监视 Kubernetes 入口资源,并根据 Kubernetes 群集状态创建和应用应用程序网关配置。AGIC monitors the Kubernetes Ingress resources, and creates and applies Application Gateway config based on the status of the Kubernetes cluster.
大纲:Outline:
- 先决条件Prerequisites
- Azure 资源管理器身份验证 (ARM)Azure Resource Manager Authentication (ARM)
- 选项 1:设置 aad-pod-identity 并在 ARM 上创建 Azure 标识Option 1: Set up aad-pod-identity and create Azure Identity on ARMs
- 选项 2:使用服务主体Option 2: Using a Service Principal
- 使用 Helm 安装入口控制器Install Ingress Controller using Helm
- 多群集/共享应用程序网关:在一个或多个 AKS 群集和/或其他 Azure 组件之间共享应用程序网关的环境中安装 AGIC。Multi-cluster / Shared Application Gateway: Install AGIC in an environment, where Application Gateway is shared between one or more AKS clusters and/or other Azure components.
先决条件Prerequisites
本文档假设已安装以下工具和基础结构:This document assumes you already have the following tools and infrastructure installed:
- 已启用高级网络的 AKSAKS with Advanced Networking enabled
- AKS 所在的同一虚拟网络中的应用程序网关 v2Application Gateway v2 in the same virtual network as AKS
- 已在 AKS 群集上安装 AAD Pod IdentityAAD Pod Identity installed on your AKS cluster
在安装 AGIC 之前,请 备份应用程序网关的配置:Please backup your Application Gateway's configuration before installing AGIC:
- 使用 Azure 门户导航到
Application Gateway
实例using Azure portal navigate to yourApplication Gateway
instance - 在
Export template
中单击Download
fromExport template
clickDownload
下载的 zip 文件包含 JSON 模板、bash 和 PowerShell 脚本,如果需要,可使用它们来还原应用程序网关The zip file you downloaded will have JSON templates, bash, and PowerShell scripts you could use to restore App Gateway should that become necessary
安装 HelmInstall Helm
Helm 是 Kubernetes 的包管理器。Helm is a package manager for Kubernetes. 我们将利用它来安装 application-gateway-kubernetes-ingress
包。We will leverage it to install the application-gateway-kubernetes-ingress
package.
安装 Helm 并运行以下命令来添加
application-gateway-kubernetes-ingress
Helm 包:Install Helm and run the following to addapplication-gateway-kubernetes-ingress
helm package:- 已启用 Kubernetes RBAC 的 AKS 群集Kubernetes RBAC enabled AKS cluster
kubectl create serviceaccount --namespace kube-system tiller-sa kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=kube-system:tiller-sa helm init --tiller-namespace kube-system --service-account tiller-sa
- 已禁用 Kubernetes RBAC 的 AKS 群集Kubernetes RBAC disabled AKS cluster
helm init
添加 AGIC Helm 存储库:Add the AGIC Helm repository:
helm repo add application-gateway-kubernetes-ingress https://appgwingress.blob.core.windows.net/ingress-azure-helm-package/ helm repo update
Azure 资源管理器身份验证Azure Resource Manager Authentication
AGIC 与 Kubernetes API 服务器和 Azure 资源管理器通信。AGIC communicates with the Kubernetes API server and the Azure Resource Manager. 它需要一个标识来访问这些 API。It requires an identity to access these APIs.
设置 AAD Pod IdentitySet up AAD Pod Identity
AAD Pod Identity 是一个类似于 AGIC 的控制器,它也在 AKS 上运行。AAD Pod Identity is a controller, similar to AGIC, which also runs on your AKS. 它将 Azure Active Directory 标识绑定到 Kubernetes pod。It binds Azure Active Directory identities to your Kubernetes pods. Kubernetes pod 中的应用程序需有标识才能与其他 Azure 组件通信。Identity is required for an application in a Kubernetes pod to be able to communicate with other Azure components. 在这种特定的情况下,我们需要授权 AGIC pod 向 ARM 发出 HTTP 请求。In the particular case here, we need authorization for the AGIC pod to make HTTP requests to ARM.
请根据 AAD Pod Identity 安装说明将此组件添加到 AKS。Follow the AAD Pod Identity installation instructions to add this component to your AKS.
接下来,需要创建一个 Azure 标识并向其授予对 ARM 的权限。Next we need to create an Azure identity and give it permissions ARM. 运行以下所有命令并创建标识:Run all of the following commands and create an identity:
在 AKS 节点所在的同一个资源组 中创建 Azure 标识。Create an Azure identity in the same resource group as the AKS nodes. 选取正确的资源组十分重要。Picking the correct resource group is important. 以下命令中所需的资源组不是 AKS 门户窗格中提到的资源组,The resource group required in the command below is not the one referenced on the AKS portal pane. 而是
aks-agentpool
虚拟机的资源组。This is the resource group of theaks-agentpool
virtual machines. 通常,该资源组以MC_
开头并包含 AKS 的名称。Typically that resource group starts withMC_
and contains the name of your AKS. 例如:MC_resourceGroup_aksABCD_chinanorth2
For instance:MC_resourceGroup_aksABCD_chinanorth2
az identity create -g <agent-pool-resource-group> -n <identity-name>
对于以下角色分配命令,需要获取新建标识的
principalId
:For the role assignment commands below we need to obtainprincipalId
for the newly created identity:az identity show -g <resourcegroup> -n <identity-name>
授予该标识对应用程序网关的
Contributor
访问权限。Give the identityContributor
access to your Application Gateway. 为此,需要获取应用程序网关的 ID,如下所示:/subscriptions/A/resourceGroups/B/providers/Microsoft.Network/applicationGateways/C
For this you need the ID of the Application Gateway, which will look something like this:/subscriptions/A/resourceGroups/B/providers/Microsoft.Network/applicationGateways/C
使用以下命令获取订阅中的应用程序网关 ID 列表:
az network application-gateway list --query '[].id'
Get the list of Application Gateway IDs in your subscription with:az network application-gateway list --query '[].id'
az role assignment create \ --role Contributor \ --assignee <principalId> \ --scope <App-Gateway-ID>
授予标识对应用程序网关资源组的
Reader
访问权限。Give the identityReader
access to the Application Gateway resource group. 资源组 ID 如下所示:/subscriptions/A/resourceGroups/B
。The resource group ID would look like:/subscriptions/A/resourceGroups/B
. 可使用以下命令获取所有资源组:az group list --query '[].id'
You can get all resource groups with:az group list --query '[].id'
az role assignment create \ --role Reader \ --assignee <principalId> \ --scope <App-Gateway-Resource-Group-ID>
使用服务主体Using a Service Principal
还可以通过 Kubernetes 机密为 AGIC 提供对 ARM 的访问权限。It is also possible to provide AGIC access to ARM via a Kubernetes secret.
- 创建 Active Directory 服务主体并使用 base64 编码。Create an Active Directory Service Principal and encode with base64. JSON Blob 需要使用 base64 编码才能保存到 Kubernetes 中。The base64 encoding is required for the JSON blob to be saved to Kubernetes.
az ad sp create-for-rbac --sdk-auth | base64 -w0
- 将 base64 编码的 JSON Blob 添加到
helm-config.yaml
文件中。Add the base64 encoded JSON blob to thehelm-config.yaml
file. 下一部分提供了有关helm-config.yaml
的详细信息。More information onhelm-config.yaml
is in the next section.
armAuth:
type: servicePrincipal
secretJSON: <Base64-Encoded-Credentials>
以 Helm 图表的形式安装入口控制器Install Ingress Controller as a Helm Chart
前几个步骤将在 Kubernetes 群集上安装 Helm 的 Tiller。In the first few steps, we install Helm's Tiller on your Kubernetes cluster. 安装 AGIC Helm 包:Install the AGIC Helm package:
添加
application-gateway-kubernetes-ingress
Helm 存储库并执行 Helm 更新Add theapplication-gateway-kubernetes-ingress
helm repo and perform a helm updatehelm repo add application-gateway-kubernetes-ingress https://appgwingress.blob.core.windows.net/ingress-azure-helm-package/ helm repo update
下载 helm-config.yaml 用于配置 AGIC:Download helm-config.yaml, which will configure AGIC:
wget https://raw.githubusercontent.com/Azure/application-gateway-kubernetes-ingress/master/docs/examples/sample-helm-config.yaml -O helm-config.yaml
或复制以下 YAML 文件:Or copy the YAML file below:
# This file contains the essential configs for the ingress controller helm chart # Verbosity level of the App Gateway Ingress Controller verbosityLevel: 3 ################################################################################ # Specify which application gateway the ingress controller will manage # appgw: subscriptionId: <subscriptionId> resourceGroup: <resourceGroupName> name: <applicationGatewayName> environment: AzureChinaCloud # Setting appgw.shared to "true" will create an AzureIngressProhibitedTarget CRD. # This prohibits AGIC from applying config for any host/path. # Use "kubectl get AzureIngressProhibitedTargets" to view and change this. shared: false ################################################################################ # Specify which kubernetes namespace the ingress controller will watch # Default value is "default" # Leaving this variable out or setting it to blank or empty string would # result in Ingress Controller observing all acessible namespaces. # # kubernetes: # watchNamespace: <namespace> ################################################################################ # Specify the authentication with Azure Resource Manager # # Two authentication methods are available: # - Option 1: AAD-Pod-Identity (https://github.com/Azure/aad-pod-identity) armAuth: type: aadPodIdentity identityResourceID: <identityResourceId> identityClientID: <identityClientId> ## Alternatively you can use Service Principal credentials # armAuth: # type: servicePrincipal # secretJSON: <<Generate this value with: "az ad sp create-for-rbac --sdk-auth | base64 -w0" >> ################################################################################ # Specify if the cluster is Kubernetes RBAC enabled or not rbac: enabled: false # true/false # Specify aks cluster related information. THIS IS BEING DEPRECATED. aksClusterConfiguration: apiServerAddress: <aks-api-server-address>
编辑 helm-config.yaml 并填写
appgw
和armAuth
的值。Edit helm-config.yaml and fill in the values forappgw
andarmAuth
.nano helm-config.yaml
备注
<identity-resource-id>
和<identity-client-id>
是在上一部分设置的 Azure AD 标识的属性。The<identity-resource-id>
and<identity-client-id>
are the properties of the Azure AD Identity you setup in the previous section. 可过运行以下命令检索此信息:az identity show -g <resourcegroup> -n <identity-name>
,其中,<resourcegroup>
是部署顶级 AKS 群集对象、应用程序网关和托管标识的资源组。You can retrieve this information by running the following command:az identity show -g <resourcegroup> -n <identity-name>
, where<resourcegroup>
is the resource group in which the top level AKS cluster object, Application Gateway and Managed Identify are deployed.使用上一步骤中的
helm-config.yaml
配置安装 Helm 图表application-gateway-kubernetes-ingress
Install Helm chartapplication-gateway-kubernetes-ingress
with thehelm-config.yaml
configuration from the previous stephelm install -f <helm-config.yaml> application-gateway-kubernetes-ingress/ingress-azure
或者,可以在一个步骤中结合使用
helm-config.yaml
和 Helm 命令:Alternatively you can combine thehelm-config.yaml
and the Helm command in one step:helm install ./helm/ingress-azure \ --name ingress-azure \ --namespace default \ --debug \ --set appgw.name=applicationgatewayABCD \ --set appgw.resourceGroup=your-resource-group \ --set appgw.subscriptionId=subscription-uuid \ --set appgw.environment=AzureChinaCloud \ --set appgw.shared=false \ --set armAuth.type=servicePrincipal \ --set armAuth.secretJSON=$(az ad sp create-for-rbac --sdk-auth | base64 -w0) \ --set rbac.enabled=true \ --set verbosityLevel=3 \ --set kubernetes.watchNamespace=default \ --set aksClusterConfiguration.apiServerAddress=aks-abcdefg.hcp.chinanorth2.azmk8s.io
检查新建 pod 的日志,以确认它是否已正确启动Check the log of the newly created pod to verify if it started properly
请参阅此操作指南,了解如何使用 Azure 应用程序网关通过 HTTP 或 HTTPS 向 Internet 公开 AKS 服务。Refer to this how-to guide to understand how you can expose an AKS service over HTTP or HTTPS, to the internet, using an Azure Application Gateway.
多群集/共享应用程序网关Multi-cluster / Shared Application Gateway
默认情况下,AGIC 对它所链接到的应用程序网关拥有完全所有权。By default AGIC assumes full ownership of the Application Gateway it is linked to. AGIC 0.8.0 和更高版本可与其他 Azure 组件共享单个应用程序网关。AGIC version 0.8.0 and later can share a single Application Gateway with other Azure components. 例如,我们可以对虚拟机规模集上托管的某个应用以及某个 AKS 群集使用同一个应用程序网关。For instance, we could use the same Application Gateway for an app hosted on Virtual Machine Scale Set as well as an AKS cluster.
在启用此设置之前,请 备份应用程序网关的配置:Please backup your Application Gateway's configuration before enabling this setting:
- 使用 Azure 门户导航到
Application Gateway
实例using Azure portal navigate to yourApplication Gateway
instance - 在
Export template
中单击Download
fromExport template
clickDownload
下载的 zip 文件包含可用于还原应用程序网关的 JSON 模板、bash 和 PowerShell 脚本The zip file you downloaded will have JSON templates, bash, and PowerShell scripts you could use to restore Application Gateway
示例方案Example Scenario
让我们探讨一个虚构的应用程序网关,它将管理两个网站的流量:Let's look at an imaginary Application Gateway, which manages traffic for two web sites:
dev.contoso.com
- 托管在新 AKS 上,使用应用程序网关和 AGICdev.contoso.com
- hosted on a new AKS, using Application Gateway and AGICprod.contoso.com
- 托管在 Azure 虚拟机规模集上prod.contoso.com
- hosted on an Azure Virtual Machine Scale Set
使用默认设置时,AGIC 拥有它所指向的应用程序网关的完全所有权。With default settings, AGIC assumes 100% ownership of the Application Gateway it is pointed to. AGIC 将覆盖应用程序网关的所有配置。AGIC overwrites all of App Gateway's configuration. 如果我们手动为 prod.contoso.com
创建侦听器(在应用程序网关上),但未在 Kubernetes 入口中定义该侦听器,则 AGIC 很快就会删除 prod.contoso.com
配置。If we were to manually create a listener for prod.contoso.com
(on Application Gateway), without defining it in the Kubernetes Ingress, AGIC will delete the prod.contoso.com
config within seconds.
若要安装 AGIC 并从虚拟机规模集计算机为 prod.contoso.com
提供服务,必须将 AGIC 约束为仅配置 dev.contoso.com
。To install AGIC and also serve prod.contoso.com
from our Virtual Machine Scale Set machines, we must constrain AGIC to configuring dev.contoso.com
only. 实例化以下 CRD 可化简化此过程:This is facilitated by instantiating the following CRD:
cat <<EOF | kubectl apply -f -
apiVersion: "appgw.ingress.k8s.io/v1"
kind: AzureIngressProhibitedTarget
metadata:
name: prod-contoso-com
spec:
hostname: prod.contoso.com
EOF
以上命令创建一个 AzureIngressProhibitedTarget
对象。The command above creates an AzureIngressProhibitedTarget
object. 这会使 AGIC(0.8.0 和更高版本)意识到 prod.contoso.com
的应用程序网关配置的存在,并显式指示应用程序网关避免更改与该主机名相关的任何配置。This makes AGIC (version 0.8.0 and later) aware of the existence of Application Gateway config for prod.contoso.com
and explicitly instructs it to avoid changing any configuration related to that hostname.
在新的 AGIC 安装中启用Enable with new AGIC installation
若要将 AGIC(0.8.0 和更高版本)限制为一部分应用程序网关配置,请修改 helm-config.yaml
模板。To limit AGIC (version 0.8.0 and later) to a subset of the Application Gateway configuration modify the helm-config.yaml
template.
在 appgw:
节下,添加 shared
键并将其设置为 true
。Under the appgw:
section, add shared
key and set it to to true
.
appgw:
subscriptionId: <subscriptionId> # existing field
resourceGroup: <resourceGroupName> # existing field
name: <applicationGatewayName> # existing field
environment: AzureChinaCloud # existing field
shared: true # <<<<< Add this field to enable shared Application Gateway >>>>>
应用 Helm 更改:Apply the Helm changes:
- 使用以下命令确保安装
AzureIngressProhibitedTarget
CRD:Ensure theAzureIngressProhibitedTarget
CRD is installed with:kubectl apply -f https://raw.githubusercontent.com/Azure/application-gateway-kubernetes-ingress/ae695ef9bd05c8b708cedf6ff545595d0b7022dc/crds/AzureIngressProhibitedTarget.yaml
- 更新 Helm:Update Helm:
helm upgrade \ --recreate-pods \ -f helm-config.yaml \ ingress-azure application-gateway-kubernetes-ingress/ingress-azure
因此,AKS 将包含名为 prohibit-all-targets
的 AzureIngressProhibitedTarget
新实例:As a result your AKS will have a new instance of AzureIngressProhibitedTarget
called prohibit-all-targets
:
kubectl get AzureIngressProhibitedTargets prohibit-all-targets -o yaml
顾名思义,prohibit-all-targets
对象将禁止 AGIC 更改任何主机和路径的配置。 The object prohibit-all-targets
, as the name implies, prohibits AGIC from changing config for any host and path.
使用 appgw.shared=true
的 Helm 安装将部署 AGIC,但不会对应用程序网关进行任何更改。Helm install with appgw.shared=true
will deploy AGIC, but will not make any changes to Application Gateway.
放宽权限Broaden permissions
使用 appgw.shared=true
的 Helm 和默认的 prohibit-all-targets
会阻止 AGIC 应用任何配置。Since Helm with appgw.shared=true
and the default prohibit-all-targets
blocks AGIC from applying any config.
使用以下命令放宽 AGIC 的权限:Broaden AGIC permissions with:
创建使用特定设置的新
AzureIngressProhibitedTarget
:Create a newAzureIngressProhibitedTarget
with your specific setup:cat <<EOF | kubectl apply -f - apiVersion: "appgw.ingress.k8s.io/v1" kind: AzureIngressProhibitedTarget metadata: name: your-custom-prohibitions spec: hostname: your.own-hostname.com EOF
只有在创建自己的自定义禁止规则之后,才能删除默认的禁止规则(该规则过于宽泛):Only after you have created your own custom prohibition, you can delete the default one, which is too broad:
kubectl delete AzureIngressProhibitedTarget prohibit-all-targets
为现有的 AGIC 安装启用Enable for an existing AGIC installation
假设我们的群集中已有一个正常运行的 AKS、应用程序网关且已配置 AGIC。Let's assume that we already have a working AKS, Application Gateway, and configured AGIC in our cluster. 我们有 prod.contoso.com
的入口,并且能够成功地从 AKS 为它提供流量。We have an Ingress for prod.contoso.com
and are successfully serving traffic for it from AKS. 我们想要将 staging.contoso.com
添加到现有的应用程序网关,但需要将此网关托管在 VM 上。We want to add staging.contoso.com
to our existing Application Gateway, but need to host it on a VM. 我们将重复使用现有的应用程序网关,并为 staging.contoso.com
手动配置侦听器和后端池。We are going to reuse the existing Application Gateway and manually configure a listener and backend pools for staging.contoso.com
. 但是,手动调整应用程序网关配置(通过门户、ARM API 或 Terraform)将与 AGIC 拥有完全所有权的事实相冲突。But manually tweaking Application Gateway config (via portal, ARM APIs or Terraform) would conflict with AGIC's assumptions of full ownership. 应用更改后不久,AGIC 将会覆盖或删除这些更改。Shortly after we apply changes, AGIC will overwrite or delete them.
我们可以禁止 AGIC 对一部分配置进行更改。We can prohibit AGIC from making changes to a subset of configuration.
创建
AzureIngressProhibitedTarget
对象:Create anAzureIngressProhibitedTarget
object:cat <<EOF | kubectl apply -f - apiVersion: "appgw.ingress.k8s.io/v1" kind: AzureIngressProhibitedTarget metadata: name: manually-configured-staging-environment spec: hostname: staging.contoso.com EOF
查看新建的对象:View the newly created object:
kubectl get AzureIngressProhibitedTargets
通过门户修改应用程序网关配置 - 添加侦听器、路由规则、后端等。创建的新对象 (
manually-configured-staging-environment
) 将禁止 AGIC 覆盖与staging.contoso.com
相关的应用程序网关配置。Modify Application Gateway config via portal - add listeners, routing rules, backends etc. The new object we created (manually-configured-staging-environment
) will prohibit AGIC from overwriting Application Gateway configuration related tostaging.contoso.com
.