在使用应用程序网关入口控制器的 AKS 群集中启用多命名空间支持Enable multiple Namespace support in an AKS cluster with Application Gateway Ingress Controller

动机Motivation

Kubernetes 命名空间使 Kubernetes 群集可分区并分配到较大团队的子组。Kubernetes Namespaces make it possible for a Kubernetes cluster to be partitioned and allocated to subgroups of a larger team. 然后,这些子团队可以通过对资源、安全性、配置等的更精细控制来部署和管理基础结构。Kubernetes 允许在每个命名空间内部单独定义一个或多个入口资源。These subteams can then deploy and manage infrastructure with finer controls of resources, security, configuration etc. Kubernetes allows for one or more ingress resources to be defined independently within each namespace.

从版本 0.7 开始,Azure 应用程序网关 Kubernetes 入口控制器 (AGIC) 可以从多个命名空间引入事件,并可以观察这些命名空间。As of version 0.7 Azure Application Gateway Kubernetes IngressController (AGIC) can ingest events from and observe multiple namespaces. 如果 AKS 管理员决定使用应用程序网关作为入口,则所有命名空间将使用相同的应用程序网关实例。Should the AKS administrator decide to use App Gateway as an ingress, all namespaces will use the same instance of Application Gateway. 入口控制器的单个安装将会监视可访问的命名空间,并配置关联的应用程序网关。A single installation of Ingress Controller will monitor accessible namespaces and will configure the Application Gateway it is associated with.

AGIC 版本 0.7 将继续专门观察 default 命名空间,除非在 Helm 配置中将其显式更改为一个或多个不同的命名空间(请参阅以下部分)。Version 0.7 of AGIC will continue to exclusively observe the default namespace, unless this is explicitly changed to one or more different namespaces in the Helm configuration (see section below).

启用多命名空间支持Enable multiple namespace support

若要启用多命名空间支持:To enable multiple namespace support:

  1. 按以下方式之一修改 helm-config.yaml 文件:modify the helm-config.yaml file in one of the following ways:
    • helm-config.yaml 中删除整个 watchNamespace 键 - AGIC 将观察所有命名空间delete the watchNamespace key entirely from helm-config.yaml - AGIC will observe all namespaces
    • watchNamespace 设置为空字符串 - AGIC 将观察所有命名空间set watchNamespace to an empty string - AGIC will observe all namespaces
    • 添加以逗号 (watchNamespace: default,secondNamespace) 分隔的多个命名空间 - AGIC 将专门观察这些命名空间add multiple namespaces separated by a comma (watchNamespace: default,secondNamespace) - AGIC will observe these namespaces exclusively
  2. 使用 helm install -f helm-config.yaml application-gateway-kubernetes-ingress/ingress-azure 应用 Helm 模板更改apply Helm template changes with: helm install -f helm-config.yaml application-gateway-kubernetes-ingress/ingress-azure

部署并可以观察多个命名空间后,AGIC 将会:Once deployed with the ability to observe multiple namespaces, AGIC will:

  • 列出所有可访问的命名空间中的资源list ingress resources from all accessible namespaces
  • 筛选带有 kubernetes.io/ingress.class: azure/application-gateway 批注的入口资源filter to ingress resources annotated with kubernetes.io/ingress.class: azure/application-gateway
  • 编写组合的应用程序网关配置compose combined Application Gateway config
  • 通过 ARM 将配置应用到关联的应用程序网关apply the config to the associated Application Gateway via ARM

有冲突的配置Conflicting Configurations

多个带有命名空间的入口资源可能会指示 AGIC 为单个应用程序网关创建有冲突的配置。Multiple namespaced ingress resources could instruct AGIC to create conflicting configurations for a single Application Gateway. (例如,两个入口声明同一个域。)(Two ingresses claiming the same domain for instance.)

在层次结构的顶层 - 侦听器(IP 地址、端口和主机)和路由规则(绑定侦听器、后端池和 HTTP 设置)可由多个命名空间/入口创建和共享。At the top of the hierarchy - listeners (IP address, port, and host) and routing rules (binding listener, backend pool, and HTTP settings) could be created and shared by multiple namespaces/ingresses.

另一方面 - 路径、后端池、HTTP 设置和 TLS 证书只能由一个命名空间创建,重复项将被删除。On the other hand - paths, backend pools, HTTP settings, and TLS certificates could be created by one namespace only and duplicates will be removed.

例如,假设以下重复入口资源为 www.contoso.com 定义了命名空间 stagingproductionFor example, consider the following duplicate ingress resources defined namespaces staging and production for www.contoso.com:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: websocket-ingress
  namespace: staging
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
spec:
  rules:
    - host: www.contoso.com
      http:
        paths:
          - backend:
              serviceName: web-service
              servicePort: 80
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: websocket-ingress
  namespace: production
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
spec:
  rules:
    - host: www.contoso.com
      http:
        paths:
          - backend:
              serviceName: web-service
              servicePort: 80

尽管这两个入口资源要求将 www.contoso.com 的流量路由到相应的 Kubernetes 命名空间,但只有一个后端能够为该流量提供服务。Despite the two ingress resources demanding traffic for www.contoso.com to be routed to the respective Kubernetes namespaces, only one backend can service the traffic. AGIC 将会根据“先到者先接受服务”的原则,为其中的一个资源创建配置。AGIC would create a configuration on "first come, first served" basis for one of the resources. 如果同时创建了两个入口资源,则字母顺序靠前的资源将优先接受服务。If two ingresses resources are created at the same time, the one earlier in the alphabet will take precedence. 在上述示例中,我们只能为 production 入口创建设置。From the example above we will only be able to create settings for the production ingress. 将在应用程序网关中配置以下资源:Application Gateway will be configured with the following resources:

  • 侦听器:fl-www.contoso.com-80Listener: fl-www.contoso.com-80
  • 路由规则:rr-www.contoso.com-80Routing Rule: rr-www.contoso.com-80
  • 后端池:pool-production-contoso-web-service-80-bp-80Backend Pool: pool-production-contoso-web-service-80-bp-80
  • HTTP 设置:bp-production-contoso-web-service-80-80-websocket-ingressHTTP Settings: bp-production-contoso-web-service-80-80-websocket-ingress
  • 运行状况探测:pb-production-contoso-web-service-80-websocket-ingressHealth Probe: pb-production-contoso-web-service-80-websocket-ingress

请注意,创建的应用程序网关资源(侦听器和路由规则除外)包含创建时所针对的命名空间 (production) 的名称。 Note that except for listener and routing rule, the Application Gateway resources created include the name of the namespace (production) for which they were created.

如果在不同时间点将两个入口资源引入 AKS 群集,AGIC 最终可能会出现这种情况:重新配置应用程序网关,并将流量从 namespace-B 重新路由到 namespace-AIf the two ingress resources are introduced into the AKS cluster at different points in time, it is likely for AGIC to end up in a scenario where it reconfigures Application Gateway and re-routes traffic from namespace-B to namespace-A.

例如,如果先添加 staging,则 AGIC 会将应用程序网关配置为向临时后端池路由流量。For example if you added staging first, AGIC will configure Application Gateway to route traffic to the staging backend pool. 在后续阶段引入 production 入口会导致 AGIC 对应用程序网关重新编程,从而开始将流量路由到 production 后端池。At a later stage, introducing production ingress, will cause AGIC to reprogram Application Gateway, which will start routing traffic to the production backend pool.

限制对命名空间的访问Restrict Access to Namespaces

默认情况下,AGIC 根据任何命名空间中带批注的入口配置应用程序网关。By default AGIC will configure Application Gateway based on annotated Ingress within any namespace. 如果你想要限制此行为,可使用以下选项:Should you want to limit this behavior you have the following options:

  • 通过 helm-config.yaml 中的 watchNamespace YAML 键定义 AGIC 应观察的命名空间,以限制命名空间limit the namespaces, by explicitly defining namespaces AGIC should observe via the watchNamespace YAML key in helm-config.yaml
  • 使用角色/角色绑定来限制 AGIC 只能访问特定的命名空间use Role/RoleBinding to limit AGIC to specific namespaces

示例 Helm 配置文件Sample Helm config file

    # This file contains the essential configs for the ingress controller helm chart

    # Verbosity level of the App Gateway Ingress Controller
    verbosityLevel: 3
    
    ################################################################################
    # Specify which application gateway the ingress controller will manage
    #
    appgw:
        subscriptionId: <subscriptionId>
        resourceGroup: <resourceGroupName>
        name: <applicationGatewayName>
    
        # Setting appgw.shared to "true" will create an AzureIngressProhibitedTarget CRD.
        # This prohibits AGIC from applying config for any host/path.
        # Use "kubectl get AzureIngressProhibitedTargets" to view and change this.
        shared: false
    
    ################################################################################
    # Specify which kubernetes namespace the ingress controller will watch
    # Default value is "default"
    # Leaving this variable out or setting it to blank or empty string would
    # result in Ingress Controller observing all acessible namespaces.
    #
    # kubernetes:
    #   watchNamespace: <namespace>
    
    ################################################################################
    # Specify the authentication with Azure Resource Manager
    #
    # Two authentication methods are available:
    # - Option 1: AAD-Pod-Identity (https://github.com/Azure/aad-pod-identity)
    armAuth:
        type: aadPodIdentity
        identityResourceID: <identityResourceId>
        identityClientID:  <identityClientId>
    
    ## Alternatively you can use Service Principal credentials
    # armAuth:
    #    type: servicePrincipal
    #    secretJSON: <<Generate this value with: "az ad sp create-for-rbac --subscription <subscription-uuid> --sdk-auth | base64 -w0" >>
    
    ################################################################################
    # Specify if the cluster is RBAC enabled or not
    rbac:
        enabled: false # true/false
    
    # Specify aks cluster related information. THIS IS BEING DEPRECATED.
    aksClusterConfiguration:
        apiServerAddress: <aks-api-server-address>
    ```