Annotations for Application Gateway Ingress Controller

  • Article

You can annotate the Kubernetes ingress resource with arbitrary key/value pairs. Application Gateway Ingress Controller (AGIC) relies on annotations to program Azure Application Gateway features that aren't configurable via the ingress YAML. Ingress annotations are applied to all HTTP settings, backend pools, and listeners derived from an ingress resource.

List of supported annotations

For AGIC to observe an ingress resource, the resource must be annotated with kubernetes.io/ingress.class: azure/application-gateway.

Annotation key Value type Default value Allowed values
appgw.ingress.kubernetes.io/backend-path-prefix string nil
appgw.ingress.kubernetes.io/backend-hostname string nil
appgw.ingress.kubernetes.io/health-probe-hostname string 127.0.0.1
appgw.ingress.kubernetes.io/health-probe-port int32 80
appgw.ingress.kubernetes.io/health-probe-path string /
appgw.ingress.kubernetes.io/health-probe-status-code string 200-399
appgw.ingress.kubernetes.io/health-probe-interval int32 30 (seconds)
appgw.ingress.kubernetes.io/health-probe-timeout int32 30 (seconds)
appgw.ingress.kubernetes.io/health-probe-unhealthy-threshold int32 3
appgw.ingress.kubernetes.io/ssl-redirect bool false
appgw.ingress.kubernetes.io/connection-draining bool false
appgw.ingress.kubernetes.io/connection-draining-timeout int32 (seconds) 30
appgw.ingress.kubernetes.io/use-private-ip bool false
appgw.ingress.kubernetes.io/override-frontend-port bool false
appgw.ingress.kubernetes.io/cookie-based-affinity bool false
appgw.ingress.kubernetes.io/request-timeout int32 (seconds) 30
appgw.ingress.kubernetes.io/use-private-ip bool false
appgw.ingress.kubernetes.io/backend-protocol string http http, https
appgw.ingress.kubernetes.io/hostname-extension string nil
appgw.ingress.kubernetes.io/waf-policy-for-path string nil
appgw.ingress.kubernetes.io/appgw-ssl-certificate string nil
appgw.ingress.kubernetes.io/appgw-ssl-profile string nil
appgw.ingress.kubernetes.io/appgw-trusted-root-certificate string nil
appgw.ingress.kubernetes.io/rewrite-rule-set string nil
appgw.ingress.kubernetes.io/rewrite-rule-set-custom-resource
appgw.ingress.kubernetes.io/rule-priority int32 nil

Backend Path Prefix

The following annotation allows the backend path specified in an ingress resource to be rewritten with the specified prefix. Use it to expose services whose endpoints are different from the endpoint names that you use to expose a service in an ingress resource.

Usage

appgw.ingress.kubernetes.io/backend-path-prefix: <path prefix>

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-bkprefix
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/backend-path-prefix: "/test/"
spec:
  rules:
  - http:
      paths:
      - path: /hello/
        pathType: Exact
        backend:
          service:
            name: go-server-service
            port:
              number: 80

The preceding example defines an ingress resource named go-server-ingress-bkprefix with an annotation named appgw.ingress.kubernetes.io/backend-path-prefix: "/test/". The annotation tells Application Gateway to create an HTTP setting that has a path prefix override for the path /hello to /test/.

The example defines only one rule. However, the annotations apply to the entire ingress resource. So if you define multiple rules, you set up the backend path prefix for each of the specified paths. If you want different rules with different path prefixes (even for the same service), you need to define different ingress resources.

Backend Hostname

Use the following annotation to specify the hostname that Application Gateway should use while talking to the pods.

Usage

appgw.ingress.kubernetes.io/backend-hostname: "internal.example.com"

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-timeout
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/backend-hostname: "internal.example.com"
spec:
  rules:
  - http:
      paths:
      - path: /hello/
        backend:
          service:
            name: store-service
            port:
              number: 80
        pathType: Exact

Custom Health Probe

You can configure Application Gateway to send custom health probes to the backend address pool. When the following annotations are present, the Kubernetes ingress controller creates a custom probe to monitor the backend application. The controller then applies the changes to Application Gateway.

  • health-probe-hostname: This annotation allows a custom hostname on the health probe.
  • health-probe-port: This annotation configures a custom port for the health probe.
  • health-probe-path: This annotation defines a path for the health probe.
  • health-probe-status-code: This annotation allows the health probe to accept different HTTP status codes.
  • health-probe-interval: This annotation defines the interval at which the health probe runs.
  • health-probe-timeout: This annotation defines how long the health probe waits for a response before failing the probe.
  • health-probe-unhealthy-threshold: This annotation defines how many health probes must fail for the backend to be marked as unhealthy.

Usage

appgw.ingress.kubernetes.io/health-probe-hostname: "contoso.com"
appgw.ingress.kubernetes.io/health-probe-port: 80
appgw.ingress.kubernetes.io/health-probe-path: "/"
appgw.ingress.kubernetes.io/health-probe-status-code: "100-599"
appgw.ingress.kubernetes.io/health-probe-interval: 30
appgw.ingress.kubernetes.io/health-probe-timeout: 30
appgw.ingress.kubernetes.io/health-probe-unhealthy-threshold: 2

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/health-probe-hostname: "contoso.com"
    appgw.ingress.kubernetes.io/health-probe-port: 81
    appgw.ingress.kubernetes.io/health-probe-path: "/probepath"
    appgw.ingress.kubernetes.io/health-probe-status-code: "100-599"
    appgw.ingress.kubernetes.io/health-probe-interval: 31
    appgw.ingress.kubernetes.io/health-probe-timeout: 31
    appgw.ingress.kubernetes.io/health-probe-unhealthy-threshold: 2
spec:
  rules:
  - http:
      paths:
      - path: /
        pathType: Exact
        backend:
          service:
            name: go-server-service
            port:
              number: 80

TLS Redirect

You can configure Application Gateway to automatically redirect HTTP URLs to their HTTPS counterparts. When this annotation is present and TLS is properly configured, the Kubernetes ingress controller creates a routing rule with a redirection configuration. The controller then applies the changes to your Application Gateway instance. The created redirect is HTTP 301 Moved Permanently.

Usage

appgw.ingress.kubernetes.io/ssl-redirect: "true"

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-redirect
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/ssl-redirect: "true"
spec:
  tls:
   - hosts:
     - www.contoso.com
     secretName: testsecret-tls
  rules:
  - host: www.contoso.com
    http:
      paths:
      - backend:
          service:
            name: websocket-repeater
            port:
              number: 80

Connection Draining

Use the following annotations if you want to use connection draining:

  • connection-draining: This annotation specifies whether to enable connection draining.
  • connection-draining-timeout: This annotation specifies a timeout, after which Application Gateway terminates the requests to the draining backend endpoint.

Usage

appgw.ingress.kubernetes.io/connection-draining: "true"
appgw.ingress.kubernetes.io/connection-draining-timeout: "60"

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-drain
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/connection-draining: "true"
    appgw.ingress.kubernetes.io/connection-draining-timeout: "60"
spec:
  rules:
  - http:
      paths:
      - path: /hello/
        pathType: Exact
        backend:
          service:
            name: go-server-service
            port:
              number: 80

Use the following annotation to enable cookie-based affinity.

Usage

appgw.ingress.kubernetes.io/cookie-based-affinity: "true"

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-affinity
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/cookie-based-affinity: "true"
spec:
  rules:
  - http:
      paths:
      - path: /hello/
        pathType: Exact
        backend:
          service:
            name: go-server-service
            port:
              number: 80

Request Timeout

Use the following annotation to specify the request timeout in seconds. After the timeout, Application Gateway fails a request if the response isn't received.

Usage

appgw.ingress.kubernetes.io/request-timeout: "20"

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-timeout
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/request-timeout: "20"
spec:
  rules:
  - http:
      paths:
      - path: /hello/
        pathType: Exact
        backend:
          service:
            name: go-server-service
            port:
              number: 80

Use Private IP

Use the following annotation to specify whether to expose this endpoint on a private IP address of Application Gateway.

For an Application Gateway instance that doesn't have a private IP, ingresses with appgw.ingress.kubernetes.io/use-private-ip: "true" are ignored. The controller logs and ingress events for those ingresses show a NoPrivateIP warning.

Usage

appgw.ingress.kubernetes.io/use-private-ip: "true"

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-privateip
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/use-private-ip: "true"
spec:
  rules:
  - http:
      paths:
      - path: /
        pathType: Exact
        backend:
          service:
            name: go-server-service
            port:
              number: 80

Override Frontend Port

Use the following annotation to configure a frontend listener to use ports other than 80 for HTTP and 443 for HTTPS.

If the port is within the Application Gateway authorized range (1 to 64999), the listener is created on this specific port. If you set an invalid port or no port in the annotation, the configuration uses the default of 80 or 443.

Usage

appgw.ingress.kubernetes.io/override-frontend-port: "port"

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-overridefrontendport
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/override-frontend-port: "8080"
spec:
  rules:
  - http:
      paths:
      - path: /hello/
        backend:
          service:
            name: store-service
            port:
              number: 80
        pathType: Exact

Note

External requests need to target http://somehost:8080 instead of http://somehost.

Backend Protocol

Use the following to specify the protocol that Application Gateway should use when it communicates with the pods. Supported protocols are HTTP and HTTPS.

Although self-signed certificates are supported on Application Gateway, AGIC currently supports HTTPS only when pods use a certificate signed by a well-known certificate authority.

Don't use port 80 with HTTPS and port 443 with HTTP on the pods.

Usage

appgw.ingress.kubernetes.io/backend-protocol: "https"

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-timeout
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/backend-protocol: "https"
spec:
  rules:
  - http:
      paths:
      - path: /
        pathType: Exact
        backend:
          service:
            name: go-server-service
            port:
              number: 443

Hostname Extension

You can configure Application Gateway to accept multiple hostnames. Use the hostname-extension annotation to define multiple hostnames, including wildcard hostnames. This action appends the hostnames onto the FQDN that's defined in the ingress spec.rules.host information on the frontend listener, so it's configured as a multisite listener.

Usage

appgw.ingress.kubernetes.io/hostname-extension: "hostname1, hostname2"

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-multisite
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/hostname-extension: "hostname1, hostname2"
spec:
  rules:
  - host: contoso.com
    http:
      paths:
      - path: /
        pathType: Exact
        backend:
          service:
            name: go-server-service
            port:
              number: 443

The preceding example configures the listener to accept traffic for the hostnames hostname1.contoso.com and hostname2.contoso.com.

WAF Policy for Path

Use the following annotation to attach an existing web application firewall (WAF) policy to the list paths for a host within a Kubernetes ingress resource that's being annotated. The WAF policy is applied to both /ad-server and /auth URLs.

Usage

appgw.ingress.kubernetes.io/waf-policy-for-path: "/subscriptions/aaaa0a0a-bb1b-cc2c-dd3d-eeeeee4e4e4e/resourceGroups/SampleRG/providers/Microsoft.Network/applicationGatewayWebApplicationFirewallPolicies/AGICWAFPolcy"

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ad-server-ingress
  namespace: commerce
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/waf-policy-for-path: "/subscriptions/abcd/resourceGroups/rg/providers/Microsoft.Network/applicationGatewayWebApplicationFirewallPolicies/adserver"
spec:
  rules:
  - http:
      paths:
      - path: /ad-server
        backend:
          service:
            name: ad-server
            port:
              number: 80
        pathType: Exact
      - path: /auth
        backend:
          service:
            name: auth-server
            port:
              number: 80
        pathType: Exact

Application Gateway SSL Certificate

You can configure the SSL certificate to Application Gateway from either a local PFX certificate file or a reference to an Azure Key Vault unversioned secret ID. When the annotation is present with a certificate name and the certificate is preinstalled in Application Gateway, the Kubernetes ingress controller creates a routing rule with an HTTPS listener and applies the changes to your Application Gateway instance. You can also use the appgw-ssl-certificate annotation together with an ssl-redirect annotation in the case of an SSL redirect.

Note

The appgw-ssl-certificate annotation is ignored when the TLS specification is defined in ingress at the same time. If you want different certificates with different hosts (termination of multiple TLS certificates), you need to define different ingress resources.

Usage

appgw.ingress.kubernetes.io/appgw-ssl-certificate: "name-of-appgw-installed-certificate"

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-certificate
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/appgw-ssl-certificate: "name-of-appgw-installed-certificate"
spec:
  rules:
  - host: www.contoso.com
    http:
      paths:
      - backend:
          service:
            name: websocket-repeater
            port:
              number: 80

Application Gateway SSL Profile

You can configure an SSL profile on the Application Gateway instance per listener. When the annotation is present with a profile name and the profile is preinstalled in Application Gateway, the Kubernetes ingress controller creates a routing rule with an HTTPS listener and applies the changes to your Application Gateway instance.

Usage

appgw.ingress.kubernetes.io/appgw-ssl-certificate: "name-of-appgw-installed-certificate"
appgw.ingress.kubernetes.io/appgw-ssl-profile: "SampleSSLProfile"

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-certificate
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/appgw-ssl-certificate: "name-of-appgw-installed-certificate"
    appgw.ingress.kubernetes.io/appgw-ssl-profile: "SampleSSLProfile"
spec:
  rules:
  - host: www.contoso.com
    http:
      paths:
      - backend:
          service:
            name: websocket-repeater
            port:
              number: 80

Application Gateway Trusted Root Certificate

You now can configure your own root certificates to Application Gateway to be trusted via AGIC. You can use the appgw-trusted-root-certificate annotation together with the backend-protocol annotation to indicate end-to-end SSL encryption. If you specify multiple root certificates, separate them with a comma; for example, name-of-my-root-cert1,name-of-my-root-cert2.

Usage

appgw.ingress.kubernetes.io/backend-protocol: "https"
appgw.ingress.kubernetes.io/appgw-trusted-root-certificate: "name-of-my-root-cert1"

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-certificate
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/backend-protocol: "https"
    appgw.ingress.kubernetes.io/appgw-trusted-root-certificate: "name-of-my-root-cert1"
spec:
  rules:
  - host: www.contoso.com
    http:
      paths:
      - backend:
          service:
            name: websocket-repeater
            port:
              number: 80

Rewrite Rule Set

Use the following annotation to assign an existing rewrite rule set to the corresponding request routing rule.

Usage

appgw.ingress.kubernetes.io/rewrite-rule-set: <rewrite rule set name>

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-bkprefix
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/rewrite-rule-set: add-custom-response-header
spec:
  rules:
  - http:
      paths:
      - path: /
        pathType: Exact
        backend:
          service:
            name: go-server-service
            port:
              number: 8080

Rule Priority

The following annotation allows for the Application Gateway ingress controller to explicitly set the priority of the associated request routing rules.

Usage

appgw.ingress.kubernetes.io/rule-priority:

Example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: go-server-ingress-rulepriority
  namespace: test-ag
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/rule-priority: 10
spec:
  rules:
  - http:
      paths:
      - path: /
        pathType: Exact
        backend:
          service:
            name: go-server-service
            port:
              number: 8080

The preceding example sets a priority of 10 for the request routing rule.