Configure Istio ingress with the Kubernetes Gateway API

In addition to Istio's own ingress traffic management API, the Istio service mesh add-on also supports the Kubernetes Gateway API for ingress traffic management. In order to receive support from Azure for Gateway API-based deployments with the Istio add-on, you must have the Managed Gateway API installation enabled on your cluster. You can use both the Istio Gateway API automated deployment model or the manual deployment model for ingress traffic management. ConfigMap customizations must fall under the resource customization allowlist.

Limitations

• Using the Kubernetes Gateway API for egress traffic management with the Istio add-on is only supported for the manual deployment model.
• ConfigMap customizations for Gateway resources must fall within the resource customization allowlist. Fields not on the allowlist are disallowed and blocked via add-on managed webhooks. See the Istio add-on support policy for more information allowed, blocked, and supported features.

Prerequisites

• Enable the Managed Gateway API Installation.
• Install Istio add-on revision asm-1-26 or higher. Follow the installation guide if you don't have the Istio add-on installed yet, or the upgrade guide if you are on a lower minor revision.

Configure ingress using a Kubernetes Gateway

Deploy sample application

First, deploy the sample httpbin application in the default namespace:

kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.26/samples/httpbin/httpbin.yaml

Create Kubernetes Gateway and HTTPRoute

Next, deploy a Gateway API configuration in the default namespace with the gatewayClassName set to istio.

kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: httpbin-gateway
spec:
  gatewayClassName: istio
  listeners:
  - name: http
    port: 80
    protocol: HTTP
    allowedRoutes:
      namespaces:
        from: Same
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: http
  namespace: default
spec:
  parentRefs:
  - name: httpbin-gateway
  hostnames: ["httpbin.example.com"]
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /get
    backendRefs:
    - name: httpbin
      port: 8000
EOF

Verify that a Deployment, Service, HorizontalPodAutoscaler, and PodDisruptionBudget get created for httpbin-gateway:

kubectl get deployment httpbin-gateway-istio

NAME                    READY   UP-TO-DATE   AVAILABLE   AGE
httpbin-gateway-istio   2/2     2            2           31m

kubectl get service httpbin-gateway-istio

NAME                    TYPE           CLUSTER-IP   EXTERNAL-IP      PORT(S)                        AGE
httpbin-gateway-istio   LoadBalancer   10.0.65.45   <external-ip>    15021:32053/TCP,80:31587/TCP   33m

kubectl get hpa httpbin-gateway-istio

NAME                    REFERENCE                          TARGETS       MINPODS   MAXPODS   REPLICAS   AGE
httpbin-gateway-istio   Deployment/httpbin-gateway-istio   cpu: 3%/80%   2         5         3          34m

kubectl get pdb httpbin-gateway-istio

NAME                    MIN AVAILABLE   MAX UNAVAILABLE   ALLOWED DISRUPTIONS   AGE
httpbin-gateway-istio   1               N/A               2                     36m

You can configure these resource settings by modifying the GatewayClass defaults ConfigMap or by attaching a ConfigMap to a specific Gateway.

Send request to sample application

Finally, try sending a curl request to the httpbin application. First, set the INGRESS_HOST environment variable:

kubectl wait --for=condition=programmed gateways.gateway.networking.k8s.io httpbin-gateway
export INGRESS_HOST=$(kubectl get gateways.gateway.networking.k8s.io httpbin-gateway -ojsonpath='{.status.addresses[0].value}')

Then, try sending an HTTP request to httpbin:

curl -s -I -HHost:httpbin.example.com "http://$INGRESS_HOST/get"

You should see an HTTP 200 response.

Securing Istio ingress traffic with the Kubernetes Gateway API

The Istio add-on supports syncing secrets from Azure Key Vault (AKV) for securing Gateway API-based ingress traffic with Transport Layer Security (TLS) termination or Server Name Indication (SNI) passthrough. You can follow the secure ingress gateway document for syncing secrets from AKV onto your AKS cluster using the AKV Secrets Store Container Storage Interface (CSI) Driver add-on.

Resource customizations

The Istio add-on also supports customizations of the resources generated for the Gateways, namely:

• Service
• Deployment
• Horizontal Pod Autoscaler (HPA)
• Pod Disruption Budget (PDB)

The default settings for these resources are set in the istio-gateway-class-defaults ConfigMap in the aks-istio-system namespace. This ConfigMap must have the gateway.istio.io/defaults-for-class label set to istio for the customizations to take effect for all Gateways with spec.gatewayClassName: istio. The GatewayClass-level ConfigMap is installed by default in the aks-istio-system namespace when the Managed Gateway API installation is enabled. It could take up to ~5 minutes for the istio-gateway-class-defaults ConfigMap to get deployed after installing the Managed Gateway API CRDs.

kubectl get configmap istio-gateway-class-defaults -n aks-istio-system -o yaml

...
data:
  horizontalPodAutoscaler: |
    spec:
      minReplicas: 2
      maxReplicas: 5
  podDisruptionBudget: |
    spec:
      minAvailable: 1
...

As detailed in the subsequent sections, you can modify these settings for all Istio Gateways at a GatewayClass level by updating the istio-gateway-class-defaults ConfigMap, or you can set them for individual Gateway resources. For both the GatewayClass-level and Gateway-level ConfigMaps, fields must be allowlisted for the given resource. If there are customizations both for the GatewayClass and an individual Gateway, the Gateway-level configuration takes precedence.

Resource customization allowlist

Fields not on the allowlist for the resource are disallowed and blocked via add-on managed webhooks. See the Istio add-on support policy for more information allowed, blocked, and supported features.

Deployment Fields

Service Fields

HorizontalPodAutoscaler (HPA) Fields

PodDisruptionBudget (PDB) Fields

Configure GatewayClass-level settings

Update the GatewayClass-level ConfigMap in the aks-istio-system namespace:

kubectl edit cm istio-gateway-class-defaults -n aks-istio-system

Edit the resource settings:

...
data:
  deployment: |
    metadata:
      labels:
        test.azureservicemesh.io/deployment-config: "updated"
  horizontalPodAutoscaler: |
    spec:
      minReplicas: 3
      maxReplicas: 6
  podDisruptionBudget: |
    spec:
      minAvailable: 1
...

Now, you should see the the HPA for httpbin-gateway that you created earlier get updated:

kubectl get hpa httpbin-gateway-istio

NAME                    REFERENCE                          TARGETS       MINPODS   MAXPODS   REPLICAS   AGE
httpbin-gateway-istio   Deployment/httpbin-gateway-istio   cpu: 3%/80%   3         6         3          36m

Also verify that the Deployment is updated with the new label:

kubectl get deployment httpbin-gateway-istio -ojsonpath='{.metadata.labels.test\.azureservicemesh\.io\/deployment-config}'

updated

Configure settings for a specific gateway

Create a ConfigMap with resource customizations for the httpbin Gateway:

kubectl apply -f - <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
  name: gw-options
data:
  horizontalPodAutoscaler: |
    spec:
      minReplicas: 2
      maxReplicas: 4
  deployment: |
    metadata:
      labels:
        test.azureservicemesh.io/deployment-config: "updated-per-gateway"
EOF

Update the httpbin Gateway to reference the ConfigMap:

kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: httpbin-gateway
spec:
  gatewayClassName: istio
  infrastructure:
    parametersRef:
      group: ""
      kind: ConfigMap
      name: gw-options
  listeners:
  - name: http
    port: 80
    protocol: HTTP
    allowedRoutes:
      namespaces:
        from: Same
EOF

Verify that the HPA is updated with the new min/max values. If you also configured the GatewayClass-level ConfigMap, the Gateway-level settings should take precedence:

kubectl get hpa httpbin-gateway-istio

NAME                    REFERENCE                          TARGETS       MINPODS   MAXPODS   REPLICAS   AGE
httpbin-gateway-istio   Deployment/httpbin-gateway-istio   cpu: 3%/80%   2         4         2          4h14m

Also inspect the Deployment labels to ensure that the test.azureservicemesh.io/deployment-config is updated to the new value:

kubectl get deployment httpbin-gateway-istio -ojsonpath='{.metadata.labels.test\.azureservicemesh\.io\/deployment-config}'

updated-per-gateway

Next steps

• Deploy egress gateways for the Istio service mesh add-on