Available extensions for Azure Arc-enabled Kubernetes clusters

Cluster extensions for Azure Arc-enabled Kubernetes provide an Azure Resource Manager-based experience to install and manage lifecycles for different Azure capabilities in your cluster. You can deploy extensions to your clusters to support different scenarios and to improve cluster management.

The following extensions are currently available to use with Azure Arc-enabled Kubernetes clusters. All the extensions that are described in this article are cluster-scoped.

• Supported distributions: All Cloud Native Computing Foundation (CNCF)-certified Kubernetes clusters.

The Container insights feature in Azure Monitor gives you a view into the performance of workloads that are deployed on your Kubernetes cluster. Use this extension to collect memory and CPU utilization metrics from controllers, nodes, and containers.

For more information, see Container insights for Azure Arc-enabled Kubernetes clusters.

Azure Policy extends Gatekeeper, an admission controller webhook for Open Policy Agent (OPA). Use Gatekeeper with OPA to consistently apply centralized, at-scale enforcements and safeguards on your clusters.

For more information, see Understand Azure Policy for Kubernetes clusters.

• Supported distributions: AKS on Azure Local, AKS enabled by Azure Arc, Cluster API Azure, Google Kubernetes Engine, Canonical Kubernetes Distribution, OpenShift Kubernetes Distribution, Amazon Elastic Kubernetes Service, and VMware Tanzu Kubernetes Grid.

Use the Azure Key Vault Provider for Secrets Store CSI Driver to integrate an instance of Azure Key Vault as a secrets store with a Kubernetes cluster via a CSI volume. For Azure Arc-enabled Kubernetes clusters, you can install the Azure Key Vault Secrets Provider extension to fetch secrets.

For more information, see Use the Azure Key Vault Secrets Provider extension to fetch secrets into Azure Arc-enabled Kubernetes clusters.

• Supported distributions: All CNCF-certified Kubernetes clusters that are connected to Azure Arc and running Kubernetes 1.27 or later.

The Azure Key Vault Secret Store extension for Kubernetes (Secret Store) automatically syncs secrets from an instance of Azure Key Vault to a Kubernetes cluster for offline access. You can use Azure Key Vault to store, maintain, and rotate your secrets, even when you run your Kubernetes cluster in a semi-disconnected state.

We recommend the Secret Store extension for scenarios that require offline access, or if you need secrets synced to the Kubernetes secret store. If you don't need to use these features, we recommend that you instead use the Azure Key Vault Secrets Provider extension.

For more information, see Use the Secret Store extension to fetch secrets for offline access in Azure Arc-enabled Kubernetes clusters.

• Supported distributions: AKS enabled by Azure Arc, Cluster API Azure, Azure Red Hat OpenShift, Red Hat OpenShift (version 4.6 or later), Google Kubernetes Engine Standard, Amazon Elastic Kubernetes Service, VMware Tanzu Kubernetes Grid, Rancher Kubernetes Engine, and Canonical Kubernetes Distribution.

Microsoft Defender for Containers is the cloud-native solution that is used to secure your containers so you can improve, monitor, and maintain the security of your clusters, containers, and their applications. Microsoft Defender for Containers gathers information related to security, such as audit log data, from the Kubernetes cluster. Then, it provides recommendations and threat alerts based on the gathered data.

For more information, see Enable Microsoft Defender for Containers.

• Supported distributions: All CNCF-certified Kubernetes clusters.

The ArgoCD (GitOps) extension (preview) lets you use your Git repository as the source of truth for cluster configuration and application deployment.

For more information, see Tutorial: Deploy applications using GitOps with ArgoCD.

• Supported distributions: All CNCF-certified Kubernetes clusters.

GitOps on AKS and Azure Arc-enabled Kubernetes can be enabled through Flux v2, a popular open-source tool set, to help manage cluster configuration and application deployment. With the Flux extension, GitOps is enabled in the cluster as a Microsoft.KubernetesConfiguration/extensions/microsoft.flux cluster extension resource.

For more information, see Tutorial: Deploy applications using GitOps with Flux v2.

The most recent version of the Flux v2 extension and the two previous versions (N-2) are supported. We generally recommend that you use the most recent version of the extension.

The source-controller recently updated its dependency on the "github.com/Masterminds/semver/v3" Go package from version v3.3.0 to v3.3.1. This update changed semantic versioning (semver) validation rules.

What changed? In the latest version (v3.3.1) of the semver package, certain version formats that were previously considered valid are now being rejected. Specifically, version strings with leading zeroes in numeric segments (e.g., 1.0.029903) are no longer accepted as valid semver.

• GitHub Issue for reference: Previously supported chart version numbers are now invalid - fluxcd/source-controller #17380
• Package change log: Comparing v3.3.0...v3.3.1 · Masterminds/semver

Impact on users:

• Existing deployments are unaffected. Anything currently deployed will continue to function as expected.
• Future deployments or reconciliations may fail if they rely on chart versions that don't follow the stricter semver rules.
• A common error you might see: invalid chart reference: validation: chart.metadata.version "1.0.029903" is invalid

What you should do: Review your chart versions and ensure they comply with proper semantic versioning. Avoid leading zeroes in version components, and follow the semver.org specification closely.

Flux version: Release v2.5.1

• source-controller: v1.5.0
• kustomize-controller: v1.5.1
• helm-controller: v1.2.0
• notification-controller: v1.5.0
• image-automation-controller: v0.40.0
• image-reflector-controller: v0.34.0

Changes in this version include:

• Simplified flux extension's kubelet identity configuration by removing the obsolete tenant-id.

Flux version: Release v2.5.1

• source-controller: v1.5.0
• kustomize-controller: v1.5.1
• helm-controller: v1.2.0
• notification-controller: v1.5.0
• image-automation-controller: v0.40.0
• image-reflector-controller: v0.34.0

Changes in this version include:

• Addressed security vulnerabilities in the fluxconfig-agent, fluxconfig-controller and fluent-bit-mdm by updating the Go packages.
• Can now specify tenant ID when enabling workload identity in Arc-enabled Kubernetes clusters and AKS clusters.
• Support for image-automation controller in workload identity in Arc-enabled Kubernetes clusters and AKS clusters.

Breaking changes:

• Semantic versioning changes in source controller (see note above)

Flux version: Release v2.4.0

• source-controller: v1.4.1
• kustomize-controller: v1.4.0
• helm-controller: v1.1.0
• notification-controller: v1.4.0
• image-automation-controller: v0.39.0
• image-reflector-controller: v0.33.0

Changes in this version include:

• Simplified flux extension's kubelet identity configuration by removing the obsolete tenant-id.

Flux version: Release v2.4.0

• source-controller: v1.4.1
• kustomize-controller: v1.4.0
• helm-controller: v1.1.0
• notification-controller: v1.4.0
• image-automation-controller: v0.39.0
• image-reflector-controller: v0.33.0

Changes in this version include:

• Addressed security vulnerabilities in the fluxconfig-agent, fluxconfig-controller and fluent-bit-mdm by updating the Go packages.
• Support of workload identity in Arc-enabled clusters. For more information, see Workload identity in Arc-enabled Kubernetes clusters and AKS clusters.

• Read more about cluster extensions for Azure Arc-enabled Kubernetes.
• Learn how to deploy extensions to an Azure Arc-enabled Kubernetes cluster.