Choosing an Azure Kubernetes Fleet Manager option

This article provides an overview of the various Azure Kubernetes Fleet Manager options and the considerations you should use to guide your selection of a specific configuration.

A Fleet Manager resource can be created with or without a hub cluster. A hub cluster is a managed Azure Kubernetes Service (AKS) cluster that acts as a hub to stage Kubernetes resources ready for placement.

Let's compares the scenarios enabled by the using a hub cluster.

Without a hub cluster, Fleet Manager acts solely as a grouping entity in Azure Resource Manager (ARM). Certain features, such as Kubernetes and node image update runs and auto-upgrades, don't require a Kubernetes API and thus don't require a hub cluster. To take full advantage of all the features available, you need a Fleet Manager with a hub cluster.

For more information, see Create a Kubernetes Fleet resource without a hub cluster.

A Fleet Manager with a hub cluster has an associated AKS-managed cluster, which hosts the open source KubeFleet solution for workload placement and the fleet network manager for network management and load balancing.

Upon the creation of a Fleet Manager with a hub cluster, a hub AKS cluster is automatically created in the same Azure subscription under a managed resource group that begins with FL_. To improve reliability, hub clusters are denied any user-initiated mutations to the corresponding AKS cluster (under the Fleet-managed resource group FL_) and their underlying Azure resources (under the AKS-managed resource group MC_FL_*), such as virtual machines (VMs).

Control plane operations, such as changing the hub cluster's configuration through Azure Resource Manager (ARM) or deleting the cluster entirely, are denied. Data plane operations, such as connecting to the hub cluster's Kubernetes API server in order to configure workload placement, are allowed.

Hub clusters are exempted from Azure policies to avoid undesirable policy effects.

For a Fleet Manager with a hub cluster, there are two network access modes:

• Public hub clusters expose the hub cluster to the internet. This means that with the right credentials, anyone on the internet can connect to the hub cluster. This configuration can be useful during the development and testing phase, but represents a security concern, which is largely undesirable in production.

For more information, see Create a Kubernetes Fleet resource with a public hub cluster.

• Private hub clusters use a private AKS cluster as the hub, which prevents open access over the internet. All considerations for a private AKS cluster apply, so review the prerequisites and limitations to determine whether a Fleet Manager with a private hub cluster meets your needs.

Some other details to consider:

• Whether you choose a public or private hub, the type can't be changed after creation.
• When using an AKS private cluster, you have the ability to configure fully qualified domain names (FQDNs) and FQDN subdomains. This functionality doesn't apply to the private hub cluster of the Fleet Manager.
• When you connect to a private hub cluster, you can use the same methods that you would use to connect to any private AKS cluster. However, connecting using AKS command invoke and private endpoints aren't currently supported.
• When you use private hub clusters, you're required to specify the subnet in which the Fleet Manager hub cluster's node VMs reside. This process differs slightly from the AKS private cluster equivalent. For more information, see create a Fleet Manager with a private hub cluster.

Now that you understand the different types of Kubernetes fleet resources, see Create an Azure Kubernetes Fleet Manager resource and join member clusters.