Azure Kubernetes Fleet Manager scheduler and scheduling framework

  • Article

This article provides a conceptual overview of the scheduler and scheduling framework in Azure Kubernetes Fleet Manager (Fleet).

What is the scheduler?

The scheduler is a core component in the fleet workload with the primary responsibility of determining scheduling decisions for a bundle of resources based on the latest ClusterSchedulingPolicySnapshot generated by the ClusterResourcePlacement.

By default, the scheduler operates in batch mode, which enhances performance. In this mode, it binds a ClusterResourceBinding from a ClusterResourcePlacement to multiple clusters whenever possible.

Batch mode

Scheduling resources within a ClusterResourcePlacement involves more dependencies compared to scheduling pods within a Kubernetes Deployment. There are two notable distinctions:

  • In a ClusterResourcePlacement, multiple replicas of resources can't be scheduled on the same cluster.
  • The ClusterResourcePlacement supports different placement types within a single object.

For more information, see the upstream Fleet Scheduler documentation.

What is the scheduling framework?

The fleet scheduling framework closely aligns with the native Kubernetes scheduling framework, incorporating several modifications and tailored functionalities to support the fleet workload.

This screenshot shows an overview diagram of the fleet scheduling framework.

The primary advantage of this framework is its capability to compile plugins directly into the scheduler. Its API facilitates the implementation of diverse scheduling features as plugins, ensuring a lightweight and maintainable core.

The fleet scheduler integrates the following fundamental built-in plugins:

  • Topology spread plugin: Supports the TopologySpreadConstraints in the placement policy.
  • Cluster affinity plugin: Facilitates the affinity clause in the placement policy.
  • Same placement affinity plugin: Designed specifically for fleet and prevents multiple replicas from being placed within the same cluster.
  • Cluster eligibility plugin: Enables cluster selection based on specific status criteria.
  • Taint & toleration plugin: Enables cluster selection based on taints on the cluster and tolerations on the ClusterResourcePlacement.

For more information, see the upstream Fleet Scheduling Framework documentation.

Next steps