To keep applications running smoothly, Kubernetes clusters frequently include several master nodes for improved resilience and minimal downtime. Yet many people wonder how leadership functions when more than one master is present. This article explores how various control plane components manage leadership within a multi-master Kubernetes environment.
Understanding Kubernetes Masters and Control Plane
In Kubernetes, people now refer to what were traditionally called “masters” as control plane nodes. These nodes manage cluster state, perform scheduling, and handle coordination tasks. Deploying multiple masters improves redundancy and reliability, but the system still needs to determine leadership responsibilities internally.
Visualising Leadership in a Kubernetes Cluster
To help you understand how Kubernetes distributes leadership in a cluster with multiple masters, look at the diagram below. It shows how control plane components interact, how they elect leaders, and why no single master node acts as the overall leader.
Kubernetes deploys multiple masters (control plane nodes) to achieve high availability. Each master runs critical components:
- kube-API server: Handles client requests (kubectl, users, applications). All API servers run in active-active mode and sit behind a load balancer, so there’s no single API server leader.
- etcd: A distributed key-value store that saves all cluster state. In an etcd cluster, one node always becomes the leader and handles write operations and coordination for data consistency through the Raft algorithm.
- kube-controller-manager (kube-cm): Although this runs on all masters, only one becomes the leader to perform tasks like managing replicas, node monitoring, and more.
- kube-scheduler: Also runs on all masters but elects one leader to assign Pods to nodes efficiently.
Worker nodes connect to the masters for API requests, updates, and workload management. Leadership ensures that:
- etcd remains consistent
- Controllers don’t duplicate work
- The cluster stays healthy even if a master fails
This architecture allows Kubernetes clusters to tolerate failures while maintaining consistent operations.
Leadership in etcd
The etcd database is crucial in Kubernetes for storing cluster state and configuration data.
- etcd uses a distributed architecture where one node functions as the leader, and the other nodes act as followers.
- The Raft consensus algorithm dynamically elects leaders in etcd, ensuring data consistency and fault tolerance.
- Any etcd node can serve read requests, but only the leader handles write operations.
Hence, even with multiple master nodes, only one etcd leader exists at any given time.
Leadership in kube-controller-manager and kube-scheduler
Kubernetes runs several critical components in multiple instances across control plane nodes. However, the system maintains active leadership in the following ways:
- All masters run the kube-controller-manager. Still, through leader election, only one instance remains active while the others stand by as backups.
- The kube-scheduler follows a similar process, where one instance acts as the leader and performs scheduling tasks, while the others remain on standby to take over if needed.
Kubernetes runs several critical components in multiple instances across control plane nodes. However, the system maintains active leadership in the following ways:
kube-apiserver: No Single Leader
Unlike etcd and other components, the kube-apiserver operates differently:
- All API server instances run in an active-active mode.
- Kubernetes doesn’t designate any single API server as the leader.
- A load balancer typically routes requests from clients or kubectl and distributes traffic evenly among all API server instances.
- This approach improves performance and provides resilience, as clients are not dependent on a single API server instance.
Leadership is Component-Based
It should be noted that there’s no single leader across all masters. Instead, leadership responsibilities are distributed:
- etcd has one leader.
- kube-controller-manager and kube-scheduler each elect one leader.
- kube-apiserver runs active-active without a leader.
Leadership changes dynamically if failures occur, keeping the cluster operational and healthy.
How to Check Leadership in Kubernetes
Administrators may wish to verify which node is serving as the leader for various components:
For etcd, commands like:
etcdctl endpoint status --write-out=table
can be used to check the current leader.
For controller-manager or scheduler, logs can be reviewed:
kubectl logs -n kube-system <controller-manager-pod-name>
Look for messages such as:
leaderelection.go:XXX] became leader
Knowing where leadership resides helps in troubleshooting and maintaining cluster health.
Conclusion
In Kubernetes clusters with multiple masters, leadership isn’t held by one node overall. Instead, each control plane component manages its own leader election or operates in active-active mode. This distributed leadership ensures high availability, fault tolerance, and smooth cluster operations.
Understanding this architecture is crucial for anyone maintaining production-grade Kubernetes environments.
Other docs:
Check other blogs on Kubernetes – click here
Check out official documentation for coordinated leader election.