Skip to main content

Multi-Cluster Federation

Key Takeaways for AI & Readers
  • Centralized Multi-Cluster Management: Multi-cluster federation allows managing several Kubernetes clusters as a unified logical unit for global deployments. Workloads are defined once and distributed to member clusters based on policies.
  • Enhanced Resilience: Distributing workloads across multiple clusters reduces the "blast radius" of outages. If one cluster fails completely (API server crash, cloud region outage), other clusters continue serving traffic independently.
  • Geographical Optimization: Federation enables compliance with data residency requirements (GDPR, HIPAA) and routes users to the nearest cluster for reduced latency, supporting multi-region and multi-cloud architectures.
  • Specialized Tools: Karmada handles workload distribution with PropagationPolicies, Cilium ClusterMesh provides cross-cluster pod-to-pod networking, Admiralty enables transparent overflow scheduling, and ArgoCD ApplicationSets deploy configurations across fleet members.
  • Complexity Trade-off: Multi-cluster architectures add significant operational complexity. Teams must weigh the benefits of isolation and resilience against the cost of managing cross-cluster networking, service discovery, secret synchronization, and coordinated upgrades.

As companies grow, they move from one large cluster to many smaller ones spread across regions and cloud providers. A single cluster has practical limits: etcd performance degrades beyond roughly 10,000 nodes, a misconfigured admission webhook can freeze all deployments, and a cloud region outage takes everything offline. Multi-cluster federation allows you to manage these clusters as a single logical entity while maintaining the isolation benefits of separate clusters.

1. Global Traffic Distribution​

The fundamental goal of federation is to deploy workloads across multiple clusters and route traffic intelligently. A single application definition is propagated to clusters in different regions, and a global load balancer directs users to the nearest healthy instance.

Global Load Balancer
app.global.io
🇺🇸
AWS us-east-1
🇪🇺
GCP europe-west1
Multi-cluster federation (via Karmada or ClusterMesh) allows you to define an app once and have Kubernetes automatically distribute it across multiple regions or clouds for disaster recovery and low latency.

This architecture provides resilience (one cluster can fail without user impact), compliance (data stays in the correct jurisdiction), and performance (users connect to the geographically closest cluster).

2. Why a Single Cluster Is Not Enough​

Blast Radius​

When a single cluster hosts all workloads, any cluster-level failure affects everything. An etcd corruption, a botched control plane upgrade, or a misconfigured network policy can bring down hundreds of services simultaneously. Multiple clusters limit the blast radius: a failure in the US-East cluster does not affect the EU-West cluster.

Compliance and Data Sovereignty​

Regulations like GDPR require that personal data of EU citizens remains within the EU. HIPAA imposes strict controls on healthcare data in the United States. A multi-cluster architecture places workloads and their data in the correct jurisdiction by design, rather than relying on complex in-cluster isolation.

Latency​

A single cluster in us-east-1 means that users in Tokyo experience 150-200ms of network latency on every request. Deploying the same service in an Asia-Pacific cluster reduces latency to under 20ms, dramatically improving user experience for latency-sensitive applications.

Organizational Boundaries​

Large organizations often have separate teams or business units that need independent cluster control. Federation allows each team to own their cluster while still enabling central visibility and shared policies.

3. Kubefed: The Legacy Approach​

Kubefed (Kubernetes Federation v2) was the original SIG-sponsored federation project. It introduced the concept of "federated resources" -- custom wrappers around standard Kubernetes objects that were distributed to member clusters. Kubefed is no longer actively maintained and is not recommended for new deployments. Its limitations included:

  • Complex federated resource types that duplicated every Kubernetes API
  • No support for cross-cluster service discovery without external tools
  • Difficulty handling heterogeneous clusters with different capabilities
  • Limited community adoption and maintenance

Understanding Kubefed's limitations explains why the ecosystem moved toward more focused, composable tools.

4. Modern Federation Tools​

Karmada​

Karmada is the leading CNCF incubating project for multi-cluster workload management. It uses a dedicated Karmada API server that accepts standard Kubernetes resources and distributes them to member clusters using PropagationPolicies.

# Define a standard Deployment (no federation-specific changes needed)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-server
  namespace: production
spec:
  replicas: 6
  selector:
    matchLabels:
      app: api-server
  template:
    metadata:
      labels:
        app: api-server
    spec:
      containers:
        - name: api
          image: myregistry.io/api-server:v2.4.1
          resources:
            requests:
              cpu: "500m"
              memory: "256Mi"
---
# PropagationPolicy distributes the Deployment across clusters
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: api-server-propagation
  namespace: production
spec:
  resourceSelectors:
    - apiVersion: apps/v1
      kind: Deployment
      name: api-server
  placement:
    clusterAffinity:
      clusterNames:                    # Target specific clusters
        - us-east-prod
        - eu-west-prod
        - ap-southeast-prod
    replicaScheduling:
      replicaDivisionPreference: Weighted
      replicaSchedulingType: Divided
      weightPreference:
        staticWeightList:
          - targetCluster:
              clusterNames: [us-east-prod]
            weight: 3                  # 3/6 replicas go to US
          - targetCluster:
              clusterNames: [eu-west-prod]
            weight: 2                  # 2/6 replicas go to EU
          - targetCluster:
              clusterNames: [ap-southeast-prod]
            weight: 1                  # 1/6 replicas go to Asia

Karmada's key strength is that workload authors write standard Kubernetes YAML. The federation layer is entirely in the PropagationPolicy, managed by platform teams.

Admiralty​

Admiralty takes a different approach: transparent multi-cluster scheduling. When a pod cannot be scheduled in the source cluster (due to resource pressure or explicit policy), Admiralty creates a "proxy pod" in the source cluster and schedules the actual pod in a target cluster. The application is unaware it has been moved.

This is ideal for burst-to-cloud scenarios: your on-premises cluster handles base load, and overflow is transparently scheduled into a cloud-based cluster.

Liqo​

Liqo creates virtual nodes that represent remote clusters. When you peer two Liqo-enabled clusters, each cluster gains a virtual node representing the other. Pods scheduled on the virtual node actually run in the remote cluster but appear local. Liqo handles cross-cluster networking transparently.

Clusternet​

Clusternet, a CNCF sandbox project, manages both cluster registration and workload distribution. It supports pull-based and push-based deployment models and integrates with Helm for deploying charts across clusters.

5. Cross-Cluster Networking​

Federation is only useful if services in different clusters can communicate. Several approaches exist:

Cilium ClusterMesh​

Cilium ClusterMesh creates a flat network across clusters where pods can communicate directly using their pod IPs. Each cluster runs Cilium as its CNI, and the ClusterMesh feature synchronizes pod identities and endpoints across clusters. This enables cross-cluster service discovery: a pod in Cluster A can call a Kubernetes Service that has backends in both Cluster A and Cluster B.

Submariner​

Submariner creates encrypted tunnels between clusters, connecting pod and service CIDRs across cluster boundaries. It works with any CNI plugin and supports cross-cluster service discovery through a Lighthouse DNS component.

Istio Multi-Cluster​

Istio can span multiple clusters, providing unified service mesh capabilities (mTLS, traffic management, observability) across all member clusters. This is powerful but adds significant complexity.

6. Multi-Cluster Service Discovery​

For services to communicate across clusters, DNS and service discovery must work globally.

# Kubernetes Multi-Cluster Services API (KEP-1645)
# Export a service to make it discoverable from other clusters
apiVersion: multicluster.x-k8s.io/v1alpha1
kind: ServiceExport
metadata:
  name: api-server
  namespace: production
---
# In another cluster, ServiceImport makes the service available
apiVersion: multicluster.x-k8s.io/v1alpha1
kind: ServiceImport
metadata:
  name: api-server
  namespace: production
spec:
  type: ClusterSetIP
  ports:
    - port: 80
      protocol: TCP

The Multi-Cluster Services API (MCS API) is a SIG-sponsored standard that multiple tools implement, including Cilium ClusterMesh, Submariner Lighthouse, and GKE Multi-Cluster Services.

7. Failover Patterns​

Active-Active​

All clusters actively serve traffic simultaneously. A global load balancer (AWS Global Accelerator, GCP Cloud Load Balancing, Cloudflare) distributes requests based on latency or geography. If one cluster becomes unhealthy, the load balancer automatically shifts traffic to remaining clusters.

This is the most resilient pattern but requires that all clusters have identical configurations and that your application handles distributed state correctly (or uses an external database).

Active-Passive​

One cluster serves all traffic while a standby cluster remains ready to take over. The passive cluster runs the same workloads but receives no user traffic until failover. This is simpler to reason about but wastes resources and introduces failover latency.

Active-Active with Regional Affinity​

A hybrid approach where each cluster serves its geographic region's traffic. Users in Europe hit the EU cluster, users in Asia hit the APAC cluster. If a regional cluster fails, its traffic is redirected to the nearest healthy cluster. This combines low latency with cross-region resilience.

8. Multi-Cluster GitOps with ArgoCD ApplicationSets​

ArgoCD ApplicationSets let you define a template that generates one ArgoCD Application per cluster, ensuring consistent deployments across your fleet.

# ArgoCD ApplicationSet: deploy monitoring stack to every cluster
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: monitoring-stack
  namespace: argocd
spec:
  generators:
    - clusters:
        selector:
          matchLabels:
            env: production            # Target all production clusters
  template:
    metadata:
      name: "monitoring-{{name}}"      # Generates: monitoring-us-east, etc.
    spec:
      project: platform
      source:
        repoURL: https://github.com/org/platform-configs
        targetRevision: main
        path: "clusters/{{name}}/monitoring"
      destination:
        server: "{{server}}"           # Each cluster's API server URL
        namespace: monitoring
      syncPolicy:
        automated:
          prune: true
          selfHeal: true

This pattern is the standard for fleet management: platform teams define the ApplicationSet once, and every cluster registered in ArgoCD automatically receives the configured applications.

9. Cost and Complexity Trade-offs​

Multi-cluster federation is not free. Before adopting it, consider these costs:

  • Infrastructure overhead: Each cluster has its own control plane (3 etcd nodes, API servers, controllers). Ten clusters means ten control planes.
  • Operational complexity: Upgrades, monitoring, certificate rotation, and backup procedures must be coordinated across clusters.
  • Networking costs: Cross-cluster traffic may traverse cloud provider networks, incurring egress charges. Encrypted tunnels add latency.
  • State synchronization: Databases, caches, and session state must be handled at the application or infrastructure level. Kubernetes federation does not solve distributed data.
  • Cognitive overhead: Engineers must understand which cluster their workloads run in, how failover works, and how to debug cross-cluster issues.

A common mistake is adopting multi-cluster federation prematurely. Many organizations can run a single well-managed cluster until they hit genuine scaling, compliance, or resilience requirements that demand multiple clusters.

Common Pitfalls​

  1. Inconsistent cluster configurations: If clusters drift (different Kubernetes versions, different CNI plugins, different admission policies), workloads that work in one cluster may fail in another. Use GitOps to enforce consistent configurations.
  2. Ignoring cross-cluster latency: Pod-to-pod calls across clusters add 5-50ms of latency depending on distance. Design services to be tolerant of this, or keep tightly coupled services in the same cluster.
  3. No global observability: Running separate Prometheus instances per cluster without aggregation (using Thanos or Cortex) means you have no fleet-wide view of health and performance.
  4. Split-brain scenarios: During network partitions, clusters may diverge. Ensure your failover logic handles partial connectivity gracefully and that global load balancers have appropriate health check thresholds.
  5. Over-reliance on a single control plane tool: If your Karmada control plane goes down, you can still interact with member clusters directly. Design your architecture so that member clusters are independently functional.

What's Next?​

  • Evaluate Karmada for multi-cluster workload distribution with fine-grained placement policies.
  • Deploy Cilium ClusterMesh for transparent cross-cluster networking and service discovery.
  • Implement ArgoCD ApplicationSets for fleet-wide GitOps configuration management.
  • Study the Multi-Cluster Services API (KEP-1645) for standardized cross-cluster service discovery.
  • Explore external DNS and global load balancers for intelligent traffic routing between clusters.
  • Review your compliance requirements to determine which data and workloads must be isolated in specific regions.