Blog HCP Consul on Azure goes GA, plus more Consul news from HashiConf EU Read more
  • Overview
    • Consul on Kubernetes
    • Control access with Consul API Gateway
    • Discover Services with Consul
    • Enforce Zero Trust Networking with Consul
    • Load Balancing with Consul
    • Manage Traffic with Consul
    • Multi-Platform Service Mesh with Consul
    • Network Infrastructure Automation with Consul
    • Observability with Consul
  • Enterprise
  • Tutorials
  • Docs
  • API
  • CLI
  • Community
GitHub
Download
Try HCP Consul
    • v1.12.x (latest)
    • v1.11.x
    • v1.10.x
    • v1.9.x
    • v1.8.x
    • Overview
      • Overview
      • What is a Service Mesh?
      • Overview
      • Chef, Puppet, etc.
      • Nagios
      • SkyDNS
      • SmartStack
      • Serf
      • Eureka
      • Istio
      • Envoy and Other Proxies
      • Custom Solutions
    • Overview
    • Manual Bootstrap
    • Consul Agent
    • Glossary
    • Required Ports
    • Bootstrapping a Datacenter
    • Cloud Auto-join
    • Server Performance
    • Kubernetes
  • API
  • Commands (CLI)
    • Register Services - Service Definitions
    • Find Services - DNS Interface
    • Monitor Services - Check Definitions
    • Overview
    • How Service Mesh Works
    • Configuration
      • Overview
      • Ingress Gateway
      • Mesh
      • Exported Services
      • Proxy Defaults
      • Service Defaults
      • Service Intentions
      • Service Resolver
      • Service Router
      • Service Splitter
      • Terminating Gateway
      • Overview
      • Envoy
      • Built-in Proxy
      • Proxy Integration
      • Managed (Deprecated)
      • Overview
      • Proxy Service Registration
      • Sidecar Service Registration
    • Service-to-service permissions - Intentions
    • Service-to-service permissions - Intentions (Legacy Mode)
    • Transparent Proxy
      • Overview
      • UI Visualization
      • Overview
      • Discovery Chain
    • Connectivity Tasks
    • Distributed Tracing
      • Overview
        • WAN Federation
        • Enabling Service-to-service Traffic Across Datacenters
        • Enabling Service-to-service Traffic Across Admin Partitions
      • Ingress Gateways
      • Terminating Gateways
      • What is Cluster Peering
      • Create and Manage Peering Connections
      • Cluster Peering on Kubernetes
    • Nomad
    • Kubernetes
      • Overview
      • Go Integration
      • Overview
      • Built-In CA
      • Vault
      • ACM Private CA
    • Develop and Debug
    • Security
    • Overview
    • Installation
    • Technical Specifications
    • Common Errors
    • Upgrades
    • Overview
    • Architecture
      • Installing Consul on Kubernetes
      • Installing Consul K8s CLI
        • Minikube
        • Kind
        • AKS (Azure)
        • EKS (AWS)
        • GKE (Google Cloud)
        • Red Hat OpenShift
        • Self Hosted Kubernetes
        • Consul Clients Outside Kubernetes
        • Consul Servers Outside Kubernetes
        • Single Consul Datacenter in Multiple Kubernetes Clusters
        • Consul Enterprise
        • Overview
        • Federation Between Kubernetes Clusters
        • Federation Between VMs and Kubernetes
        • Overview
        • Systems Integration
          • Overview
          • Bootstrap Token
          • Enterprise License
          • Gossip Encryption Key
          • Partition Token
          • Replication Token
          • Server TLS
          • Service Mesh Certificates
          • Snapshot Agent Config
          • Webhook Certificates
        • WAN Federation
      • Overview
      • Transparent Proxy
      • Ingress Gateways
      • Terminating Gateways
      • Ingress Controllers
      • Configuring a Connect CA Provider
      • Health Checks
        • Metrics
    • Service Sync
      • Overview
      • Upgrade An Existing Cluster to CRDs
    • Annotations and Labels
    • Consul DNS
      • Upgrading Consul on Kubernetes
      • Upgrading Consul K8s CLI
      • Uninstall
      • Certificate Rotation
      • Gossip Encryption Key Rotation
      • Configure TLS on an Existing Cluster
      • Common Error Messages
      • FAQ
    • Compatibility Matrix
    • Helm Chart Configuration
    • Consul K8s CLI Reference
    • Overview
    • Requirements
    • Task Resource Usage
      • Installation
      • Secure Configuration
      • Migrate Existing Tasks
      • Installation
      • Secure Configuration
      • ACL Controller
    • Architecture
    • Consul Enterprise
    • Configuration Reference
    • Overview
    • Register Lambda Functions
    • Invoke Lambda Functions
    • Overview
      • Installation
      • Requirements
      • Configure
      • Run Consul-Terraform-Sync
    • Architecture
      • Overview
      • Status
      • Tasks
      • Health
      • Overview
      • task
      • start
    • Configuration
    • Tasks
    • Terraform Modules
      • Overview
      • License
      • Terraform Cloud Driver
      • Overview
      • Terraform
      • Terraform Cloud
    • Compatibility
    • Consul KV
    • Sessions
    • Watches
    • Overview
      • General
      • CLI Reference
      • Configuration Reference
    • Configuration Entries
    • Telemetry
    • Sentinel
    • RPC
    • Overview
      • ACL System Overview
      • Tokens
      • Policies
      • Roles
      • Rules Reference
      • Legacy Mode
      • Token Migration
      • ACLs in Federated Datacenters
        • Overview
        • Kubernetes
        • JWT
        • OIDC
        • AWS IAM
    • Encryption
      • Overview
      • Core
      • Network Infrastructure Automation
    • Overview
    • Admin Partitions
    • Audit Logging
    • Automated Backups
    • Automated Upgrades
    • Enhanced Read Scalability
    • Single sign-on - OIDC
    • Redundancy Zones
    • Advanced Federation
    • Network Segments
    • Namespaces
    • NIA with TFE
    • Sentinel
      • Overview
      • FAQ
    • Overview
    • Improving Consul Resilience
    • Anti-Entropy
    • Consensus Protocol
    • Gossip Protocol
    • Jepsen Testing
    • Network Coordinates
    • Consul Integration Program
    • NIA Integration Program
    • Vault Integration
    • Proxy Integration
  • Consul Tools
    • Overview
    • Compatibility Promise
    • Specific Version Details
      • Overview
      • General Process
      • Upgrading to 1.2.4
      • Upgrading to 1.6.9
      • Upgrading to 1.8.13
      • Upgrading to 1.10.0
    • Common Error Messages
    • FAQ
    • Overview
      • v1.11.x
      • v1.10.x
      • v1.9.x
      • v0.3.x
      • v0.2.x
      • v0.1.x
      • v0.4.x
      • v0.3.x
      • v0.2.x
      • v0.6.x
      • v0.5.x
    • Overview
    • ACL
  • Guides
Type '/' to Search

»Cluster Peering on Kubernetes

Cluster peering is currently in technical preview: Functionality associated with cluster peering is subject to change. You should never use the technical preview release in secure environments or production scenarios. Features in technical preview may have performance issues, scaling issues, and limited support.

To establish a cluster peering connection on Kubernetes, you need to enable the feature in the Helm chart and create custom resource definitions for each side of the peering.

The following Custom Resource Definitions (CRDs) are used to create and manage a peering connection:

  • PeeringAcceptor: Generates a peering token and accepts an incoming peering connection.
  • PeeringDialer: Uses a peering token to make an outbound peering connection with the cluster that generated the token.

»Prerequisites

You must implement the following requirements to create and use cluster peering connections with Kubernetes:

  • Consul 1.13 Alpha 2 or later
  • At least two Kubernetes clusters
  • The Kubernetes clusters must be running in a flat network
  • The network must be running on Consul on Kubernetes v.0.45 or later

»Helm chart configuration

To establish cluster peering through Kubernetes, deploy clusters with the following Helm values.

global:
  image: "hashicorp/consul:1.13.0-alpha2"
  peering:
    enabled: true
connectInject:
  enabled: true
meshGateway:
  enabled: true
  replicas: 1
values.yaml
global:
  image: "hashicorp/consul:1.13.0-alpha2"
  peering:
    enabled: true
connectInject:
  enabled: true
meshGateway:
  enabled: true
  replicas: 1

Install Consul on Kubernetes on each Kubernetes cluster by applying values.yaml using the Helm CLI.

$ export HELM_RELEASE_NAME=cluster-name
$ export HELM_RELEASE_NAME=cluster-name
$ helm install ${HELM_RELEASE_NAME} hashicorp/consul --version "0.45.0" --values values.yaml
$ helm install ${HELM_RELEASE_NAME} hashicorp/consul --version "0.45.0" --values values.yaml

»Create a peering connection

To peer Kubernetes clusters running Consul, you need to create a peering token and share it with the other cluster.

  1. In cluster-01, create the PeeringAcceptor custom resource.

    apiVersion: consul.hashicorp.com/v1alpha1
    kind: PeeringAcceptor
    metadata:
      name: cluster-02 ## The name of the peer you want to connect to
    spec:
      peer:
        secret:
          name: "peering-token"
          key: "data"
          backend: "kubernetes"
    
    acceptor.yml
    apiVersion: consul.hashicorp.com/v1alpha1
    kind: PeeringAcceptor
    metadata:
      name: cluster-02 ## The name of the peer you want to connect to
    spec:
      peer:
        secret:
          name: "peering-token"
          key: "data"
          backend: "kubernetes"
    
  2. Apply the PeeringAcceptor resource to the first cluster.

    $ kubectl apply --filename acceptor.yml
    
    $ kubectl apply --filename acceptor.yml
    
  3. Save your peering token so that you can export it to the other cluster.

    $ kubectl get secret peering-token --output yaml > peering-token.yml
    
    $ kubectl get secret peering-token --output yaml > peering-token.yml
    
  4. Apply the peering token to the second cluster.

    $ kubectl apply --filename peering-token.yml
    
    $ kubectl apply --filename peering-token.yml
    
  5. In “cluster-02,” create the PeeringDialer custom resource.

    apiVersion: consul.hashicorp.com/v1alpha1
    kind: PeeringDialer
    metadata:
      name: cluster-01 ## The name of the peer you want to connect to
    spec:
      peer:
        secret:
          name: "peering-token"
          key: "data"
          backend: "kubernetes"
    
    dialer.yml
    apiVersion: consul.hashicorp.com/v1alpha1
    kind: PeeringDialer
    metadata:
      name: cluster-01 ## The name of the peer you want to connect to
    spec:
      peer:
        secret:
          name: "peering-token"
          key: "data"
          backend: "kubernetes"
    
  6. Apply the PeeringDialer resource to the second cluster.

    $ kubectl apply --filename dialer.yml
    
    $ kubectl apply --filename dialer.yml
    

»Deploy and export cluster services

  1. For the service in “cluster-02” that you want to export, add the following annotations to your service's pods. This service is referred to as "backend-service" in the following steps.

    ##…
    annotations:
      "consul.hashicorp.com/connect-inject": "true"
      "consul.hashicorp.com/transparent-proxy": "false"
    ##…
    
    backend-service.yml
    ##…
    annotations:
      "consul.hashicorp.com/connect-inject": "true"
      "consul.hashicorp.com/transparent-proxy": "false"
    ##…
    
  2. In “cluster-02,” create an ExportedServices custom resource.

    apiVersion: consul.hashicorp.com/v1alpha1
    kind: ExportedServices
    metadata:
      name: default ## The name of the partition containing the service
    spec:
      services:
        name: backend-service ## The name of the service you want to export
        consumers:
          peerName: cluster-01 ## The name of the peer that receives the service
    
    exportedsvc.yml
    apiVersion: consul.hashicorp.com/v1alpha1
    kind: ExportedServices
    metadata:
      name: default ## The name of the partition containing the service
    spec:
      services:
        name: backend-service ## The name of the service you want to export
        consumers:
          peerName: cluster-01 ## The name of the peer that receives the service
    
  3. Create service intentions for the second cluster.

    apiVersion: consul.hashicorp.com/v1alpha1
    kind: ServiceIntentions
    metadata:
      name: backend-deny
    spec:
      destination:
        name: backend-service
      sources:
       - name: "*"
         action: deny
       - name: frontend-service
         action: allow
    
    intention.yml
    apiVersion: consul.hashicorp.com/v1alpha1
    kind: ServiceIntentions
    metadata:
      name: backend-deny
    spec:
      destination:
        name: backend-service
      sources:
       - name: "*"
         action: deny
       - name: frontend-service
         action: allow
    
  4. Apply the service file, the ExportedServices resource, and the intentions to the second cluster.

    $ kubectl apply --filename backend-service.yml --filename exportedsvc.yml --filename intention.yml
    
    $ kubectl apply --filename backend-service.yml --filename exportedsvc.yml --filename intention.yml
    
  5. To confirm that you peered your clusters, in “cluster-01,” query the /health HTTP endpoint.

    $ curl "localhost:8500/v1/health/connect/backend?peer=cluster-02"
    
    $ curl "localhost:8500/v1/health/connect/backend?peer=cluster-02"
    
  6. For the services in “cluster-01” that you want to access the “backend-service,” add the following annotations to the service file.

    ##…
    annotations:
      "consul.hashicorp.com/connect-inject": "true"
      "consul.hashicorp.com/transparent-proxy": "false"
      "consul.hashicorp.com/connect-service-upstreams": "backend-service.svc.cluster-02.peer:1234"
    ##…
    
    frontend-service.yml
    ##…
    annotations:
      "consul.hashicorp.com/connect-inject": "true"
      "consul.hashicorp.com/transparent-proxy": "false"
      "consul.hashicorp.com/connect-service-upstreams": "backend-service.svc.cluster-02.peer:1234"
    ##…
    
  7. Apply the service file to the first cluster.

    $ kubectl apply --filename frontend-service.yml
    
    $ kubectl apply --filename frontend-service.yml
    
  8. Run the following command and check the output to confirm that you peered your clusters successfully.

    $ curl localhost:1234
    {
        “name”: “backend-service”,
        ##…
        “body”: “Response from backend”,
        “code”: 200
    }
    
    $ curl localhost:1234
    {
        “name”: “backend-service”,
        ##…
        “body”: “Response from backend”,
        “code”: 200
    }
    

»End a peering connection

To end a peering connection, delete both the PeeringAcceptor and PeeringDialer resources.

To confirm that you deleted your peering connection, in “cluster-01,” query the /health HTTP endpoint. The peered services should no longer appear.

$ curl "localhost:8500/v1/health/connect/backend?peer=cluster-02"
$ curl "localhost:8500/v1/health/connect/backend?peer=cluster-02"
github logoEdit this page
IntroGuidesDocsCommunityPrivacySecurityBrandConsent Manager