Blog HCP Consul on Azure goes GA, plus more Consul news from HashiConf EU Read more
  • Overview
    • Consul on Kubernetes
    • Control access with Consul API Gateway
    • Discover Services with Consul
    • Enforce Zero Trust Networking with Consul
    • Load Balancing with Consul
    • Manage Traffic with Consul
    • Multi-Platform Service Mesh with Consul
    • Network Infrastructure Automation with Consul
    • Observability with Consul
  • Enterprise
  • Tutorials
  • Docs
  • API
  • CLI
  • Community
GitHub
Download
Try HCP Consul
    • v1.12.x (latest)
    • v1.11.x
    • v1.10.x
    • v1.9.x
    • v1.8.x
    • Overview
      • Overview
      • What is a Service Mesh?
      • Overview
      • Chef, Puppet, etc.
      • Nagios
      • SkyDNS
      • SmartStack
      • Serf
      • Eureka
      • Istio
      • Envoy and Other Proxies
      • Custom Solutions
    • Overview
    • Manual Bootstrap
    • Consul Agent
    • Glossary
    • Required Ports
    • Bootstrapping a Datacenter
    • Cloud Auto-join
    • Server Performance
    • Kubernetes
  • API
  • Commands (CLI)
    • Register Services - Service Definitions
    • Find Services - DNS Interface
    • Monitor Services - Check Definitions
    • Overview
    • How Service Mesh Works
    • Configuration
      • Overview
      • Ingress Gateway
      • Mesh
      • Exported Services
      • Proxy Defaults
      • Service Defaults
      • Service Intentions
      • Service Resolver
      • Service Router
      • Service Splitter
      • Terminating Gateway
      • Overview
      • Envoy
      • Built-in Proxy
      • Proxy Integration
      • Managed (Deprecated)
      • Overview
      • Proxy Service Registration
      • Sidecar Service Registration
    • Service-to-service permissions - Intentions
    • Service-to-service permissions - Intentions (Legacy Mode)
    • Transparent Proxy
      • Overview
      • UI Visualization
      • Overview
      • Discovery Chain
    • Connectivity Tasks
    • Distributed Tracing
      • Overview
        • WAN Federation
        • Enabling Service-to-service Traffic Across Datacenters
        • Enabling Service-to-service Traffic Across Admin Partitions
      • Ingress Gateways
      • Terminating Gateways
      • What is Cluster Peering
      • Create and Manage Peering Connections
      • Cluster Peering on Kubernetes
    • Nomad
    • Kubernetes
      • Overview
      • Go Integration
      • Overview
      • Built-In CA
      • Vault
      • ACM Private CA
    • Develop and Debug
    • Security
    • Overview
    • Installation
    • Technical Specifications
    • Common Errors
    • Upgrades
    • Overview
    • Architecture
      • Installing Consul on Kubernetes
      • Installing Consul K8s CLI
        • Minikube
        • Kind
        • AKS (Azure)
        • EKS (AWS)
        • GKE (Google Cloud)
        • Red Hat OpenShift
        • Self Hosted Kubernetes
        • Consul Clients Outside Kubernetes
        • Consul Servers Outside Kubernetes
        • Single Consul Datacenter in Multiple Kubernetes Clusters
        • Consul Enterprise
        • Overview
        • Federation Between Kubernetes Clusters
        • Federation Between VMs and Kubernetes
        • Overview
        • Systems Integration
          • Overview
          • Bootstrap Token
          • Enterprise License
          • Gossip Encryption Key
          • Partition Token
          • Replication Token
          • Server TLS
          • Service Mesh Certificates
          • Snapshot Agent Config
          • Webhook Certificates
        • WAN Federation
      • Overview
      • Transparent Proxy
      • Ingress Gateways
      • Terminating Gateways
      • Ingress Controllers
      • Configuring a Connect CA Provider
      • Health Checks
        • Metrics
    • Service Sync
      • Overview
      • Upgrade An Existing Cluster to CRDs
    • Annotations and Labels
    • Consul DNS
      • Upgrading Consul on Kubernetes
      • Upgrading Consul K8s CLI
      • Uninstall
      • Certificate Rotation
      • Gossip Encryption Key Rotation
      • Configure TLS on an Existing Cluster
      • Common Error Messages
      • FAQ
    • Compatibility Matrix
    • Helm Chart Configuration
    • Consul K8s CLI Reference
    • Overview
    • Requirements
    • Task Resource Usage
      • Installation
      • Secure Configuration
      • Migrate Existing Tasks
      • Installation
      • Secure Configuration
      • ACL Controller
    • Architecture
    • Consul Enterprise
    • Configuration Reference
    • Overview
    • Register Lambda Functions
    • Invoke Lambda Functions
    • Overview
      • Installation
      • Requirements
      • Configure
      • Run Consul-Terraform-Sync
    • Architecture
      • Overview
      • Status
      • Tasks
      • Health
      • Overview
      • task
      • start
    • Configuration
    • Tasks
    • Terraform Modules
      • Overview
      • License
      • Terraform Cloud Driver
      • Overview
      • Terraform
      • Terraform Cloud
    • Compatibility
    • Consul KV
    • Sessions
    • Watches
    • Overview
      • General
      • CLI Reference
      • Configuration Reference
    • Configuration Entries
    • Telemetry
    • Sentinel
    • RPC
    • Overview
      • ACL System Overview
      • Tokens
      • Policies
      • Roles
      • Rules Reference
      • Legacy Mode
      • Token Migration
      • ACLs in Federated Datacenters
        • Overview
        • Kubernetes
        • JWT
        • OIDC
        • AWS IAM
    • Encryption
      • Overview
      • Core
      • Network Infrastructure Automation
    • Overview
    • Admin Partitions
    • Audit Logging
    • Automated Backups
    • Automated Upgrades
    • Enhanced Read Scalability
    • Single sign-on - OIDC
    • Redundancy Zones
    • Advanced Federation
    • Network Segments
    • Namespaces
    • NIA with TFE
    • Sentinel
      • Overview
      • FAQ
    • Overview
    • Improving Consul Resilience
    • Anti-Entropy
    • Consensus Protocol
    • Gossip Protocol
    • Jepsen Testing
    • Network Coordinates
    • Consul Integration Program
    • NIA Integration Program
    • Vault Integration
    • Proxy Integration
  • Consul Tools
    • Overview
    • Compatibility Promise
    • Specific Version Details
      • Overview
      • General Process
      • Upgrading to 1.2.4
      • Upgrading to 1.6.9
      • Upgrading to 1.8.13
      • Upgrading to 1.10.0
    • Common Error Messages
    • FAQ
    • Overview
      • v1.11.x
      • v1.10.x
      • v1.9.x
      • v0.3.x
      • v0.2.x
      • v0.1.x
      • v0.4.x
      • v0.3.x
      • v0.2.x
      • v0.6.x
      • v0.5.x
    • Overview
    • ACL
  • Guides
Type '/' to Search

»Consul Servers Outside of Kubernetes

If you have a Consul cluster already running, you can configure your Consul clients inside Kubernetes to join this existing cluster.

The below config.yaml file shows how to configure the Helm chart to install Consul clients that will join an existing cluster.

The global.enabled value first disables all chart components by default so that each component is opt-in. This allows us to only setup the client agents. We then opt-in to the client agents by setting client.enabled to true.

Next, client.exposeGossipPorts can be set to true or false depending on if you want the clients to be exposed on the Kubernetes internal node IPs (true) or their pod IPs (false).

Finally, client.join is set to an array of valid -retry-join values. In the example above, a fake cloud auto-join value is specified. This should be set to resolve to the proper addresses of your existing Consul cluster.

global:
  enabled: false

client:
  enabled: true
  # Set this to true to expose the Consul clients using the Kubernetes node
  # IPs. If false, the pod IPs must be routable from the external servers.
  exposeGossipPorts: true
  join:
    - 'provider=my-cloud config=val ...'
config.yaml
global:
  enabled: false

client:
  enabled: true
  # Set this to true to expose the Consul clients using the Kubernetes node
  # IPs. If false, the pod IPs must be routable from the external servers.
  exposeGossipPorts: true
  join:
    - 'provider=my-cloud config=val ...'

Networking: Note that for the Kubernetes nodes to join an existing cluster, the nodes (and specifically the agent pods) must be able to connect to all other server and client agents inside and outside of Kubernetes over LAN. If this isn't possible, consider running a separate Consul cluster inside Kubernetes and federating it with your cluster outside Kubernetes. You may also consider adopting Consul Enterprise for network segments.

»Configuring TLS with Auto-encrypt

Note: Consul on Kubernetes currently does not support external servers that require mutual authentication for the HTTPS clients of the Consul servers, that is when servers have either tls.defaults.verify_incoming or tls.https.verify_incoming set to true. As noted in the Security Model, that setting isn't strictly necessary to support Consul's threat model as it is recommended that all requests contain a valid ACL token.

Consul's auto-encrypt feature allows clients to automatically provision their certificates by making a request to the servers at startup. If you would like to use this feature with external Consul servers, you need to configure the Helm chart with information about the servers so that it can retrieve the clients' CA to use for securing the rest of the cluster. To do that, you must add the following values, in addition to the values mentioned above:

global:
  tls:
    enabled: true
    enableAutoEncrypt: true
externalServers:
  enabled: true
  hosts:
    - 'provider=my-cloud config=val ...'
config.yaml
global:
  tls:
    enabled: true
    enableAutoEncrypt: true
externalServers:
  enabled: true
  hosts:
    - 'provider=my-cloud config=val ...'

In most cases, externalServers.hosts will be the same as client.join, however, both keys must be set because they are used for different purposes: one for Serf LAN and the other for HTTPS connections. Please see the reference documentation for more info. If your HTTPS port is different from Consul's default 8501, you must also set externalServers.httpsPort.

»Configuring ACLs

If you are running external servers with ACLs enabled, there are a couple of ways to configure the Helm chart to help initialize ACL tokens for Consul clients and consul-k8s components for you.

»Manually Bootstrapping ACLs

If you would like to call the ACL bootstrapping API yourself or if your cluster has already been bootstrapped with ACLs, you can provide the bootstrap token to the Helm chart. The Helm chart will then use this token to configure ACLs for Consul clients and any consul-k8s components you are enabling.

First, create a Kubernetes secret containing your bootstrap token:

kubectl create secret generic bootstrap-token --from-literal='token=<your bootstrap token>'
kubectl create secret generic bootstrap-token --from-literal='token=<your bootstrap token>'

Then provide that secret to the Helm chart:

global:
  acls:
    manageSystemACLs: true
    bootstrapToken:
      secretName: bootstrap-token
      secretKey: token
config.yaml
global:
  acls:
    manageSystemACLs: true
    bootstrapToken:
      secretName: bootstrap-token
      secretKey: token

The bootstrap token requires the following minimal permissions:

  • acl:write
  • operator:write if enabling Consul namespaces
  • agent:read if using WAN federation over mesh gateways

Next, configure external servers. The Helm chart will use this configuration to talk to the Consul server's API to create policies, tokens, and an auth method. If you are enabling Consul Connect, k8sAuthMethodHost should be set to the address of your Kubernetes API server so that the Consul servers can validate a Kubernetes service account token when using the Kubernetes auth method with consul login.

externalServers:
  enabled: true
  hosts:
    - 'provider=my-cloud config=val ...'
  k8sAuthMethodHost: 'https://kubernetes.example.com:443'
config.yaml
externalServers:
  enabled: true
  hosts:
    - 'provider=my-cloud config=val ...'
  k8sAuthMethodHost: 'https://kubernetes.example.com:443'

Your resulting Helm configuration will end up looking similar to this:

global:
  enabled: false
  acls:
    manageSystemACLs: true
    bootstrapToken:
      secretName: bootstrap-token
      secretKey: token
client:
  enabled: true
  # Set this to true to expose the Consul clients using the Kubernetes node
  # IPs. If false, the pod IPs must be routable from the external servers.
  exposeGossipPorts: true
  join:
    - 'provider=my-cloud config=val ...'
externalServers:
  enabled: true
  hosts:
    - 'provider=my-cloud config=val ...'
  k8sAuthMethodHost: 'https://kubernetes.example.com:443'
config.yaml
global:
  enabled: false
  acls:
    manageSystemACLs: true
    bootstrapToken:
      secretName: bootstrap-token
      secretKey: token
client:
  enabled: true
  # Set this to true to expose the Consul clients using the Kubernetes node
  # IPs. If false, the pod IPs must be routable from the external servers.
  exposeGossipPorts: true
  join:
    - 'provider=my-cloud config=val ...'
externalServers:
  enabled: true
  hosts:
    - 'provider=my-cloud config=val ...'
  k8sAuthMethodHost: 'https://kubernetes.example.com:443'

»Bootstrapping ACLs via the Helm chart

If you would like the Helm chart to call the bootstrapping API and set the server tokens for you, then the steps are similar. The only difference is that you don't need to set the bootstrap token. The Helm chart will save the bootstrap token as a Kubernetes secret.

global:
  enabled: false
  acls:
    manageSystemACLs: true
client:
  enabled: true
  # Set this to true to expose the Consul clients using the Kubernetes node
  # IPs. If false, the pod IPs must be routable from the external servers.
  exposeGossipPorts: true
  join:
    - 'provider=my-cloud config=val ...'
externalServers:
  enabled: true
  hosts:
    - 'provider=my-cloud config=val ...'
  k8sAuthMethodHost: 'https://kubernetes.example.com:443'
config.yaml
global:
  enabled: false
  acls:
    manageSystemACLs: true
client:
  enabled: true
  # Set this to true to expose the Consul clients using the Kubernetes node
  # IPs. If false, the pod IPs must be routable from the external servers.
  exposeGossipPorts: true
  join:
    - 'provider=my-cloud config=val ...'
externalServers:
  enabled: true
  hosts:
    - 'provider=my-cloud config=val ...'
  k8sAuthMethodHost: 'https://kubernetes.example.com:443'
github logoEdit this page
IntroGuidesDocsCommunityPrivacySecurityBrandConsent Manager