From this page you can download various tools for Consul. These tools are
maintained by HashiCorp and the Consul Community.

Consul Tools

Welcome to the Consul documentation! This documentation is reference material for all available features and options of Consul.

Documentation

We expect Consul to run in large clusters of long-running agents. Because safely upgrading agents in this sort of environment relies heavily on backwards compatibility, we have a strong commitment to keeping different Consul versions protocol-compatible with each other.

Consul Protocol Compatibility Promise

Compatibility Promise

Consul is meant to be a long-running agent on any nodes participating in a Consul cluster. These nodes consistently communicate with each other. As such, protocol level compatibility and ease of upgrades is an important thing to keep in mind when using Consul.

Upgrade Consul

Upgrade

Specific versions of Consul may have additional information about the upgrade process beyond the standard flow.

Upgrading Specific Versions

Specific Version Details

General Upgrade Process

General Process

Upgrade Instructions

Upgrading to 1.2.4

Upgrading to 1.6.9

Upgrading to 1.8.4

Common Error Messages

Frequently Asked Questions

The Consul agent supports encrypting all of its network traffic. The exact method of encryption is described on the encryption internals page. There are two separate encryption systems, one for gossip traffic and one for RPC.

Encryption

Consul relies on both a lightweight gossip mechanism and an RPC system to provide various features. Both of the systems have different security mechanisms that stem from their designs. However, the security mechanisms of Consul have a common goal: to provide confidentiality, integrity, and authentication.

Security

Security model including requirements, recommendations, and threats for the core Consul product.

Consul Core Security Model

Core

Overview and links to various Consul security models.

Security Models

Security model including requirements, recommendations, and threats for Consul Network Infrastructure Automation (NIA).

Consul NIA Security Model

Network Infrastructure Automation <sup>Tech Preview</sup>

Consul provides an optional Access Control List (ACL) system which can be used to control access to data and APIs. The ACL system is a Capability-based system that relies on tokens which can have fine grained rules applied to them. It is very similar to AWS IAM in many ways.

ACL System (Legacy Mode)

Legacy Mode

Consul 1.4.0 introduces a new ACL system with improvements for the security and
management of ACL tokens and policies. This guide documents how to upgrade
existing (now called "legacy") tokens after upgrading to 1.4.0.

ACL Token Migration

Token Migration

ACL Rules

ACL System

Consul provides an optional Access Control List (ACL) system which can be used to control access to data and APIs. Select the following guide for your use case.

ACL Guides

Access Control (ACLs)

An auth method is a component in Consul that performs authentication against a trusted external party to authorize the creation of an ACL tokens usable within the local datacenter.

ACL Auth Methods

Auth Methods

The JWT auth method can be used to authenticate with Consul by providing a JWT directly. The JWT is cryptographically verified using locally-provided keys, or, if configured, an OIDC Discovery service can be used to fetch the appropriate keys.

JWT Auth Method

The Kubernetes auth method type allows for a Kubernetes service account token to be used to authenticate to Consul. This method of authentication makes it easy to introduce a Consul token into a Kubernetes pod.

Kubernetes Auth Method

Kubernetes

The OIDC auth method can be used to authenticate with Consul using OpenID Connect (OIDC). This method allows authentication via a configured OIDC provider using the user's web browser. This method may be initiated from the Consul UI or the command line.

OIDC Auth Method

OIDC

Consul-Terraform-Sync API

Architecture

Consul-Terraform-Sync CLI

Consul-Terraform-Sync requires a Terraform Provider, a Terraform Module and a running Consul Cluster outside of the consul-terraform-sync daemon.

Consul-Terraform-Sync Configuration

Configuration

Network Infrastructure Automation (NIA) is the concept of dynamically updating infrastructure devices triggered by service changes. Consul-Terraform-Sync is a tool that performs NIA and utilizes Consul as a data source that contains networking information about services and monitors those services. Terraform is used as the underlying automation tool and leverages the Terraform provider ecosystem to drive relevant changes to the network infrastructure.

Network Infrastructure Automation

Network Drivers

Tasks

A high level guide to configure Consul-Terraform-Sync.

Configure Consul-Terraform-Sync

Configure

Consul-Terraform-Sync is a daemon that runs alongside Consul. Consul-Terraform-Sync is not included with the Consul binary and will need to be installed separately.

Install Consul and Consul-Terraform-Sync

Installation

Consul-Terraform-Sync requires a Terraform Provider, a Terraform Module, and a running Consul cluster outside of the consul-terraform-sync daemon.

Requirements

Run Consul-Terraform-Sync

One of the primary query interfaces to Consul is the DNS interface. The Consul DNS interface can be exposed for all pods in Kubernetes using a stub-domain configuration.

Consul DNS - Kubernetes

Consul DNS

Helm Chart Configuration

Consul has many integrations with Kubernetes. You can deploy Consul to Kubernetes using the Helm chart, sync services between Consul and Kubernetes, automatically secure Pod communication with Connect, and more. This section documents the official integrations between Consul and Kubernetes.

The services in Kubernetes and Consul can be automatically synced so that Kubernetes services are available to Consul agents and services in Consul can be available as first-class Kubernetes services.

Service Sync - Kubernetes

Service Sync

Configure TLS on an existing Consul cluster running in Kubernetes

Configure TLS on an Existing Cluster

Uninstall

Compatibility Matrix for Consul Kubernetes and Consul

Compatibility Matrix

Consul can run directly on Kubernetes, both in server or client mode. For pure-Kubernetes workloads, this enables Consul to also exist purely within Kubernetes. For heterogeneous workloads, Consul agents can join a server running inside or outside of Kubernetes.

Install with Helm Chart - Kubernetes

Install with Helm Chart

Installing Consul on Self Hosted Kubernetes

Self Hosted Kubernetes

Installing on multiple Kubernetes clusters.

Multi-Cluster Federation Overview

Multi-Cluster Federation

Federation Between Kubernetes Clusters

Federation Between VMs and Kubernetes

Consul clients running on non-Kubernetes nodes can join a Consul cluster running within Kubernetes.

Consul Clients Outside of Kubernetes - Kubernetes

Consul Clients Outside Kubernetes

Configuration for running Consul Enterprise

Consul Enterprise

Running Consul servers outside of Kubernetes

Consul Servers Outside of Kubernetes - Kubernetes

Consul Servers Outside Kubernetes

Consul supports managing configuration entries via Kubernetes Custom Resources. These custom resource can be used to manage the configuration for workloads deployed within the cluster.

Consul Custom Resource Definitions

Custom Resource Definitions

Upgrade an existing cluster to use custom resources.

Upgrade An Existing Cluster to CRDs

Ambassador is a Kubernetes-native API gateway and ingress controller that
integrates well with Consul Connect.

Ambassador Integration - Kubernetes

Ambassador Integration

Configuring a Connect CA Provider

Kubernetes Health Checks

Health Checks

Connect is a feature built into to Consul that enables automatic service-to-service authorization and connection encryption across your Consul services. Connect can be used with Kubernetes to secure pod communication with other services.

Service Mesh - Kubernetes

Service Mesh

Configuring Ingress Gateways on Kubernetes

Ingress Gateways - Kubernetes

Ingress Gateways

Configuring Terminating Gateways on Kubernetes

Terminating Gateways - Kubernetes

Terminating Gateways

Welcome to the intro guide to Consul! This guide is the best place to start with Consul. We cover what Consul is, what problems it can solve, how it compares to existing software, and how you can get started using it. If you are familiar with the basics of Consul, the documentation provides a more detailed reference of available features.

Intro to Consul

It is not uncommon to find people using Chef, Puppet, and other configuration management tools to build service discovery mechanisms. This is usually done by querying global state to construct configuration files on each node during a periodic convergence run.

Consul vs. Chef, Puppet, etc.

Chef, Puppet, etc.

As a codebase grows, a monolithic app often evolves into a Service Oriented Architecture (SOA). A universal pain point for SOA is service discovery and configuration. In many cases, this leads to organizations building home grown solutions. It is an undisputed fact that distributed systems are hard; building one is error-prone and time-consuming. Most systems cut corners by introducing single points of failure such as a single Redis or RDBMS to maintain cluster state. These solutions may work in the short term, but they are rarely fault tolerant or scalable. Besides these limitations, they require time and resources to build and maintain.

Consul vs. Custom Solutions

Custom Solutions

Eureka is a service discovery tool that provides a best effort registry and discovery service. It uses central servers and clients which are typically natively integrated with SDKs. Consul provides a super set of features, such as health checking, key/value storage, ACLs, and multi-datacenter awareness.

Consul vs. Eureka

Eureka

The problems Consul solves are varied, but each individual feature has been solved by many different systems. Although there is no single system that provides all the features of Consul, there are other options available to solve some of these problems.

Consul vs. Other Software

Istio is a platform for connecting and securing microservices. This page describes the similarities and differences between Istio and Consul.

Consul vs. Istio

Istio

Nagios and Sensu are both tools built for monitoring. They are used to quickly notify operators when an issue occurs.

Consul vs. Nagios, Sensu

Nagios, Sensu

Modern service proxies provide high-level service routing, authentication, telemetry, and more for microservice and cloud environments. Envoy is a popular and feature rich proxy. This page describes how Consul relates to proxies such as Envoy.

Consul vs. Envoy and Other Proxies

Envoy and Other Proxies

Serf is a node discovery and orchestration tool and is the only tool discussed so far that is built on an eventually-consistent gossip model with no centralized servers. It provides a number of features, including group membership, failure detection, event broadcasts, and a query mechanism. However, Serf does not provide any high-level features such as service discovery, health checking or key/value storage. Consul is a complete system providing all of those features.

Consul vs. Serf

Serf

SkyDNS is a tool designed to provide service discovery. It uses multiple central servers that are strongly-consistent and fault-tolerant. Nodes register services using an HTTP API, and queries can be made over HTTP or DNS to perform discovery.

Consul vs. SkyDNS

SkyDNS

SmartStack is a tool which tackles the service discovery problem. It has a rather unique architecture and has 4 major components: ZooKeeper, HAProxy, Synapse, and Nerve. The ZooKeeper servers are responsible for storing cluster state in a consistent and fault-tolerant manner. Each node in the SmartStack cluster then runs both Nerves and Synapses. The Nerve is responsible for running health checks against a service and registering with the ZooKeeper servers. Synapse queries ZooKeeper for service providers and dynamically configures HAProxy. Finally, clients speak to HAProxy, which does health checking and load balancing across service providers.

Consul vs. SmartStack

SmartStack

ZooKeeper, doozerd, and etcd are all similar in their architecture. All three have server nodes that require a quorum of nodes to operate (usually a simple majority). They are strongly-consistent and expose various primitives that can be used through client libraries within applications to build complex distributed systems.

Consul vs. ZooKeeper, doozerd, etcd

ZooKeeper, doozerd, etcd

This section covers some of the internals of Consul, such as the architecture, consensus and gossip protocols, and security model.

Internals

Guide to partnership integrations for Consul NIA

Network Infrastructure Automation Integration Program

NIA Integration Program

Guide to partnership integrations for Consul.

Consul Integration Program

An agent can run in both client and server mode. Server nodes are responsible for running the consensus protocol and storing the cluster state. Before a Consul cluster can begin to service requests, a server node must be elected leader. Thus, the first nodes that are started are generally the server nodes. Bootstrapping is the process of joining these server nodes into a cluster.

Bootstrapping a Datacenter

Consul supports automatically joining a Consul datacenter using cloud metadata on various providers.

Cloud Auto-join

This page collects brief definitions of some of the technical terms used in the documentation.

Glossary

Installing Consul is simple. You can download a precompiled binary, compile
from source or run on Kubernetes. This page details these methods.

Get Started

When deploying Consul to a datacenter for the first time, there is an initial bootstrapping that must be done. As of Consul 0.4, an automatic bootstrapping is available and is the recommended approach. However, older versions only support a manual bootstrap that is documented here.

Manual Bootstrapping

Consul requires different amounts of compute resources, depending on cluster size and expected workload. This guide provides guidance on choosing compute resources.

Server Performance

Before starting Consul it is important to have the necessary bind ports accessible.

Required Ports

This section provides various guides for common actions. Due to the nature of
Consul, some of these procedures can be complex, so our goal is to provide
guidance to do them safely.

Guides

Consul Enterprise provides the ability to write events of user behavior with Consul’s API so operations and security users can perform legal compliance auditing.

Consul Enterprise Audit Logging

Audit Logging

Consul Enterprise provides a highly available service that manages taking snapshots, rotation and sending backup files offsite to Amazon S3 (or S3-compatible endpoints), Google Cloud Storage, or Azure Blob Storage.

Consul Enterprise Automated Backups

Automated Backups

Consul Enterprise enables you to federate Consul datacenters together on a pairwise basis, enabling partially-connected network topologies like hub-and-spoke.

Consul Enterprise Advanced Federation

Advanced Federation

Consul Enterprise features a number of capabilities beyond the open source offering that may be beneficial in certain workflows.

Enterprise Features

Consul Enterprise enables data isolation with Namespaces.

Consul Enterprise Namespaces

Namespaces

Consul Enterprise enables you create separate LAN gossip pools within one
cluster to segment network groups.

Consul Enterprise Network Segments

Network Segments

Consul Enterprise supports increased read scalability without impacting write latency by introducing read replicas.

»ACL System