Blog HCP Consul on Azure goes GA, plus more Consul news from HashiConf EU Read more
  • Overview
    • Consul on Kubernetes
    • Control access with Consul API Gateway
    • Discover Services with Consul
    • Enforce Zero Trust Networking with Consul
    • Load Balancing with Consul
    • Manage Traffic with Consul
    • Multi-Platform Service Mesh with Consul
    • Network Infrastructure Automation with Consul
    • Observability with Consul
  • Enterprise
  • Tutorials
  • Docs
  • API
  • CLI
  • Community
GitHub
Download
Try HCP Consul
    • v1.12.x (latest)
    • v1.11.x
    • v1.10.x
    • v1.9.x
    • v1.8.x
    • Overview
      • Overview
      • What is a Service Mesh?
      • Overview
      • Chef, Puppet, etc.
      • Nagios
      • SkyDNS
      • SmartStack
      • Serf
      • Eureka
      • Istio
      • Envoy and Other Proxies
      • Custom Solutions
    • Overview
    • Manual Bootstrap
    • Consul Agent
    • Glossary
    • Required Ports
    • Bootstrapping a Datacenter
    • Cloud Auto-join
    • Server Performance
    • Kubernetes
  • API
  • Commands (CLI)
    • Register Services - Service Definitions
    • Find Services - DNS Interface
    • Monitor Services - Check Definitions
    • Overview
    • How Service Mesh Works
    • Configuration
      • Overview
      • Ingress Gateway
      • Mesh
      • Exported Services
      • Proxy Defaults
      • Service Defaults
      • Service Intentions
      • Service Resolver
      • Service Router
      • Service Splitter
      • Terminating Gateway
      • Overview
      • Envoy
      • Built-in Proxy
      • Proxy Integration
      • Managed (Deprecated)
      • Overview
      • Proxy Service Registration
      • Sidecar Service Registration
    • Service-to-service permissions - Intentions
    • Service-to-service permissions - Intentions (Legacy Mode)
    • Transparent Proxy
      • Overview
      • UI Visualization
      • Overview
      • Discovery Chain
    • Connectivity Tasks
    • Distributed Tracing
      • Overview
        • WAN Federation
        • Enabling Service-to-service Traffic Across Datacenters
        • Enabling Service-to-service Traffic Across Admin Partitions
      • Ingress Gateways
      • Terminating Gateways
      • What is Cluster Peering
      • Create and Manage Peering Connections
      • Cluster Peering on Kubernetes
    • Nomad
    • Kubernetes
      • Overview
      • Go Integration
      • Overview
      • Built-In CA
      • Vault
      • ACM Private CA
    • Develop and Debug
    • Security
    • Overview
    • Installation
    • Technical Specifications
    • Common Errors
    • Upgrades
    • Overview
    • Architecture
      • Installing Consul on Kubernetes
      • Installing Consul K8s CLI
        • Minikube
        • Kind
        • AKS (Azure)
        • EKS (AWS)
        • GKE (Google Cloud)
        • Red Hat OpenShift
        • Self Hosted Kubernetes
        • Consul Clients Outside Kubernetes
        • Consul Servers Outside Kubernetes
        • Single Consul Datacenter in Multiple Kubernetes Clusters
        • Consul Enterprise
        • Overview
        • Federation Between Kubernetes Clusters
        • Federation Between VMs and Kubernetes
        • Overview
        • Systems Integration
          • Overview
          • Bootstrap Token
          • Enterprise License
          • Gossip Encryption Key
          • Partition Token
          • Replication Token
          • Server TLS
          • Service Mesh Certificates
          • Snapshot Agent Config
          • Webhook Certificates
        • WAN Federation
      • Overview
      • Transparent Proxy
      • Ingress Gateways
      • Terminating Gateways
      • Ingress Controllers
      • Configuring a Connect CA Provider
      • Health Checks
        • Metrics
    • Service Sync
      • Overview
      • Upgrade An Existing Cluster to CRDs
    • Annotations and Labels
    • Consul DNS
      • Upgrading Consul on Kubernetes
      • Upgrading Consul K8s CLI
      • Uninstall
      • Certificate Rotation
      • Gossip Encryption Key Rotation
      • Configure TLS on an Existing Cluster
      • Common Error Messages
      • FAQ
    • Compatibility Matrix
    • Helm Chart Configuration
    • Consul K8s CLI Reference
    • Overview
    • Requirements
    • Task Resource Usage
      • Installation
      • Secure Configuration
      • Migrate Existing Tasks
      • Installation
      • Secure Configuration
      • ACL Controller
    • Architecture
    • Consul Enterprise
    • Configuration Reference
    • Overview
    • Register Lambda Functions
    • Invoke Lambda Functions
    • Overview
      • Installation
      • Requirements
      • Configure
      • Run Consul-Terraform-Sync
    • Architecture
      • Overview
      • Status
      • Tasks
      • Health
      • Overview
      • task
      • start
    • Configuration
    • Tasks
    • Terraform Modules
      • Overview
      • License
      • Terraform Cloud Driver
      • Overview
      • Terraform
      • Terraform Cloud
    • Compatibility
    • Consul KV
    • Sessions
    • Watches
    • Overview
      • General
      • CLI Reference
      • Configuration Reference
    • Configuration Entries
    • Telemetry
    • Sentinel
    • RPC
    • Overview
      • ACL System Overview
      • Tokens
      • Policies
      • Roles
      • Rules Reference
      • Legacy Mode
      • Token Migration
      • ACLs in Federated Datacenters
        • Overview
        • Kubernetes
        • JWT
        • OIDC
        • AWS IAM
    • Encryption
      • Overview
      • Core
      • Network Infrastructure Automation
    • Overview
    • Admin Partitions
    • Audit Logging
    • Automated Backups
    • Automated Upgrades
    • Enhanced Read Scalability
    • Single sign-on - OIDC
    • Redundancy Zones
    • Advanced Federation
    • Network Segments
    • Namespaces
    • NIA with TFE
    • Sentinel
      • Overview
      • FAQ
    • Overview
    • Improving Consul Resilience
    • Anti-Entropy
    • Consensus Protocol
    • Gossip Protocol
    • Jepsen Testing
    • Network Coordinates
    • Consul Integration Program
    • NIA Integration Program
    • Vault Integration
    • Proxy Integration
  • Consul Tools
    • Overview
    • Compatibility Promise
    • Specific Version Details
      • Overview
      • General Process
      • Upgrading to 1.2.4
      • Upgrading to 1.6.9
      • Upgrading to 1.8.13
      • Upgrading to 1.10.0
    • Common Error Messages
    • FAQ
    • Overview
      • v1.11.x
      • v1.10.x
      • v1.9.x
      • v0.3.x
      • v0.2.x
      • v0.1.x
      • v0.4.x
      • v0.3.x
      • v0.2.x
      • v0.6.x
      • v0.5.x
    • Overview
    • ACL
  • Guides
Type '/' to Search

»Ingress Gateways

1.8.0+: This feature is available in Consul versions 1.8.0 and newer.

Ingress gateways enable connectivity within your organizational network from services outside the Consul service mesh to services in the mesh. An ingress gateway is a type of proxy and must be registered as a service in Consul, with the kind set to "ingress-gateway". They are an entrypoint for outside traffic and allow you to define what services should be exposed and on what port. You configure an ingress gateway by defining a set of listeners that each map to a set of backing services.

To enable easier service discovery, a new Consul DNS subdomain is provided, on <service>.ingress.<domain>.

For listeners with a protocol other than tcp, multiple services can be specified for a single listener. In this case, the ingress gateway relies on host/authority headers to decide the service that should receive the traffic. The host used to match traffic defaults to the Consul DNS ingress subdomain, but can be changed using the hosts field.

Ingress Gateway Architecture

»Prerequisites

Ingress gateways also require that your Consul datacenters are configured correctly:

  • You'll need to use Consul version 1.8.0 or newer.
  • Consul Connect must be enabled on the datacenter's Consul servers.
  • gRPC must be enabled on all client agents.

Currently, Envoy is the only proxy with ingress gateway capabilities in Consul.

»Running and Using an Ingress Gateway

For a complete example of how to allow external traffic inside your Consul service mesh, review the ingress gateway tutorial.

»Ingress Gateway Configuration

Ingress gateways are configured in service definitions and registered with Consul like other services, with two exceptions. The first is that the kind must be "ingress-gateway". Second, the ingress gateway service definition may contain a Proxy.Config entry just like a Connect proxy service, to define opaque configuration parameters useful for the actual proxy software. For Envoy there are some supported gateway options as well as escape-hatch overrides.

Note: If ACLs are enabled, ingress gateways must be registered with a token granting service:write for the ingress gateway's service name, service:read for all services in the ingress gateway's configuration entry, and node:read for all nodes of the services in the ingress gateway's configuration entry. These privileges authorize the token to route communications to other Connect services. If the Consul client agent on the gateway's node is not configured to use the default gRPC port, 8502, then the gateway's token must also provide agent:read for its node's name in order to discover the agent's gRPC port. gRPC is used to expose Envoy's xDS API to Envoy proxies.

Configuration entries are global in scope. A configuration entry for a gateway name applies across all federated Consul datacenters. If ingress gateways in different Consul datacenters need to route to different sets of services within their datacenter, then the ingress gateways must be registered with different names.

»Custom TLS Certificates via Secret Discovery Service (SDS)

Advanced Topic: This topic describes a low-level feature designed for developers building integrations with custom TLS management solutions.

Consul 1.11 added support for ingress gateways to serve TLS certificates to inbound traffic that are sourced from an external service. The external service must implement Envoy's gRPC Secret Discovery Service (or SDS) API.

The following procedure describes how to configure an ingress gateway with TLS certificates from an SDS source. The instructions assume that you are familiar with Envoy configuration and the SDS protocol.

»Configure Static SDS Cluster(s)

Each Envoy proxy that makes up this Ingress Gateway must define one or more additional static clusters when registering. These additional clusters define how Envoy should connect to the required SDS service(s). Defining extra clusters in Envoy's bootstrap configuration requires a manual registration of the Ingress Gateway with Consul proxy. It's not possible to use the -register flag with consul connect envoy -gateway=ingress to automatically register the proxy in this case.

The cluster(s) must provide connection information and any necessary authentication information such as mTLS credentials.

The following example will demonstrate how to use:

  • A DNS name to discover the SDS service addresses
  • Local certificate files for TLS client authentication with the SDS server. The certificates are assumed to be created and managed by some other process.
  1. Register the proxy service.

    The following Proxy Service Definition defines the additional cluster configuration that will be provided to Envoy when it starts. With this TLS configuration, Envoy will detect changes to the certificate and key files on disk so an external process may maintain and rotate them without needing an Envoy restart.

    // public-ingress.hcl
    Services {
      Name = "public-ingress"
      Kind = "ingress-gateway"
    
      Proxy {
        Config {
          envoy_extra_static_clusters_json = <<EOF
    {
      "name": "sds-cluster",
      "connect_timeout": "5s",
      "http2_protocol_options": {},
      "type": "LOGICAL_DNS",
      "transport_socket": {
        "name":"tls",
        "typed_config": {
          "@type":"type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext",
          "common_tls_context":{
            "tls_certificate_sds_secret_configs": [
              {
                "name":"tls_sds",
                "sds_config":{
                  "path":"/certs/sds-auth-cert.json"
                }
              }
            ],
            "validation_context_sds_secret_config": {
              "name":"validation_context_sds",
              "sds_config":{
                  "path":"/certs/sds-validation.json"
              }
            }
          }
        }
      },
      "load_assignment": {
        "cluster_name": "sds-cluster",
        "endpoints": [
          {
            "lb_endpoints": [
              {
                "endpoint": {
                  "address": {
                    "socket_address": {
                      "address": "sds-server.svc.cluster.local",
                      "port_value": 8080,
                    }
                  }
                }
              }
            ]
          }
        ]
      }
    }
    EOF
        }
      }
    }
    
    // public-ingress.hcl
    Services {
      Name = "public-ingress"
      Kind = "ingress-gateway"
    
      Proxy {
        Config {
          envoy_extra_static_clusters_json = <<EOF
    {
      "name": "sds-cluster",
      "connect_timeout": "5s",
      "http2_protocol_options": {},
      "type": "LOGICAL_DNS",
      "transport_socket": {
        "name":"tls",
        "typed_config": {
          "@type":"type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext",
          "common_tls_context":{
            "tls_certificate_sds_secret_configs": [
              {
                "name":"tls_sds",
                "sds_config":{
                  "path":"/certs/sds-auth-cert.json"
                }
              }
            ],
            "validation_context_sds_secret_config": {
              "name":"validation_context_sds",
              "sds_config":{
                  "path":"/certs/sds-validation.json"
              }
            }
          }
        }
      },
      "load_assignment": {
        "cluster_name": "sds-cluster",
        "endpoints": [
          {
            "lb_endpoints": [
              {
                "endpoint": {
                  "address": {
                    "socket_address": {
                      "address": "sds-server.svc.cluster.local",
                      "port_value": 8080,
                    }
                  }
                }
              }
            ]
          }
        ]
      }
    }
    EOF
        }
      }
    }
    
  2. Issue the following command to create the registration.

    consul services register public-ingress.hcl
    
    consul services register public-ingress.hcl
    

    The command must be executed against the Consul agent on the Envoy proxy's node.

»Setup TLS Client Authentication for SDS

Configuration files similar to the following examples must be available on the disk where the Envoy proxy will run. The actual certificates and keys referenced in the configuration files must also be present.

  1. Configure TLS client authentication for SDS.

    The certificates and keys must be saved to the same disk where the Envoy proxy will run. The following example files reference the PEM-encoded certificate and key files to be used for TLS Client Authentication with the SDS service (sds-client-auth.{crt,key}) and the certificate authority certificate used to validate the SDS server's TLS credentials (sds-ca.crt).

    Refer to [Envoy's documentation] https://www.envoyproxy.io/docs/envoy/latest/api-v3/bootstrap/bootstrap) for more details on this configuration and other possible authentication options.

    // /certs/sds-auth-cert.json
    {
      "resources": [
        {
          "@type": "type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.Secret",
          "name": "tls_sds",
          "tls_certificate": {
            "certificate_chain": {
              "filename": "/certs/sds-client-auth.crt"
            },
            "private_key": {
              "filename": "/certs/sds-client-auth.key"
            }
          }
        }
      ]
    }
    
    // /certs/sds-auth-cert.json
    {
      "resources": [
        {
          "@type": "type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.Secret",
          "name": "tls_sds",
          "tls_certificate": {
            "certificate_chain": {
              "filename": "/certs/sds-client-auth.crt"
            },
            "private_key": {
              "filename": "/certs/sds-client-auth.key"
            }
          }
        }
      ]
    }
    
    // /certs/sds-validation.json
    {
      "resources": [
        {
          "@type": "type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.Secret",
          "name": "validation_context_sds",
          "validation_context": {
            "trusted_ca": {
              "filename": "/certs/sds-ca.crt"
            }
          }
        }
      ]
    }
    
    // /certs/sds-validation.json
    {
      "resources": [
        {
          "@type": "type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.Secret",
          "name": "validation_context_sds",
          "validation_context": {
            "trusted_ca": {
              "filename": "/certs/sds-ca.crt"
            }
          }
        }
      ]
    }
    
  2. Issue the following command to start Envoy.

    $ consul connect envoy -gateway=ingress -service public-ingress
    
    $ consul connect envoy -gateway=ingress -service public-ingress
    

»Configure the Ingress Gateway to Use Certificates from SDS

SDS certificates may now be configured in the ingress-gateway Config Entry.

The following example shows a single default certificate and key being used for all listeners.

// public-ingress-cfg.hcl
Kind = "ingress-gateway"
Name = "public-ingress"

TLS {
  SDS {
    # This must match the name of the static cluster from step #1
    ClusterName = "sds-cluster"
    # This is the name of the certificate resource to load.
    CertResource = "example.com-public-cert"
  }
}

Listeners = [
  {
    Port = 8443
    Protocol = "http"
    Services = ["*"]
  }
]

// public-ingress-cfg.hcl
Kind = "ingress-gateway"
Name = "public-ingress"

TLS {
  SDS {
    # This must match the name of the static cluster from step #1
    ClusterName = "sds-cluster"
    # This is the name of the certificate resource to load.
    CertResource = "example.com-public-cert"
  }
}

Listeners = [
  {
    Port = 8443
    Protocol = "http"
    Services = ["*"]
  }
]

  1. Run consul config write public-ingress-cfg.hcl to write this configuration.

    The Envoy instance will now start a listener on port 8443 and attempt to fetch the TLS certificate named from the SDS server.

Separate certificates may be loaded per listener or per-service with hostname (SNI) switching. See the Config Entry reference for more details.

github logoEdit this page
IntroGuidesDocsCommunityPrivacySecurityBrandConsent Manager