Blog HCP Consul on Azure goes GA, plus more Consul news from HashiConf EU Read more
  • Overview
    • Consul on Kubernetes
    • Control access with Consul API Gateway
    • Discover Services with Consul
    • Enforce Zero Trust Networking with Consul
    • Load Balancing with Consul
    • Manage Traffic with Consul
    • Multi-Platform Service Mesh with Consul
    • Network Infrastructure Automation with Consul
    • Observability with Consul
  • Enterprise
  • Tutorials
  • Docs
  • API
  • CLI
  • Community
GitHub
Download
Try HCP Consul
    • v1.12.x (latest)
    • v1.11.x
    • v1.10.x
    • v1.9.x
    • v1.8.x
    • Overview
      • Overview
      • What is a Service Mesh?
      • Overview
      • Chef, Puppet, etc.
      • Nagios
      • SkyDNS
      • SmartStack
      • Serf
      • Eureka
      • Istio
      • Envoy and Other Proxies
      • Custom Solutions
    • Overview
    • Manual Bootstrap
    • Consul Agent
    • Glossary
    • Required Ports
    • Bootstrapping a Datacenter
    • Cloud Auto-join
    • Server Performance
    • Kubernetes
  • API
  • Commands (CLI)
    • Register Services - Service Definitions
    • Find Services - DNS Interface
    • Monitor Services - Check Definitions
    • Overview
    • How Service Mesh Works
    • Configuration
      • Overview
      • Ingress Gateway
      • Mesh
      • Exported Services
      • Proxy Defaults
      • Service Defaults
      • Service Intentions
      • Service Resolver
      • Service Router
      • Service Splitter
      • Terminating Gateway
      • Overview
      • Envoy
      • Built-in Proxy
      • Proxy Integration
      • Managed (Deprecated)
      • Overview
      • Proxy Service Registration
      • Sidecar Service Registration
    • Service-to-service permissions - Intentions
    • Service-to-service permissions - Intentions (Legacy Mode)
    • Transparent Proxy
      • Overview
      • UI Visualization
      • Overview
      • Discovery Chain
    • Connectivity Tasks
    • Distributed Tracing
      • Overview
        • WAN Federation
        • Enabling Service-to-service Traffic Across Datacenters
        • Enabling Service-to-service Traffic Across Admin Partitions
      • Ingress Gateways
      • Terminating Gateways
      • What is Cluster Peering
      • Create and Manage Peering Connections
      • Cluster Peering on Kubernetes
    • Nomad
    • Kubernetes
      • Overview
      • Go Integration
      • Overview
      • Built-In CA
      • Vault
      • ACM Private CA
    • Develop and Debug
    • Security
    • Overview
    • Installation
    • Technical Specifications
    • Common Errors
    • Upgrades
    • Overview
    • Architecture
      • Installing Consul on Kubernetes
      • Installing Consul K8s CLI
        • Minikube
        • Kind
        • AKS (Azure)
        • EKS (AWS)
        • GKE (Google Cloud)
        • Red Hat OpenShift
        • Self Hosted Kubernetes
        • Consul Clients Outside Kubernetes
        • Consul Servers Outside Kubernetes
        • Single Consul Datacenter in Multiple Kubernetes Clusters
        • Consul Enterprise
        • Overview
        • Federation Between Kubernetes Clusters
        • Federation Between VMs and Kubernetes
        • Overview
        • Systems Integration
          • Overview
          • Bootstrap Token
          • Enterprise License
          • Gossip Encryption Key
          • Partition Token
          • Replication Token
          • Server TLS
          • Service Mesh Certificates
          • Snapshot Agent Config
          • Webhook Certificates
        • WAN Federation
      • Overview
      • Transparent Proxy
      • Ingress Gateways
      • Terminating Gateways
      • Ingress Controllers
      • Configuring a Connect CA Provider
      • Health Checks
        • Metrics
    • Service Sync
      • Overview
      • Upgrade An Existing Cluster to CRDs
    • Annotations and Labels
    • Consul DNS
      • Upgrading Consul on Kubernetes
      • Upgrading Consul K8s CLI
      • Uninstall
      • Certificate Rotation
      • Gossip Encryption Key Rotation
      • Configure TLS on an Existing Cluster
      • Common Error Messages
      • FAQ
    • Compatibility Matrix
    • Helm Chart Configuration
    • Consul K8s CLI Reference
    • Overview
    • Requirements
    • Task Resource Usage
      • Installation
      • Secure Configuration
      • Migrate Existing Tasks
      • Installation
      • Secure Configuration
      • ACL Controller
    • Architecture
    • Consul Enterprise
    • Configuration Reference
    • Overview
    • Register Lambda Functions
    • Invoke Lambda Functions
    • Overview
      • Installation
      • Requirements
      • Configure
      • Run Consul-Terraform-Sync
    • Architecture
      • Overview
      • Status
      • Tasks
      • Health
      • Overview
      • task
      • start
    • Configuration
    • Tasks
    • Terraform Modules
      • Overview
      • License
      • Terraform Cloud Driver
      • Overview
      • Terraform
      • Terraform Cloud
    • Compatibility
    • Consul KV
    • Sessions
    • Watches
    • Overview
      • General
      • CLI Reference
      • Configuration Reference
    • Configuration Entries
    • Telemetry
    • Sentinel
    • RPC
    • Overview
      • ACL System Overview
      • Tokens
      • Policies
      • Roles
      • Rules Reference
      • Legacy Mode
      • Token Migration
      • ACLs in Federated Datacenters
        • Overview
        • Kubernetes
        • JWT
        • OIDC
        • AWS IAM
    • Encryption
      • Overview
      • Core
      • Network Infrastructure Automation
    • Overview
    • Admin Partitions
    • Audit Logging
    • Automated Backups
    • Automated Upgrades
    • Enhanced Read Scalability
    • Single sign-on - OIDC
    • Redundancy Zones
    • Advanced Federation
    • Network Segments
    • Namespaces
    • NIA with TFE
    • Sentinel
      • Overview
      • FAQ
    • Overview
    • Improving Consul Resilience
    • Anti-Entropy
    • Consensus Protocol
    • Gossip Protocol
    • Jepsen Testing
    • Network Coordinates
    • Consul Integration Program
    • NIA Integration Program
    • Vault Integration
    • Proxy Integration
  • Consul Tools
    • Overview
    • Compatibility Promise
    • Specific Version Details
      • Overview
      • General Process
      • Upgrading to 1.2.4
      • Upgrading to 1.6.9
      • Upgrading to 1.8.13
      • Upgrading to 1.10.0
    • Common Error Messages
    • FAQ
    • Overview
      • v1.11.x
      • v1.10.x
      • v1.9.x
      • v0.3.x
      • v0.2.x
      • v0.1.x
      • v0.4.x
      • v0.3.x
      • v0.2.x
      • v0.6.x
      • v0.5.x
    • Overview
    • ACL
  • Guides
Type '/' to Search

»Terraform Driver

Consul-Terraform-Sync (CTS) extends the Consul ecosystem to include Terraform as an officially supported tooling project. With the Terraform driver, CTS installs the Terraform CLI locally and runs Terraform commands based on monitored Consul changes. This page details how the Terraform driver operates using local workspaces and templated files.

»Terraform CLI Automation

On startup, CTS:

  1. Downloads and installs Terraform
  2. Prepares local workspace directories. Terraform configuration and execution for each task is organized as separate Terraform workspaces. The state files for tasks are independent of each other.
  3. Generates Terraform configuration files that make up the root module for each task.

Once all workspaces are set up, CTS monitors the Consul catalog for service changes. When relevant changes are detected, the Terraform driver dynamically updates input variables for that task using a template to render them to a file named terraform.tfvars. This file is passed as a parameter to the Terraform CLI when executing terraform plan and terraform apply to update your network infrastructure with the latest Consul service details.

»Local Workspaces

Within the CTS configuration for a task, practitioners can select the desired module to run for the task as well as set the condition to execute the task. Each task executed by the Terraform driver corresponds to an automated root module that calls the selected module in an isolated Terraform environment. CTS will manage concurrent execution of these tasks.

Autogenerated root modules for tasks are maintained in local subdirectories of the CTS working directory. Each subdirectory represents the local workspace for a task. By default, the working directory sync-tasks is created in the current directory. To configure where Terraform configuration files are stored, set working_dir to the desired path or configure the task.working_dir individually.

Note: Although Terraform state files for task workspaces are independent, this does not guarantee the infrastructure changes from concurrent task executions are independent. Ensure that modules across all tasks are not modifying the same resource objects or have overlapping changes that may result in race conditions during automation.

»Root Module

The root module proxies Consul information, configuration, and other variables to the Terraform module for the task. The content of the files that make up the root module are sourced from CTS configuration, information for task's module to use as the automation playbook, and information from Consul such as service information.

A working directory with one task named "cts-example" would have the folder structure below when running with the Terraform driver.

$ tree sync-tasks/

sync-tasks/
└── cts-example/
    ├── main.tf
    ├── variables.tf
    ├── terraform.tfvars
    └── terraform.tfvars.tmpl
$ tree sync-tasks/

sync-tasks/
└── cts-example/
    ├── main.tf
    ├── variables.tf
    ├── terraform.tfvars
    └── terraform.tfvars.tmpl

The following files of the root module are generated for each task. An example of a root module created by CTS can be found in the project repository.

  • main.tf - The main file contains the terraform block, provider blocks, and a module block calling the module configured for the task.
    • terraform block - The corresponding provider source and versions for the task from the configuration files are placed into this block for the root module. The Terraform backend from the configuration is also templated here.
    • provider blocks - The provider blocks generated in the root module resemble the terraform_provider blocks from the configuration for CTS. They have identical arguments present and are set from the intermediate variable created per provider.
    • module block - The module block is where the task's module is called as a child module. The child module contains the core logic for automation. Required and optional input variables are passed as arguments to the module.
  • variables.tf - This file contains three types of variable declarations.
    • services input variable (required) determines module compatibility with Consul-Terraform Sync (read more on compatible Terraform modules for more details).
    • Any additional optional input variables provided by CTS that the module may use.
    • Various intermediate variables used to configure providers. Intermediate provider variables are interpolated from the provider blocks and arguments configured in the CTS configuration.
  • variables.module.tf - This file is created if there are variables configured for the task and contains the interpolated variable declarations that match the variables from configuration. These are then used to proxy the configured variables to the module through explicit assignment in the module block.
  • providers.tfvars - This file is created if there are providers configured for the task and defined terraform_provider blocks. This file may contain sensitive information. To omit sensitive information from this file, you can securely configure Terraform providers for CTS using environment variables or templating.
  • terraform.tfvars - The variable definitions file is where the services input variable and any optional CTS input variables are assigned values from Consul. It is periodically updated, typically when the task condition is met, to reflect the current state of Consul.
  • terraform.tfvars.tmpl - The template file is used by CTS to template information from Consul by using the HashiCorp configuration and templating library (hashicorp/hcat).

Note: Generated template and Terraform configuration files are crucial for the automation of tasks. Any manual changes to the files may not be preserved and could be overwritten by a subsequent update. Unexpected manual changes to the format of the files may cause automation to error.

github logoEdit this page
IntroGuidesDocsCommunityPrivacySecurityBrandConsent Manager