Consul-Terraform-Sync and Terraform Enterprise/Cloud integration
Version Note
The integration with the self-hosted version of Terraform
Enterprise is available only in Consul-Terraform-Sync version 0.3.0
and later.
The integration with Terraform Cloud is available only in Consul-Terraform-Sync
version 0.4.0
and later.
Enterprise Only
The terraform-cloud
driver requires Consul-Terraform-Sync
Enterprise which is available with Consul Enterprise.
With the 0.3.0 release, Consul-Terraform-Sync added the possibility to run
remote operations directly on your Terraform Enterprise instance, using the
terraform-cloud
driver. With the 0.4.0 release the integration got extended to
the Terraform Cloud platform.
In this tutorial you will learn how to configure Consul-Terraform-Sync to interact with Terraform Enterprise and Terraform Cloud.
Prerequisites
- A running instance of Terraform Enterprise or a Terraform Cloud account.
- A Consul datacenter; you can use a local dev instance if you do not have a running Consul datacenter.
Start Consul server in dev mode.
Register a service named web
.
Check service is present in the Consul catalog.
- Consul-Terraform-Sync 0.3.0+ if interacting with a self-hosted instance of Terraform Enterprise or Consul-Terraform-Sync 0.4.0+ if interacting with Terraform Cloud.
Prepare Terraform instance for the integration
Prior to running Consul-Terraform-Sync with a Terraform Cloud driver, you need to have an account on your Terraform instance and an organization set up.
Consul-Terraform-Sync uses API integration to communicate with Terraform, so you will also need a dedicated token.
Once you have an account and organization, the next step is to create a team. We recommend using a dedicated team and team token to run and authenticate Consul-Terraform-Sync. Using a team token has the benefits of restricting organization permissions as well as associating Consul-Terraform-Sync automated actions with the team rather than an individual.
After creating a dedicated team, update the team's permissions with "Manage Workspaces" organization access level. Consul-Terraform-Sync's main workflow revolves around creating and managing workspaces. Restricting the dedicated team's permission to "Manage Workspaces" level is sufficient and reduces security risk.
Finally, create a team token.
Once created, copy the token and save it in a safe place. You need it to configure Consul-Terraform-Sync.
Install Consul-Terraform-Sync
To install Consul-Terraform-Sync, find the appropriate package
for your system and download it as a zip archive. Unzip the package to extract
the binary named consul-terraform-sync
.
Visit the Network Infrastructure Automation with Consul-Terraform-Sync Intro tutorial to learn more about the installation process.
Once installed, verify Consul-Terraform-Sync is available in your path:
Configure Consul-Terraform-Sync license
Consul-Terraform-Sync Enterprise binaries require a Consul Enterprise license to run. There is no Consul-Terraform-Sync Enterprise specific license. As a result, Consul-Terraform-Sync Enterprise's licensing is similar to Consul Enterprise licensing.
If you do not have a valid Consul Enterprise license, you can sign-up for the trial license for Consul Enterprise.
Once you have a license, you can setup the license for Consul-Terraform-Sync in three different ways, in order of precedence:
CONSUL_LICENSE
: environment variable to set the license valueCONSUL_LICENSE_PATH
: environment variable to set the path to the file containing the licenselicense_path
: configuration file variable to set the path to the file containing the license
The Install a HashiCorp Enterprise License tutorial provides detailed steps on how to configure an enterprise license.
Configure Consul-Terraform-Sync
Next, configure the daemon to interact with your Consul datacenter and your Terraform instance.
The terraform-cloud
driver block
The driver block configures the subprocess that Consul-Terraform-Sync runs in automation.
The Terraform Cloud driver enables Consul-Terraform-Sync Enterprise to integrate with Terraform Cloud and with Terraform Enterprise, the self-hosted distribution of Terraform Cloud. With this driver, Consul-Terraform-Sync automates Terraform runs and remote operations for workspaces.
To configure the driver block you need the following information:
hostname
- The Terraform Enterprise hostname to connect to. Can be overridden with theTFC_HOSTNAME
environment variable. For Terraform Cloud usehttps://app.terraform.io
as hostname.organization
- The organization that hosts the workspaces managed by Consul-Terraform-Sync. Can be overridden with theTFC_ORGANIZATION
environment variable.token
- Team API token used for authentication with Terraform Enterprise and workspace management. Only workspace permissions are needed for Consul-Terraform-Sync. If you don't want to write it directly in the configuration file, the token can also be:- provided by the
TFC_TOKEN
environment variable - securely queried from Vault
- provided by the
Note : Only one network driver can be configured per deployment of Consul-Terraform-Sync. If you need to automate workflows on multiple instances of Terraform Enterprise or have a mixed environment with Terraform Enterprise, Terraform Cloud, and Terraform OSS you need to configure and launch multiple instances of Consul-Terraform-Sync.
The task
block
A task is executed when Consul-Terraform-Sync detects any changes associated
with services in the catalog that have been configured in the task
block.
The changes may include one or more of the following events:
- changes to service values (for example the service's IP address)
- added or removed service instance
- added or removed tags
In the example configuration provided, you defined a task block that will
monitor and react to changes on web
and api
services and launch the module
defined in the module
parameter when any change is detected.
The module used in this tutorial is the Consul-Terraform-Sync Test Module - Hello. The module creates two text files, a static file and a file that has a greeting for each Consul service. The module will help you test the automation without the need to deploy infrastructure.
The task name, learn-cts-example
in this configuration, will be the name of
the workspace created in Terraform UI.
Start Consul-Terraform-Sync
Once the configuration is complete, start Consul-Terraform-Sync.
Warning
If you are using Consul-Terraform-Sync version older than
v0.6.0, the start
parameter is not supported and must be removed.
From the output, you can verify that Consul-Terraform-Sync successfully connected
to Terraform. Additionally, Consul-Terraform-Sync successfully created the
learn-cts-example
workspace before launching the task.
Verify task execution on Terraform UI
By clicking on the link in the logs, you can verify the run in Terraform Enterprise UI.
By clicking on the workspace, you can see the details of the run queued by
Consul-Terraform-Sync. Notice that the workspace takes the value of the task
name and that the workspace mentions it was created via Consul-Terraform-Sync.
On the variables tab you can find the terraform.tfvars
that, when integrating
with Terraform, gets created as a workspace variable.
Test Consul-Terraform-Sync driven changes
To test the automation, you can register a new service in Consul matching one of the names captured by the task.
Once the services is registered you can verify Consul-Terraform-Sync picks up the change from the logs.
Also, by clicking on the link in the logs, you can verify the run in Terraform UI.
Setup notification for automatic workflows
To monitor the automated runs in your workspace, you can use the "Create a Notification" feature in Terraform Enterprise and Terraform Cloud to receive notifications on your workflow events.
From your workspace tab, navigate over Settings/Notification to setup the feature.
Read more on how to configure notification in Terraform documentation
Clean up resources
To clean up your environment, stop Consul-Terraform-Sync daemon and delete the workspace from Terraform UI.
Navigate to Settings/Destruction and Deletion and click Delete from Terraform Cloud.
Next steps
In this tutorial, you learned how to integrate Consul-Terraform-Sync with your Terraform Enterprise and Terraform Cloud instance and automatically run workflows based on your Consul service catalog.
In the next tutorial you will learn the requirements to build a Consul-Terraform-Sync compatible module and the steps to test it and publish it to the Terraform registry.
To learn more about Network Infrastructure Automation with Consul-Terraform-Sync check the full documentation for it on the Consul website.
If you need guidance on how to deploy your services to Consul service mesh, refer to our tutorials for getting started with service mesh on Kubernetes or VMs.
Check out the following tutorials and collections to learn even more about operating, observing, and monitoring your Consul service mesh.