Consul-Terraform-Sync run modes and status inspection

In the previous tutorial you learned how to install Consul-Terraform-Sync, the HashiCorp's Network Infrastructure Automation tool, to configure a task to monitor services in your Consul datacenter, to run it in daemon mode and to check the information it retrieves from Consul.

In this tutorial you will learn the different run modes available with Consul-Terraform-Sync, how to monitor the task status, and to locate the Terraform state files in the Consul backend.

Prerequisites

Prerequisite for this tutorial are the same as the ones you had for the previous one, specifically you will need:

• A Consul datacenter with two registered services, web and api. You can follow the Network Infrastructure Automation with Consul-Terraform-Sync Intro for guidance on installing Consul.

• The Consul-Terraform-Sync binary

• Terraform 0.13.0+ binary (will be automatically installed if not present).

• For this tutorial, you will be using the Consul Print Module to configure Consul-Terraform-Sync. This Terraform module creates text files on your local machine containing Consul service information. It is a useful module to use to familiarize yourself with Consul Terraform Sync, without requiring deployed infrastructure and credentials.

Configure Consul-Terraform-Sync

The Consul-Terraform-Sync daemon is configured using configuration files and supports HashiCorp Configuration Language (HCL) and JSON file formats.

For this tutorial you will use the same configuration used in the previous tutorial.

Copy the content of the code block below and save it in a file named cts-config.hcl.

log_level   = "INFO"

working_dir = "sync-tasks"
port        = 8558

syslog {}

buffer_period {
  enabled = true
  min     = "5s"
  max     = "20s"
}

consul {
  address = "localhost:8500"
}

driver "terraform" {
  # version = "0.14.0"
  # path = ""
  log         = false
  persist_log = false

  backend "consul" {
    gzip = true
  }
}

task {
 name        = "learn-cts-example"
 description = "Example task with two services"
 module      = "findkim/print/cts"
 version     = "0.1.0"
 condition "services" {
  names = ["web", "api"]
 }
}

Start Consul-Terraform-Sync

Once the configuration file is created it is possible to start Consul-Terraform-Sync using the consul-terraform-sync binary. The binary requires the configuration to be passed either using the -config-file or -config-dir.

Consul-Terraform-Sync is intended to be run as a long-running daemon to be able to monitor Consul events and adapt your infrastructure accordingly. For better flexibility it also has different modes to run including some that can be useful to safely test your configuration and the changes that are going to be applied or to run them only once if you do not need to dynamically adapt your infrastructure deployment.

In this tutorial you will first inspect the validity of the configuration using the inspect mode, test the module correctness running in once mode, and finally run in daemon mode to verify the Consul-Terraform-Sync status.

Inspect mode

Inspect mode is designed to help you examine the changes that your configuration, once ran via Terraform, will apply to your infrastructure. Running Consul-Terraform-Sync in inspect mode will apply no changes to your infrastructure but will exit with a non-zero exit status if any error is encountered.

This is achieved by Consul-Terraform-Sync running terraform plan for each task in the configuration.

The inspect mode is intended to be ran before daemon mode to confirm configuration is accurate.

$ consul-terraform-sync start -config-file cts-config.hcl -inspect

[INFO]  cli: v0.6.0 (7f0d7ba)
[INFO]  cli: running controller in inspect mode
[INFO]  cli: setting up controller: type=readonly
[INFO]  ctrl: initializing Consul client and testing connection
[INFO]  driver.terraform: install terraform: install_path=/
[INFO]  driver.terraform: successfully installed terraform
[INFO]  cli: initializing controller
[INFO]  ctrl: initializing driver
[INFO]  ctrl: initializing all tasks
...

Initializing modules...
...

Initializing the backend...
...

Terraform has been successfully initialized!
...

[INFO]  ctrl: drivers initialized
[INFO]  ctrl: inspecting all tasks
[INFO]  ctrl: inspecting task: task_name=learn-cts-example
[INFO] running Terraform command: /terraform plan -no-color -input=false -detailed-exitcode -lock-timeout=0s -lock=true -parallelism=10 -refresh=true
[INFO]  ctrl: inspection results: task_name=learn-cts-example
  plan=
...
  | Plan: 2 to add, 0 to change, 0 to destroy.
  
[INFO]  ctrl: inspected task: task_name=learn-cts-example
[INFO]  ctrl: completed task inspections
[INFO]  cli: graceful shutdown

Once mode

Once inspection is successful you might still want to test-run it to make sure it fully works before delegating to Consul-Terraform-Sync the infrastructural changes. You can do this by using the once running mode that will run all tasks once and gracefully exit or exit with a non-zero status in case an error is encountered before completion.

This is achieved by Consul-Terraform-Sync running terraform apply for each task in the configuration.

$ consul-terraform-sync start -config-file cts-config.hcl -once

[INFO]  cli: v0.6.0 (7f0d7ba)
[INFO]  cli: running controller in once mode
[INFO]  cli: setting up controller: type=readwrite
[INFO]  ctrl: initializing Consul client and testing connection
[INFO]  driver.terraform: install terraform: install_path=/
[INFO]  driver.terraform: successfully installed terraform
[INFO]  cli: initializing controller
[INFO]  ctrl: initializing driver
[INFO]  ctrl: initializing all tasks
[INFO]  client.terraformcli: Terraform output is muted
[INFO]  driver.terraform: retrieved 0 Terraform handlers for task: task_name=learn-cts-example
[INFO]  ctrl: drivers initialized
[INFO]  ctrl: executing all tasks once through
[INFO]  ctrl: executing task: task_name=learn-cts-example
[INFO]  ctrl: task completed: task_name=learn-cts-example
[INFO]  ctrl: all tasks completed once
[INFO]  cli: controller in Once mode has completed
[INFO]  cli: graceful shutdown

Daemon mode

Once you verified the correctness of the configuration and the proper execution of the different tasks configured you can now launch Consul-Terraform-Sync as a long running process.

$ consul-terraform-sync start -config-file cts-config.hcl

[INFO]  cli: v0.6.0 (7f0d7ba)
[INFO]  cli: running controller in daemon mode
[INFO]  cli: setting up controller: type=readwrite
[INFO]  ctrl: initializing Consul client and testing connection
[INFO]  driver.terraform: install terraform: install_path=/
[INFO]  driver.terraform: successfully installed terraform
[INFO]  cli: initializing controller
[INFO]  ctrl: initializing driver
[INFO]  ctrl: initializing all tasks
[INFO]  client.terraformcli: Terraform output is muted
[INFO]  driver.terraform: retrieved 0 Terraform handlers for task: task_name=learn-cts-example
[INFO]  ctrl: drivers initialized
[INFO]  ctrl: executing all tasks once through
[INFO]  ctrl: executing task: task_name=learn-cts-example
[INFO]  ctrl: task completed: task_name=learn-cts-example
[INFO]  ctrl: all tasks completed once
[INFO]  api: starting server: port=8558
...

Use the status API to monitor task execution

Consul-Terraform-Sync provides the /status REST endpoints to share status-related information for tasks. This information is available for understanding the status of individual tasks and across tasks.

The health status value is determined by aggregating the success or failure of the event of a task detecting changes in Consul services and then updating network infrastructure.

The 5 most recent events are stored in Consul-Terraform-Sync.

$ curl --silent localhost:8558/v1/status

Example output:

{
  "task_summary": {
    "status": {
      "successful": 1,
      "errored": 0,
      "critical": 0,
      "unknown": 0
    },
    "enabled": {
      "true": 1,
      "false": 0
    }
  }
}

Terraform state in Consul

When using Consul as a backend for Consul-Terraform-Sync the Terraform state will be persisted in the Consul KV store as a different state file for each task.

The default path for the state object is consul-terraform-sync/terraform-env:task-name with task-name being the task identifier appended to the end of the path.

Next steps

In this tutorial you learned the different run modes for Consul-Terraform-Sync and how to inspect task status using the REST API, and Terraform state when using Consul as backend for Terraform.

Specifically you:

• Tried the three different execution modes: daemon, inspect, and once
• Verified tasks execution using the REST API
• Checked the Terraform state files in Consul using Consul UI

In the next tutorial you will learn the requirements to build a Consul-Terraform-Sync compatible module and the steps to test it and publish it to the Terraform registry.

To learn more about Network Infrastructure Automation with Consul-Terraform-Sync check the full documentation for it on the Consul website.

If you need guidance on how to deploy your services to Consul service mesh, refer to our tutorials for getting started with service mesh on Kubernetes or VMs.

To learn even more about operating, observing, and monitoring your Consul service mesh, check out the following tutorials and collections.

• Maintenance & Monitoring
• Service Discovery & Health
• Layer 7 Observability with Consul, Prometheus, Grafana, and Kubernetes