Service configuration with Consul Template

The Consul template tool provides a programmatic method for rendering configuration files from a variety of locations, including Consul KV. It is an ideal option for replacing complicated API queries that often require custom formatting. The template tool is based on Go templates and shares many of the same attributes.

Consul template is a useful tool with several applications. This tutorial will focus on two of its use cases.

1. Update configuration files. The Consul template tool can be used to update service configuration files. A common use case is managing load balancer configuration files that need to be updated regularly in a dynamic infrastructure.

2. Discover data about the Consul datacenter and service. It is possible to collect information about the services in your Consul datacenter. For example, you could collect a list of all services running on the datacenter or you could discover all service addresses for the Redis service. Note, this use case has limited scope for production.

In this tutorial, you will get an introduction on how consul-template works, how to install it, and two use cases.

Before completing this tutorial, you should have some familiarity with Consul KV and Go templates.

Introduction to Consul template

Consul template is a simple, yet powerful tool. When initiated, it reads one or more template files and queries Consul for all data needed to render them. Typically, you run consul-template as a daemon which will fetch the initial values and then continue to watch for updates, re-rendering the template whenever there are relevant changes in the datacenter. You can alternatively use the -once flag to fetch and render the template once which is useful for testing and setup scripts that are triggered by some other automation for example a provisioning tool. Finally, the template can also run arbitrary commands after the update process completes. For example, it can send the HUP signal to the load balancer service after a configuration change has been made.

The Consul template tool is flexible, it can fit into many different environments and workflows. Depending on the use-case, you may have a single consul-template instance on a handful of hosts or may need to run several instances on every host. Each consul-template process can manage multiple unrelated files though and will de-duplicate the fetches as needed if those files share data dependencies so it can reduce the load on Consul servers to share where possible.

Install Consul template

For this tutorial, you will use a local Consul agent in development mode which can be started with consul agent -dev. To quickly set up a local Consul agent, refer to the getting started tutorial. The Consul agent must be running to complete all of the following steps.

The Consul template tool is not included with the Consul binary and will need to be installed separately. It can be installed from a precompiled binary or compiled from source. You will be installing the precompiled binary.

First, download the binary from the Consul Template releases page.

$ curl -O https://releases.hashicorp.com/consul-template/0.19.5/consul-template<_version_OS>.tgz

Next, extract the binary and move it into your $PATH.

$ tar -zxf consul-template<_version_OS>.tgz

To compile from source, check the instructions in the contributing section in GitHub.

Use case: Consul KV

In this first use case example, you will render a template that pulls the HashiCorp address from Consul KV. To do this, you will create a simple template that contains the HashiCorp address, run consul-template, add a value to Consul KV for HashiCorp's address, and finally view the rendered file.

First, you will need to create a template file find_address.tpl to query Consul's KV store:

{{ key "/hashicorp/street_address" }}

Next, you will run consul-template specifying both the template to use and the file to update.

$ consul-template -template "find_address.tpl:hashicorp_address.txt"

The consul-template process will continue to run until you kill it with CTRL+c. For now, you will leave it running.

Finally, open a new terminal so you can write data to the key in Consul using the command line interface.

$ consul kv put hashicorp/street_address "101 2nd St"

Success! Data written to: hashicorp/street_address

You can verify the data was written by viewing the hashicorp_address.txt file which will be located in the same directory where consul-template was run.

$ cat hashicorp_address.txt

101 2nd St

If you update the key hashicorp/street_address, you can see the changes to the file immediately. Go ahead and try consul kv put hashicorp/street_address "22b Baker ST".

You can verify that this simple process can have powerful implications. For example, it is possible to use this same process for updating your HAProxy load balancer configuration.

You can now kill the consul-template process with CTRL+c.

Use case: discover all services

In this use case example, you will discover all the services running in the Consul datacenter. To follow along, you use the local development agent from the previous example.

First, you will need to create a new template all-services.tpl to query all services.

{{ range services -}}
# {{ .Name }}
{{- range service .Name }}
{{ .Address }}
{{- end }}

{{ end -}}

Next, run Consul template specifying the template you just created and the -once flag. The -once flag will tell the process to run once and then quit.

$ consul-template -template="all-services.tpl:all-services.txt" -once

If you complete this on your local development agent, you should still get the consul service among the results when viewing all-services.txt.

# consul
127.0.0.7

On a development or production datacenter, you would get a list of all the services. For example:

# consul
104.131.121.232

# redis
104.131.86.92
104.131.109.224
104.131.59.59

# web
104.131.86.92
104.131.109.224
104.131.59.59

Next steps

In this tutorial, you learned how to set up and use the Consul template tool. To find additional examples, refer to the examples folder in GitHub.