• HashiCorp Developer

  • HashiCorp Cloud Platform
  • Terraform
  • Packer
  • Consul
  • Vault
  • Boundary
  • Nomad
  • Waypoint
  • Vagrant
Consul
  • Install
  • Tutorials
  • Documentation
  • API
  • CLI
  • Try Cloud(opens in new tab)
  • Sign up
Security

Skip to main content
6 tutorials
  • Secure Gossip Communication with Encryption
  • Secure Consul Agent Communication with TLS Encryption
  • Secure Consul with Access Control Lists (ACLs)
  • Understand Access Control Privileges
  • Troubleshoot the Access Control System
  • Automate Consul Agent Security with Auto Config

  • Resources

  • Tutorial Library
  • Certifications
  • Community Forum
    (opens in new tab)
  • Support
    (opens in new tab)
  • GitHub
    (opens in new tab)
  1. Developer
  2. Consul
  3. Tutorials
  4. Security
  5. Secure Consul Agent Communication with TLS Encryption

Secure Consul Agent Communication with TLS Encryption

  • 14min

  • ConsulConsul
  • InteractiveInteractive

Securing your datacenter with TLS encryption is an important step for production deployments. TLS configuration is also a prerequisite of our Security Model. Correctly configuring TLS can be a complex process, especially given the wide range of deployment methodologies. This tutorial will provide you with a production ready TLS configuration for RPC and consensus communication. However, you will need to secure HTTP communication for the CLI and UI separately

Consul supports using TLS to verify the authenticity of servers and clients. To enable TLS, Consul requires that all servers have certificates that are signed by a single Certificate Authority(CA). Clients should also have certificates that are authenticated with the same CA.

If you want to secure service-to-service communication with TLS, review the secure service communication tutorial.

Prerequisites

The certificate generation and distribution steps outlined in this tutorial are meant for new Consul datacenters. All the steps should be completed before you start the agent. Review the Deployment Guide if you're getting started.

If you need to enable TLS encryption on an existing datacenter, review this tutorial and then complete the update Agents to Communicate with TLS tutorial. The tutorial for the existing datacenter includes steps for a zero-downtime update of the agents.

This tutorial is structured in way that you build knowledge with every step. It is recommended to read the whole tutorial before starting with the actual work.

Launch Terminal

This tutorial includes a free interactive command-line lab that lets you follow along on actual cloud infrastructure.

Initialize the built-in CA

One of the first steps to configuring TLS for Consul is generating certificates. In order to prevent unauthorized datacenter access, Consul requires all certificates be signed by the same Certificate Authority (CA). This should be a private CA and not a public one as any certificate signed by this CA will be allowed to communicate with the datacenter.

There are a variety of tools for managing your own CA, like the PKI secret backend in Vault, but for the sake of simplicity this tutorial you will use Consul's built-in TLS helpers to create a CA. You will only need to create one CA for the datacenter. You should generate all certificates on the same node or workstation that is used to create the CA. The node or workstation should be stable, preferably not a Consul agent or a cloud server.

You can create the CA and certificates before starting Consul, as long as you have the Consul binary installed in your path.

$ consul tls ca create
==> Saved consul-agent-ca.pem
==> Saved consul-agent-ca-key.pem

The CA certificate, consul-agent-ca.pem, contains the public key necessary to validate Consul certificates and therefore must be distributed to every node that runs a consul agent.

The CA key, consul-agent-ca-key.pem, will be used to sign certificates for Consul nodes and must be kept private. Possession of this key allows anyone to run Consul as a trusted server or generate new valid certificates for the datacenter and obtain access to all Consul data, including ACL tokens.

Create the server certificates

Create a server certificate for datacenter dc1 and domain consul, if your datacenter or domain is different, remember to use the appropriate flags.

Repeat this process on the same node where you created the CA, until there is an individual certificate for each server. The command can be called over and over again, it will automatically increase the certificate and key numbers. You will need to distribute the certificates to the servers.

$ consul tls cert create -server -dc <dc_name>

Below is an example for datacenter "dc1".

$ consul tls cert create -server -dc dc1
==> WARNING: Server Certificates grants authority to become a
    server and access all state in the cluster including root keys
    and all ACL tokens. Do not distribute them to production hosts
    that are not server nodes. Store them as securely as CA keys.
==> Using consul-agent-ca.pem and consul-agent-ca-key.pem
==> Saved dc1-server-consul-0.pem
==> Saved dc1-server-consul-0-key.pem

A federated Consul environment requires the server certificate to include the names of all Consul datacenters that are within the federated environment.

The -additional-dnsname flag allows you to provide an additional DNS name for Subject Alternative Names. localhost is always included. This flag may be provided multiple times.

$ consul tls cert create -server -dc <dc_name> -additional-dnsname=<secondary_consul_server_name>

Below is an example for a federated environment that contains two consul datacenters.

$ consul tls cert create -server -dc dc1 -additional-dnsname="*.dc2.consul"
==> WARNING: Server Certificates grants authority to become a
    server and access all state in the cluster including root keys
    and all ACL tokens. Do not distribute them to production hosts
    that are not server nodes. Store them as securely as CA keys.
==> Using consul-agent-ca.pem and consul-agent-ca-key.pem
==> Saved dc1-server-consul-0.pem
==> Saved dc1-server-consul-0-key.pem

NOTE See the consul TLS cert documentation to learn more about TLS cert creation options.

In order to authenticate Consul servers, servers are provided with a special certificate - one that contains server.dc1.consul in the Common Name and in the Subject Alternative Name. If you enable verify_server_hostname, only agents that provide such certificate are allowed to boot as a server. Without verify_server_hostname = true an attacker could compromise a Consul client agent and restart the agent as a server in order to get access to all the data in your datacenter! This is why server certificates are special, and only servers should have them provisioned.

NOTE Server keys, like the CA key, must be kept private - they effectively allow access to all Consul data.

Distribute the server certificates

After generating the server certificates, you will need to distribute them to the Consul servers and put their on-disk location in the agent configuration file.

The following files need to be copied to your Consul server:

  • consul-agent-ca.pem: CA public certificate.
  • dc1-server-consul-0.pem: Consul server node public certificate for the dc1 datacenter.
  • dc1-server-consul-0-key.pem: Consul server node private key for the dc1 datacenter.

Repeat this process until all servers have these three files.

Client certificate distribution

There are two methods for distributing client certificates: operator and auto encryption. The auto-encryption method introduced in Consul 1.5.2 alleviates the client certificate generation and distribution step for operators. This method uses the Connect CA to generate client certificates and then Consul will automatically distribute the certificates to all clients and is beneficial for large datacenters with many clients.

The operator method is recommended if you need to use a third-party CA or need more fine-grained control over certificate management. Review the Secure Agent Communication with Existing Certificate Authority tutorial for an example of using OpenSSL as a third-party CA. Additionally, you must use the operator method in Consul 1.5.1 and older.

This section describes the automated client certificate deployment process in Consul 1.5.2 and newer.

Configure the servers

At the beginning of this tutorial, you created the server certificates and distributed them. To use auto-encryption to distribute the client certificates, you will need to enable the auto_encrypt feature. Configure your servers with the following options.

Note: Consul will load all files in the configuration directory, for the following examples, you can add the configuration as new files. You can also add them to an existing configuration file, however, you should ensure that the syntax is valid before applying it to the agent. You can use consul validate to check configuration validity for both JSON and HCL files.

Consul server agent auto encrypt TLS configuration
Consul server agent auto encrypt TLS configuration
verify_incoming = true
verify_outgoing = true
verify_server_hostname = true
ca_file = "consul-agent-ca.pem"
cert_file = "dc1-server-consul-0.pem"
key_file = "dc1-server-consul-0-key.pem"
auto_encrypt {
  allow_tls = true
}
{
  "verify_incoming": true,
  "verify_outgoing": true,
  "verify_server_hostname": true,
  "ca_file": "consul-agent-ca.pem",
  "cert_file": "dc1-server-consul-0.pem",
  "key_file": "dc1-server-consul-0-key.pem",
  "auto_encrypt": {
    "allow_tls": true
  }
}

Notice that in addition to the verify settings, you need to enable allow_tls. The verify settings ensure that all communication between servers and clients is verified. When auto_encrypt is enabled, verify_outgoing is enabled by default.

Configure the clients

With auto-encryption, you can configure the Consul servers to automatically distribute certificates to the clients. To use this feature, you will need to configure clients to automatically get the certificates from the server.

Note, this method stores the certificates in memory, they are not persisted.

Configure the clients with the following options.

Consul client agent auto encrypt TLS configuration
Consul client agent auto encrypt TLS configuration
verify_incoming = false
verify_outgoing = true
verify_server_hostname = true
ca_file = "consul-agent-ca.pem"
auto_encrypt = {
  tls = true
}
{
  "verify_incoming": false,
  "verify_outgoing": true,
  "verify_server_hostname": true,
  "ca_file": "consul-agent-ca.pem",
  "auto_encrypt": {
    "tls": true
  }
}

In this section, you will use the operator method to generate and distribute all the client certificates.

Configure the servers

Configure the servers with the following options, including the agent TLS configuration for Consul servers and copied certificate files.

Consul client agent manual TLS configuration
Consul client agent manual TLS configuration
verify_incoming = true
verify_outgoing = true
verify_server_hostname = true
ca_file = "consul-agent-ca.pem"
cert_file = "dc1-server-consul-0.pem"
key_file = "dc1-server-consul-0-key.pem"
{
  "verify_incoming": true,
  "verify_outgoing": true,
  "verify_server_hostname": true,
  "ca_file": "consul-agent-ca.pem",
  "cert_file": "dc1-server-consul-0.pem",
  "key_file": "dc1-server-consul-0-key.pem"
}

TLS can be used to verify the authenticity of the servers with verify_outgoing and verify_server_hostname. It can also optionally verify client certificates when using verify_incoming.

Review the docs for specifics.

Setup the clients

To encrypt client communication, you will need to distribute the CA certificate, a client certificate, and its private key to each client. In this section you will generate the certificate and private key and then configure the clients.

Generate a client certificate

On the node where you created the CA and server certificates, generate a client certificate with consul tls cert create -client command.

$ consul tls cert create -client
==> Using consul-agent-ca.pem and consul-agent-ca-key.pem
==> Saved dc1-client-consul-0.pem
==> Saved dc1-client-consul-0-key.pem

The client certificate should also be signed by your CA, but it does not have that special Subject Alternative Name which means that if verify_server_hostname is enabled, they cannot start as a server.

Now that you have created the client certificate, distribute it to every Consul client in the Consul datacenter.

Configure the clients

Finally, you can configure the client agents.

verify_incoming = true
verify_outgoing = true
verify_server_hostname = true
ca_file = "consul-agent-ca.pem"
cert_file = "dc1-client-consul-0.pem"
key_file = "dc1-client-consul-0-key.pem"
{
  "verify_incoming": true,
  "verify_outgoing": true,
  "verify_server_hostname": true,
  "ca_file": "consul-agent-ca.pem",
  "cert_file": "dc1-client-consul-0.pem",
  "key_file": "dc1-client-consul-0-key.pem"
}

To review an explanation for each setting, return to the server configuration section.

Start Consul

Now that you have configured your servers and clients, you can start the Consul process.

$ systemctl start consul

Your agents will only be communicating with TLS encryption for RPC and consensus.

Next steps

After completing this tutorial, your agents will be configured with TLS for encrypted communication. With the auto encryption method, your client certificates are managed by the server. With the operator method, you distributed all the certificates, but have a more flexible configuration.

The other prerequisites for a secure Consul deployment are gossip encryption and ACLs with default deny.

Additional TLS encryption resources

For more TLS content review our other tutorials:

Enable TLS encryption on an existing datacenter. Generate TLS certificates with OpenSSL and secure the CLI and UI communication.

 Previous
 Next

This tutorial also appears in:

  •  
    30 tutorials
    Associate Tutorial List
    Study for the Consul Associate exam by following these tutorials. Login to Learn and bookmark them to track your progress. Study the complete list of study materials (including docs) in the Certification Prep guides.
    • Consul
  •  
    8 tutorials
    Interactive Labs
    Gain hands-on experience with Consul with interactive tutorials.
    • Consul

On this page

  1. Secure Consul Agent Communication with TLS Encryption
  2. Prerequisites
  3. Initialize the built-in CA
  4. Create the server certificates
  5. Client certificate distribution
  6. Start Consul
  7. Next steps
Give Feedback(opens in new tab)
  • Certifications
  • System Status
  • Terms of Use
  • Security
  • Privacy
  • Trademark Policy
  • Trade Controls
  • Give Feedback(opens in new tab)