HashiConf More sessions have been added to the conference agenda. Buy your pass and plan your schedule. Register
Consul
Consul Home

Consul auto-config on Docker

This topic provides an overview for Consul auto-config on Docker.

Introduction

Consul auto_config is a highly scalable method for automatically distributing secure properties from Consul servers to client agents, including Access Control List (ACL) tokens, TLS certificates, and gossip encryption keys.

The workflow to enable this method consists of two steps:

  1. Generate a JSON web token (JWT).
  2. Use the token on a Consul client to join a Consul cluster.

The Consul client then uses the JWTs to securely retrieve security setting changes from Consul servers.

Workflow

Consul's auto_config supports integrations with production-grade secrets management platforms like HashiCorp Vault, and other third-party platforms. The following examples demonstrate the processes for Vault and third party tool secint.

Consul server configuration

First, add the auto_config stanza to your Consul server agent's configuration.

consul_server.json

{
  ##... 
    "auto_config": {
        "authorization": {
            "enabled": true,
            "static": {
                "oidc_discovery_url": "http://vault-server:8200/v1/identity/oidc",
                "bound_issuer": "http://vault-server:8200/v1/identity/oidc",
                "bound_audiences": ["consul-cluster-dc1"],
                "claim_mappings": {
                    "/consul/hostname": "node_name"
                },
                "claim_assertions": [
                    "value.node_name == \"${node}\""
                ]
            }
        }
    },
  ##...
}

The auto_config configuration on a Consul server node consists of the following options:

  • authorization - Setting the sub key enabled to true enables the authorization service on this agent. This allows the server agent to process auto_config RPC requests from clients.

  • static - This object contains all static authorizer configuration settings.

    • oidc_discovery_url - The URL used to validate JSON web tokens (JWTs). In this example, Vault's OIDC URL endpoint http://vault-server:8200/v1/identity/oidc is used.
    • bound_issuer - The value for matching the iss value in a JSON web token (JWT). The issuer iss claim in a JWT is meant to refer to the resource that issued the JWT. In this example, the Vault server assigns its OIDC URL endpoint http://vault-server:8200/v1/identity/oidc as the issuer for generated JWTs.
    • bound_audiences - The value for matching the aud field of the JSON web token (JWT). The audience aud claim in a JWT is meant to refer to the authorization servers that should accept the token. In this example, Vault assigns the generated JWT tokens with an aud value of consul-cluster-dc1.
    • claim_assertions - List of assertions about the mapped claims required to authorize the incoming RPC request. In this example, "value.node_name == \"${node}\"" sets the value of the node_name variable to the hostname of the consul client making a request to the auto_config authorization servers.
    • claim_mappings - Mappings of claims (key) that will be copied to a metadata field (value). In this example, "/consul/hostname": "node_name" checks the value of the node_name variable against the metadata value of /consul/hostname.

For more information about the options you can specify, refer to the Consul agent auto_config parameter reference documentation.

Consul client configuration

On a Consul client agent with auto_config enabled, the client agent uses the value of a JSON web token (JWT) from intro_token_file when communicating with the configured server_addresses to request secure configuration settings. Consul then merges these settings into existing configurations on the client agent.

consul_client.json

{
    ##...
    "auto_config":{
        "enabled": true,
        "intro_token_file": "/consul/config/tokens/jwt",
        "server_addresses":[
            "consul-server1",
            "consul-server2",
            "consul-server3"
        ]
    },
    ##...
}

The auto_config configuration on a Consul client node consists of the following options:

  • enabled - Setting this key to true enables the auto_config client service on the agent. Enabling this option also turns on Consul service mesh because it is required for auto_config to issue certificates to client agents using the Consul service mesh CA.

  • intro_token_file - This specifies the file that contains the JSON web token (JWT) to use for the initial auto_config RPC to the Consul servers.

  • server_addresses - This specifies the addresses of servers in the local datacenter to use for the initial RPC. These addresses support Cloud Auto-Joining and can optionally include a port to use when making the outbound connection. If not port is provided the server RPC port will be used.

Check the Consul configuration documentation to learn more.

Generate JWTs

Vault uses its built-in Identity Secrets Engine to generate the JSON Web Tokens (JWTs) required to validate Consul client auto_config requests.

Named keys are used by a role to sign JSON web tokens (JWTs). The value for allowed_client_ids in this example becomes the value of aud when a JWT is generated. The audience aud claim in the JWT, consul-cluster-dc1, refers to the authorization servers that should accept the token.

Create a named key.

$ vault write identity/oidc/key/oidc-key-1 allowed_client_ids="consul-cluster-dc1"

Success! Data written to: identity/oidc/key/oidc-key-1

JSON web tokens (JWTs) are generated against a role and signed against a named key.

Create a role.

$ vault write identity/oidc/role/oidc-role-1 ttl=12h key="oidc-key-1" client_id="consul-cluster-dc1" template='{"consul": {"hostname": "consul-client" } }'

Success! Data written to: identity/oidc/role/oidc-role-1

The template template='{"consul": {"hostname": "consul-client" } }' creates additional JWT metadata for the Consul authorization servers to validate the request.

Policies are a declarative method to grant or forbid access to certain paths and operations in Vault. In this example, the policy file grants a single permission to read the token at the path identity/oidc/token/oidc-role-1.

policy.json

{
    "path": {
      "identity/oidc/token/oidc-role-1": {
        "policy": "read"
      }
    }
}

Create the policy.

$ vault policy write oidc-policy policy.json

Success! Uploaded policy: oidc-policy

Generate a signed JSON web token (JWT).

$ vault read identity/oidc/token/oidc-role-1

Key          Value
---          -----
client_id    consul-cluster-dc1
token        eyJhbGciOiJSUzI1NiIsImtpZCI6IjI4YjA2NDlmLTdlNjktMWFhMC03ZmYyLWI4ZDU5NGJhZmE5MCJ9.eyJhdWQiOiJjb25zdWwtY2x1c3Rlci1kYzEiLCJjb25zdWwiOnsiaG9zdG5hbWUiOiJjb25zdWwtY2xpZW50In0sImV4cCI6MTYyOTc5MDYwNSwiaWF0IjoxNjI5NzQ3NDA1LCJpc3MiOiJodHRwOi8vdmF1bHQtc2VydmVyOjgyMDAvdjEvaWRlbnRpdHkvb2lkYyIsIm5hbWVzcGFjZSI6InJvb3QiLCJzdWIiOiI4NWE5ZWMxYi1iMTcyLWU1YWEtZmU3Ni0xMzFkOWFjZmVjZTgifQ.eFDQ_TReNvKeMS4si92oiPOBcRbv0bGuVKq4Qns0ObnNrwFWVvDB9HLkCP7VzRuO9l9a3Jzl-Uk__Y_fF_JgWk7s2iZTg9RbBZD0TYz5-ziHU13wd7Onx9OjXjmw-5ah96dDFh3nqkuXJpV9upmVfXA7Zb5goYyfULa7gWeSNPjyjYNx2oirsFwH_xm9No9lttEA33XOGyAGi9UNBlvKdw6uXJfhnWTG2NxEt9y7JO_wNxjXKOUxhVhbb6ZxRZT_enbib1g_b-BVrNvTqB5UfSKyz6h3musoqDsAcLOEeAkl6dfWv3IezhqY2vNm5mQ3lH83AK6dZdwvVcG0DmdIpQ
ttl          12h

Copy the value of token to your clipboard.

Configure Consul client agent with the JWT

Paste the token value from your clipboard into the /consul/config/tokens/jwt file in the Consul client container, then save the changes.

/consul/config/tokens/jwt

eyJhbGciOiJSUzI1NiIsImtpZCI6IjI4YjA2NDlmLTdlNjktMWFhMC03ZmYyLWI4ZDU5NGJhZmE5MCJ9.eyJhdWQiOiJjb25zdWwtY2x1c3Rlci1kYzEiLCJjb25zdWwiOnsiaG9zdG5hbWUiOiJjb25zdWwtY2xpZW50In0sImV4cCI6MTYyOTc5MDYwNSwiaWF0IjoxNjI5NzQ3NDA1LCJpc3MiOiJodHRwOi8vdmF1bHQtc2VydmVyOjgyMDAvdjEvaWRlbnRpdHkvb2lkYyIsIm5hbWVzcGFjZSI6InJvb3QiLCJzdWIiOiI4NWE5ZWMxYi1iMTcyLWU1YWEtZmU3Ni0xMzFkOWFjZmVjZTgifQ.eFDQ_TReNvKeMS4si92oiPOBcRbv0bGuVKq4Qns0ObnNrwFWVvDB9HLkCP7VzRuO9l9a3Jzl-Uk__Y_fF_JgWk7s2iZTg9RbBZD0TYz5-ziHU13wd7Onx9OjXjmw-5ah96dDFh3nqkuXJpV9upmVfXA7Zb5goYyfULa7gWeSNPjyjYNx2oirsFwH_xm9No9lttEA33XOGyAGi9UNBlvKdw6uXJfhnWTG2NxEt9y7JO_wNxjXKOUxhVhbb6ZxRZT_enbib1g_b-BVrNvTqB5UfSKyz6h3musoqDsAcLOEeAkl6dfWv3IezhqY2vNm5mQ3lH83AK6dZdwvVcG0DmdIpQ

Restart the Consul client agent.

Now, future gossip encryption keys, TLS certificates, and other security setting changes will be distributed to the Consul client agent automatically.

Next steps

In this tutorial, you learned to deploy and configure a secure local containerized Consul datacenter using Docker Compose. You learned how to use auto_config to send secure properties throughout your datacenter. You learned the value auto_config adds by automatically distributing all future gossip encryption key, TLS certificate, and/or other security setting changes across the datacenter. Finally, you learned how to clean up your environment.

You can continue learning how to deploy a Consul datacenter in production by completing the Deployment guide. The collection includes securing the datacenter with Access Control Lists, encryption, DNS configuration, and datacenter federation.

You can also extend your Consul skills by exploring the following tutorials:

For additional reference documentation on the official Docker images for Consul and Vault, refer to the following websites:

Edit this page on GitHub