Storing Server TLS certificates in Vault

To use Vault to issue Server TLS certificates the following will be needed:

1. Bootstrap the Vault PKI engine and bootstrap it with any configuration required for your infrastructure.
2. Create Vault Policies that will allow the Consul server to access the certificate issuing url.
3. Create Vault Policies that will allow the Consul components, e.g. ingress gateways, controller, to access the CA url.
4. Create Kubernetes auth roles that link these policies to the Kubernetes service accounts of the Consul components.

Bootstrapping the PKI Engine

First, we need to bootstrap the Vault cluster by enabling and configuring the PKI Secrets Engine to be able to serve TLS certificates to Consul. The process can be as simple as the following, or more complicated such as in this example which also uses an intermediate signing authority.

• Enable the PKI Secrets Engine:

  $ vault secrets enable pki
  

• Tune the engine to enable longer TTL:

  $ vault secrets tune -max-lease-ttl=87600h pki
  

• Generate the root CA

  $ vault write -field=certificate pki/root/generate/internal \
          common_name="dc1.consul" \
          ttl=87600h
  

  Note: Where common_name is comprised of combining global.datacenter dot global.domain.

Create Vault Policies for the Server TLS Certificates

Next we will create a policy that allows ["create", "update"] access to the certificate issuing URL so the Consul servers can fetch a new certificate/key pair.

path "pki/issue/consul-server" {
  capabilities = ["create", "update"]
}

$ vault policy write consul-server consul-server-policy.hcl

Create Vault Policies for the CA URL

Next, we will create a policy that allows ["read"] access to the CA URL, this is required for the Consul components to communicate with the Consul servers in order to fetch their auto-encryption certificates.

path "pki/cert/ca" {
  capabilities = ["read"]
}

$ vault policy write ca-policy ca-policy.hcl

Create Vault Roles for the PKI engine, Consul servers and components

Next, a Vault role for the PKI engine will set the default certificate issuance parameters:

$ vault write pki/roles/consul-server \
    allowed_domains="<Allowed-domains-string>" \
    allow_subdomains=true \
    allow_bare_domains=true \
    allow_localhost=true \
    generate_lease=true \
    max_ttl="720h"

To generate the <Allowed-domains-string> use the following script as a template:

#!/bin/sh

# NAME is set to either the value from `global.name` from your Consul K8s value file, or your $HELM_RELEASE_NAME-consul
export NAME=consulk8s
# NAMESPACE is where the Consul on Kubernetes is installed
export NAMESPACE=consul
# DATACENTER is the value of `global.datacenter` from your Helm values config file
export DATACENTER=dc1

echo allowed_domains=\"$DATACENTER.consul, $NAME-server, $NAME-server.$NAMESPACE, $NAME-server.$NAMESPACE.svc\"

Prior to creating Vault auth roles for the Consul server and the Consul components, ensure that the Vault Kubernetes auth method is enabled as described in Vault Kubernetes Auth Method.

Finally, three Kubernetes auth roles need to be created, one for the Consul servers, one for the Consul clients, and one for the Consul components.

Role for Consul servers:

$ vault write auth/kubernetes/role/consul-server \
    bound_service_account_names=<Consul server service account> \
    bound_service_account_namespaces=<Consul installation namespace> \
    policies=consul-server \
    ttl=1h

To find out the service account name of the Consul server, you can run:

 $ helm template --release-name ${RELEASE_NAME} --show-only templates/server-serviceaccount.yaml hashicorp/consul

Role for Consul clients:

$ vault write auth/kubernetes/role/consul-client \
    bound_service_account_names=<Consul client service account>  \
    bound_service_account_namespaces=default \
    policies=ca-policy \
    ttl=1h

To find out the service account name of the Consul client, use the command below.

 $ helm template --release-name ${RELEASE_NAME} --show-only templates/client-serviceaccount.yaml hashicorp/consul

Role for CA components:

$ vault write auth/kubernetes/role/consul-ca \
    bound_service_account_names="*" \
    bound_service_account_namespaces=<Consul installation namespace> \
    policies=ca-policy \
    ttl=1h

The above Vault Roles will now be your Helm values for global.secretsBackend.vault.consulServerRole and global.secretsBackend.vault.consulCARole respectively.

Deploying the Consul Helm chart

Now that we've configured Vault, you can configure the Consul Helm chart to use the Server TLS certificates from Vault:

global:
  secretsBackend:
    vault:
      enabled: true
      consulServerRole: consul-server
      consulClientRole: consul-client
      consulCARole: consul-ca
  tls:
    enableAutoEncrypt: true
    enabled: true
    caCert:
      secretName: "pki/cert/ca"
server:
  serverCert:
    secretName: "pki/issue/consul-server"
  extraVolumes:
    - type: "secret"
      name: <vaultCASecret>
      load: "false"

The vaultCASecret is the Kubernetes secret that stores the CA Certificate that is used for Vault communication. To provide a CA, you first need to create a Kubernetes secret containing the CA. For example, you may create a secret with the Vault CA like so:

$ kubectl create secret generic vault-ca --from-file vault.ca=/path/to/your/vault/