Join External Servers to Consul on Kubernetes

If you have a Consul cluster already running, you can configure your Consul on Kubernetes installation to join this existing cluster.

The below values.yaml file shows how to configure the Helm chart to install Consul that will join an existing Consul server cluster.

The global.enabled value first disables all chart components by default so that each component is opt-in. This allows us to only setup the client agents. We then opt-in to the client agents by setting client.enabled to true.

Next, configure externalServers to point it to Consul servers. The externalServers.hosts value must be provided and should be set to a DNS, an IP, or an exec= string with a command returning Consul IPs. Please see this documentation on how the exec= string works.externalServers Other values in the externalServers section are optional. Please refer to Helm Chart configuration for more details.

global:
  enabled: false

externalServers:
  hosts: [<consul server DNS, IP or exec= string>]

Note: If you are looking to join Consul clients to an existing Consul server cluster, please see this documentation.

Configuring TLS

Note: Consul on Kubernetes currently does not support external servers that require mutual authentication for the HTTPS clients of the Consul servers, that is when servers have either tls.defaults.verify_incoming or tls.https.verify_incoming set to true. As noted in the Security Model, that setting isn't strictly necessary to support Consul's threat model as it is recommended that all requests contain a valid ACL token.

If the Consul server has TLS enabled, you would also need to provide the CA certificate that Consul on Kubernetes will need to talk to the server. First save this certificate in a Kubernetes secret and then provide it in your Helm values below, in addition to the values mentioned above:

global:
  tls:
    enabled: true
    caCert:
      secretName: <CA certificate secret name>
      secretKey: <CA Certificate secret key>
externalServers:
  enabled: true
  hosts: [<consul server DNS, IP or exec= string>]

If your HTTPS port is different from Consul's default 8501, you must also set externalServers.httpsPort.

Configuring ACLs

If you are running external servers with ACLs enabled, there are a couple of ways to configure the Helm chart to help initialize ACL tokens for Consul clients and consul-k8s components for you.

Manually Bootstrapping ACLs

If you would like to call the ACL bootstrapping API yourself or if your cluster has already been bootstrapped with ACLs, you can provide the bootstrap token to the Helm chart. The Helm chart will then use this token to configure ACLs for Consul clients and any consul-k8s components you are enabling.

First, create a Kubernetes secret containing your bootstrap token:

kubectl create secret generic bootstrap-token --from-literal='token=<your bootstrap token>'

Then provide that secret to the Helm chart:

global:
  acls:
    manageSystemACLs: true
    bootstrapToken:
      secretName: bootstrap-token
      secretKey: token

The bootstrap token requires the following minimal permissions:

acl:write
operator:write if enabling Consul namespaces
agent:read if using WAN federation over mesh gateways

Next, configure external servers. The Helm chart will use this configuration to talk to the Consul server's API to create policies, tokens, and an auth method. If you are enabling Consul Connect, k8sAuthMethodHost should be set to the address of your Kubernetes API server so that the Consul servers can validate a Kubernetes service account token when using the Kubernetes auth method with consul login.

Note: If externalServers.k8sAuthMethodHost is set and you are also using WAN federation (global.federation.enabled is set to true), ensure that global.federation.k8sAuthMethodHost is set to the same value as externalServers.k8sAuthMethodHost.

externalServers:
  enabled: true
  hosts: [<consul server DNS, IP or exec= string>]
  k8sAuthMethodHost: 'https://kubernetes.example.com:443'

Your resulting Helm configuration will end up looking similar to this:

global:
  enabled: false
  acls:
    manageSystemACLs: true
    bootstrapToken:
      secretName: bootstrap-token
      secretKey: token
externalServers:
  enabled: true
  hosts: [<consul server DNS, IP or exec= string>]
  k8sAuthMethodHost: 'https://kubernetes.example.com:443'

Bootstrapping ACLs via the Helm chart

If you would like the Helm chart to call the bootstrapping API and set the server tokens for you, then the steps are similar. The only difference is that you don't need to set the bootstrap token. The Helm chart will save the bootstrap token as a Kubernetes secret.

global:
  enabled: false
  acls:
    manageSystemACLs: true
externalServers:
  enabled: true
  hosts: [<consul server DNS, IP or exec= string>]
  k8sAuthMethodHost: 'https://kubernetes.example.com:443'