Install Consul on Kubernetes with Helm

This topic describes how to install Consul on Kubernetes using the official Consul Helm chart. We recommend using this method if you are installing Consul on Kubernetes for multi-cluster deployments that involve cross-partition or cross datacenter communication.

For instruction on how to install Consul on Kubernetes using the Consul K8s CLI, refer to Install Consul on Kubernetes with the Consul K8s CLI.

Introduction

We recommend using the Consul Helm chart to install Consul on Kubernetes for multi-cluster installations that involve cross-partition or cross datacenter communication. The Helm chart installs and configures all necessary components to run Consul.

Consul can run directly on Kubernetes so that you can leverage Consul functionality if your workloads are fully deployed to Kubernetes. For heterogeneous workloads, Consul agents can join a server running inside or outside of Kubernetes. Refer to the Consul on Kubernetes architecture to learn more about its general architecture.

The Helm chart exposes several useful configurations and automatically sets up complex resources, but it does not automatically operate Consul. You must still become familiar with how to monitor, backup, and upgrade the Consul cluster.

The Helm chart has no required configuration, so it installs a Consul cluster with default configurations. We strongly recommend that you learn about the configuration options before going to production.

For a hands-on experience with Consul as a service mesh for Kubernetes, follow the Getting Started with Consul service mesh tutorial.

Requirements

You must meet the following requirements to install Consul on Kubernetes with Helm:

• Helm version 3.6+. Visit the Helm website to download the latest version.
• kubectl configured to point to the Kubernetes cluster you want to install Consul on.

Depending on your requirements you can:

• Perform a default installation by using the default Helm chart configuration. Use this option if you are testing Consul or setting up a test environment that does not have security requirements.
• Perform a custom installation by modifying Helm chart configuration parameters. This is the recommended approach if you are installing Consul in production.

Add HashiCorp Helm Repository

Before the installation, add HashiCorp repository to Helm.

$ helm repo add hashicorp https://helm.releases.hashicorp.com
 "hashicorp" has been added to your repositories

Install Consul

1. Verify that you have access to the Consul chart:

   $ helm search repo hashicorp/consul
   NAME                CHART VERSION   APP VERSION DESCRIPTION
   hashicorp/consul    2.0.1           2.0.1       Official HashiCorp Consul Chart
   

2. Before you install Consul on Kubernetes with Helm, ensure that the consul Kubernetes namespace does not exist. We recommend installing Consul on a dedicated namespace.

   $ kubectl get namespace
   NAME              STATUS   AGE
   default           Active   18h
   kube-node-lease   Active   18h
   kube-public       Active   18h
   kube-system       Active   18h
   

   Note

   If you experience errors when deploying or upgrading Consul in a K8s namespace different than the default consul, refer to the following list of technical constraints.

   • Consul cannot automatically detect its target K8s namespace, and needs it explicitly listed with -namespace in the parameters of Helm or the consul-k8s tool.
   • Consul needs its target K8s namespace added to the connectInjector.k8sAllowNamspaces Helm chart value. For more information, refer to this specific setting in the Consul Helm chart reference.

3. Install Consul on Kubernetes using Helm. The Helm chart does everything to set up your deployment. After installation, agents automatically form clusters, elect leaders, and run the necessary agents.

   • Run the following command to install the latest version of Consul on Kubernetes with its default configuration.

     $ helm install consul hashicorp/consul --set global.name=consul --create-namespace --namespace consul
     

     You can also install Consul on a dedicated namespace of your choosing by modifying the value of the -n flag for the Helm install.

   • To install a specific version of Consul on Kubernetes, issue the following command with --version flag:

     $ export VERSION=1.0.1 && \
         helm install consul hashicorp/consul 
           --set global.name=consul 
           --version ${VERSION} 
           --create-namespace 
           --namespace consul
     

Custom installation

If you want to customize your installation, create a values.yaml file to override the default settings. To learn what settings are available, run helm inspect values hashicorp/consul or review the Helm Chart Reference.

The following sections contain customization examples you may want to use as reference:

• Minimal Consul service mesh installation
• Select API gateway configuration
• Install Consul on OpenShift clusters
• Enable Consul CNI plugin
• Enable Consul service mesh on selected namespaces

Minimal values.yaml for Consul service mesh

The following values.yaml config file contains the minimum required settings to enable Consul service mesh:

global:
  name: consul

After you create your values.yaml file, run helm install with the --values flag:

$ helm install consul hashicorp/consul --create-namespace --namespace consul --values values.yaml
NAME: consul
...

Select API gateway configuration

A fully-functional Consul deployment requires an API gateway. Use the diagram below to identify your platform and choose the correct API gateway implementation for your infrastructure.

This table shows the same information as the diagram and describes the required Helm values for each platform configuration.

Install Consul on OpenShift clusters

Red Hat OpenShift is a security-conscious, opinionated wrapper for Kubernetes. To install Consul on OpenShift-managed Kubernetes, run the following commands to identify your OCP version and whether TCPRoute resources are present in your cluster:

1. Check OCP version.

   oc version
   

2. Check if your cluster uses TCPRoute.

   kubectl get tcproutes -A
   

Use the output to choose the correct tab:

• If the OCP version is 4.19 or later and no TCPRoute resources are present, use the OpenShift >= 4.19, no TCPRoute required tab.
• If the OCP version is 4.19 or later and TCPRoute resources are present, use the OpenShift >= 4.19, TCPRoute required tab.
• If the OCP version is 4.18 or earlier, use the OpenShift <= 4.18 tab.

global:
  installK8sNetworkingCRDs: true
  openshift:
    enabled: true
    crds:
      enableTcpRoute: true

global:
  installK8sNetworkingCRDs: false
  openshift:
    enabled: true
    isOcpGreaterthan4_18: true
    crds:
      enableTcpRoute: false

global:
  installK8sNetworkingCRDs: false
  openshift:
    enabled: true
    isOcpGreaterthan4_18: true
    crds:
      enableTcpRoute: false
      consulapi:
        enabled: true

Refer to openshift in the Helm chart reference for additional information.

Update your Consul configuration

If you already installed Consul and want to make changes, you need to run helm upgrade. Refer to Upgrading for more details.

Enable the Consul CNI plugin

By default, Consul injects a connect-inject-init init container as part of the Kubernetes pod startup process when Consul is in transparent proxy mode. The container configures traffic redirection in the service mesh through the sidecar proxy. To configure redirection, the container requires elevated CAP_NET_ADMIN privileges, which may not be compatible with security policies in your organization.

Instead, you can enable the Consul container network interface (CNI) plugin to perform traffic redirection. Because the plugin is executed by the local Kubernetes kubelet, the plugin already has the elevated privileges necessary to configure the network.

The Consul Helm Chart is responsible for installing the Consul CNI plugin. To configure the plugin to be installed, add the following configuration to your values.yaml file:

global:
  name: consul
connectInject:
  enabled: true
  cni:
    enabled: true
    logLevel: info
    cniBinDir: "/opt/cni/bin"
    cniNetDir: "/etc/cni/net.d"

global:
  name: consul
connectInject:
  enabled: true
  cni:
    enabled: true
    logLevel: info
    cniBinDir: "/home/kubernetes/bin"
    cniNetDir: "/etc/cni/net.d"

global:
  name: consul
  openshift:
    enabled: true
connectInject:
  enabled: true
  cni:
    enabled: true
    logLevel: info
    multus: true
    cniBinDir: "/var/lib/cni/bin"
    cniNetDir: "/etc/kubernetes/cni/net.d"

The following table describes the available CNI plugin options:

Enable Consul service mesh on select namespaces

By default, Consul service mesh is enabled on almost all namespaces within a Kubernetes cluster, except kube-system and local-path-storage. To restrict the service mesh to a subset of namespaces:

1. Specify a namespaceSelector that matches a label attached to each namespace where you want to deploy the service mesh. To enable service mesh on select namespaces by label by default, you must set the connectInject.default value to true.

   values.yaml

   global:
     name: consul
   connectInject:
     enabled: true
     default: true
     namespaceSelector: |
       matchLabels:
         connect-inject : enabled
   

2. Label the namespaces where you would like to enable Consul service mesh.

   $ export NAMESPACE=foo && \
       kubectl create ns $NAMESPACE && \
       kubectl label namespace $NAMESPACE connect-inject=enabled
   

3. Run helm install with the --values flag:

   $ helm install consul hashicorp/consul --create-namespace --namespace consul --values values.yaml
   NAME: consul
   

Usage

You can view the Consul UI and access the Consul HTTP API after installation.

Viewing the Consul UI

The Consul UI is enabled by default when using the Helm chart.

For security reasons, it is not exposed through a LoadBalancer service by default. To visit the UI, you must use kubectl port-forward.

Port forward with TLS disabled

If running with TLS disabled, the Consul UI is accessible through http on port 8500:

$ kubectl port-forward service/consul-server --namespace consul 8500:8500

After you set up the port forward, navigate to http://localhost:8500.

Port forward with TLS enabled

If running with TLS enabled, the Consul UI is accessible through https on port 8501:

$ kubectl port-forward service/consul-server --namespace consul 8501:8501

After you set up the port forward, navigate to https://localhost:8501.

ACLs enabled

If ACLs are enabled, you need to input an ACL token to display all resources and make modifications in the UI.

To retrieve the bootstrap token that has full permissions, run:

$ kubectl get secrets/consul-bootstrap-acl-token --template='{{.data.token | base64decode }}'
e7924dd1-dc3f-f644-da54-81a73ba0a178%

Then paste the token into the UI under the ACLs tab (without the %).

Exposing the UI through a service

If you want to expose the UI via a Kubernetes Service, configure the ui.service chart values. Because this service allows requests to the Consul servers, it should not be open to the world.

Access the Consul HTTP API

While technically any listening agent can respond to the HTTP API, communicating with the local Consul node has important caching behavior and allows you to use the simpler /agent endpoints for services and checks.

To find information about a node, you can use the downward API.

The following is an example pod specification. In addition to pods, anything with a pod template can also access the Consul API and can therefore also access Consul: StatefulSets, Deployments, Jobs, etc.

apiVersion: v1
kind: Pod
metadata:
  name: consul-example
spec:
  containers:
    - name: example
      image: 'hashicorp/consul:latest'
      env:
        - name: HOST_IP
          valueFrom:
            fieldRef:
              fieldPath: status.hostIP
      command:
        - '/bin/sh'
        - '-ec'
        - |
          export CONSUL_HTTP_ADDR="${HOST_IP}:8500"
          consul kv put hello world
  restartPolicy: Never

The following example Deployment shows how you can access the host IP can be accessed from nested pod specifications:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: consul-example-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: consul-example
  template:
    metadata:
      labels:
        app: consul-example
    spec:
      containers:
        - name: example
          image: 'hashicorp/consul:latest'
          env:
            - name: HOST_IP
              valueFrom:
                fieldRef:
                  fieldPath: status.hostIP
          command:
            - '/bin/sh'
            - '-ec'
            - |
              export CONSUL_HTTP_ADDR="${HOST_IP}:8500"
              consul kv put hello world