HashiConf 2025 Don't miss the live stream of HashiConf Day 2 happening now View live stream
Consul Home

Configure client agents on virtual machines

This page describes the process to configure a Consul client agent on virtual machines (VMs). An automated process for production environments is demonstrated in the Consul getting started on Virtual Machines tutorials.

Overview

The Consul client agent is a long-running process that you must deploy on the nodes as your application workloads run on. They use encrypted gossip communication to communicate with Consul server agents so that Consul DNS requests return healthy results when you query the Consul catalog.

The process to deploy the client agent consists of the following steps:

  1. Verify that the correct version of the Consul binary is installed.
  2. Update the client agent configuration.
  3. Add the client agent configuration to the node.
  4. Ensure the Consul data directory has the correct permissions.
  5. Start the Consul binary.

You also have the option to configure the retry_join stanza so that when a Consul agent starts, it automatically joins a cluster. This ability is called cloud auto-join. Configuration requirements vary by cloud provider. For more information, refer to automatically join clusters to a cloud provider.

Client agent configuration

Update the configuration with the following information required to join the cluster:

  • Datacenter Name
  • Consul server address

Depending on the cluster's existing security, you may need to update the client configuration with these additional parameters:

  • CA certificate
  • TLS server name
  • Valid ACL token

For more information, refer to the Consul agents configuration file reference.

Joining a Consul cluster on K8s

Consul agents running on VMs can also join a Consul server cluster running within Kubernetes.

Auto-join

We recommend using the "k8s" cloud auto-join provider to join a cluster running within Kubernetes.

The auto-join provider dynamically discovers IP addresses to join using the Kubernetes API. It authenticates with Kubernetes using a standard kubeconfig file. Auto-join works with all major hosted Kubernetes offerings as well as self-hosted installations. The token in the kubeconfig file must have permissions to list pods in the namespace where Consul servers are deployed.

The following command has a Consul agent join a cluster running the official Helm chart. This command assumes that there are Consul servers listening on the default LAN Gossip port of 8301.

$ consul agent -retry-join 'provider=k8s label_selector="app=consul,component=server"'

Auto-join on the Pod network

The default Consul Helm chart installation makes Consul servers routable through their pod IPs for server RPCs. As a result, any external agents joining the Consul cluster running on Kubernetes need to be able to connect to those pod IPs.

In many hosted Kubernetes environments, you need to explicitly configure your hosting provider to ensure that pod IPs are routable from external VMs. For more information, refer to hosting provider documentation, such as Azure AKS CNI, AWS EKS CNI and GKE VPC-native clusters.

To join external agents with Consul on Kubernetes deployments installed with default values through the official Helm chart:

  1. Make sure the pod IPs of the servers in Kubernetes are routable from the VM and that the VM can access port 8301 (for gossip) and port 8300 (for server RPC) on those pod IPs.

  2. Make sure that the server pods running in Kubernetes can route to the VM's advertised IP on its gossip port (default 8301).

  3. Make sure you have the kubeconfig file for the Kubernetes cluster in $HOME/.kube/config on the external VM.

  4. On the external VM, run:

    consul agent \
  -advertise="$ADVERTISE_IP" \
  -retry-join='provider=k8s label_selector="app=consul,component=server"' \
  -bind=0.0.0.0 \
  -hcl='leave_on_terminate = true' \
  -hcl='ports { grpc = 8502 }' \
  -config-dir=$CONFIG_DIR \
  -datacenter=$DATACENTER \
  -data-dir=$DATA_DIR \

  5. Run consul members to check if the join was successful.

    / $ consul members
Node                                           Address           Status  Type    Build  Protocol  DC   Segment
consul-consul-server-0                         10.138.0.43:9301  alive    server  1.9.1  2         dc1  <all>
external-agent                                 10.138.0.38:8301  alive    client  1.9.0  2         dc1  <default>
gke-external-agent-default-pool-32d15192-grs4  10.138.0.43:8301  alive    client  1.9.1  2         dc1  <default>
gke-external-agent-default-pool-32d15192-otge  10.138.0.44:8301  alive    client  1.9.1  2         dc1  <default>
gke-external-agent-default-pool-32d15192-vo7k  10.138.0.42:8301  alive    client  1.9.1  2         dc1  <default>

Auto-join through host ports

If your external VMs cannot connect to Kubernetes pod IPs but they can connect to the internal host IPs of the nodes in the Kubernetes cluster, you can join the external VMs to the Kubernetes Consul service by exposing node ports on the host IP instead.

  1. Install the official Helm chart with the following values:

    client:
  exposeGossipPorts: true # exposes client gossip ports as hostPorts
server:
  exposeGossipAndRPCPorts: true # exposes the server gossip and RPC ports as hostPorts
  ports:
    # Configures the server gossip port
    serflan:
      # Note that this needs to be different than 8301, to avoid conflicting with the client gossip hostPort
      port: 9301

    This installation exposes the client gossip ports, the server gossip ports and the server RPC port at hostIP:hostPort. Note that hostIP is the internal IP of the VM that the client/server pods are deployed on.

  2. Make sure the IPs of the Kubernetes nodes are routable from the VM and that the VM can access ports 8301 and 9301 (for gossip) and port 8300 (for server RPC) on those node IPs.

  3. Make sure the server pods running in Kubernetes can route to the VM's advertise IP on its gossip port (default 8301).

  4. Make sure you have the kubeconfig file for the Kubernetes cluster in $HOME/.kube/config on the external VM.

  5. On the external VM, run:

    consul agent \
  -advertise="$ADVERTISE_IP" \
  -retry-join='provider=k8s host_network=true label_selector="app=consul,component=server"'
  -bind=0.0.0.0 \
  -hcl='leave_on_terminate = true' \
  -hcl='ports { grpc = 8502 }' \
  -config-dir=$CONFIG_DIR \
  -datacenter=$DATACENTER \
  -data-dir=$DATA_DIR \

    Note the addition of host_network=true in the retry-join argument.

  6. Run consul members to check if the join was successful.

    / $ consul members
Node                                           Address           Status  Type    Build  Protocol  DC   Segment
consul-consul-server-0                         10.138.0.43:9301  alive    server  1.9.1  2         dc1  <all>
external-agent                                 10.138.0.38:8301  alive    client  1.9.0  2         dc1  <default>
gke-external-agent-default-pool-32d15192-grs4  10.138.0.43:8301  alive    client  1.9.1  2         dc1  <default>
gke-external-agent-default-pool-32d15192-otge  10.138.0.44:8301  alive    client  1.9.1  2         dc1  <default>
gke-external-agent-default-pool-32d15192-vo7k  10.138.0.42:8301  alive    client  1.9.1  2         dc1  <default>

Manual join

If you are unable to use auto-join, you also have the option to utilise the -retry-join parameter with the address of at least one Consul server.

$ consul agent -retry-join=$CONSUL_SERVER_IP:$CONSUL_GOSSIP_PORT

A kubeconfig file is not required when using manual join.

Instead of hardcoding an IP address, we recommend you set up a DNS entry that resolves to the pod IPs or host IPs that the Consul server pods are running on.

Edit this page on GitHub