Create a Local Consul Datacenter
In this tutorial, you'll create your first datacenter with multiple members.
When a new Consul agent starts, it doesn't know about other agents; it is essentially a datacenter with one member. To add a new agent to an existing datacenter you give it the IP address of any other agent in the datacenter (either a client or a server), which causes the new agent to join the datacenter. Once the agent is a member of the new datacenter, it automatically learns about the other agents via gossip.
In this tutorial, you will join two agents together to create a two-member Consul datacenter.
Prerequisites
Consul is a distributed application that is designed to have one agent per machine. To run two agents on the same computer you will need to install VirtualBox, and Vagrant, which will run virtual machines to simulate a distributed environment.
Note
VirtualBox does not currently support Apple Silicon. As a result, you will not be able to complete this tutorial if your machine runs on an Apple M chipset.
Set up the environment
Make a directory to store Vagrant's configuration for this tutorial.
Create a new file in the directory called Vagrantfile
and paste the code below into it. This file will instruct Vagrant to create two virtual
machines on your computer with the Consul binary preinstalled.
Boot your two virtual machines. This may take a moment to download everything needed for the environment to spin up.
Start the agents
In previous tutorials, you used a single agent in development mode to test Consul's functionality. However, you should never run development agents in production. In this tutorial, you will configure your first Consul agent to run in server mode instead, via the following command line flags. (In production you would provide these settings to consul in a configuration file.)
server
- Providing this flag specifies that you want the agent to start in server mode.-bootstrap-expect
- This tells the Consul server how many servers the datacenter should have in total. All the servers will wait for this number to join before bootstrapping the replicated log, which keeps data consistent across all the servers. Because you are setting up a one-server datacenter, you'll set this value to1
.-node
- Each node in a datacenter must have a unique name. By default, Consul uses the hostname of the machine, but you'll manually override it, and set it toagent-one
.-bind
- This is the address that this agent will listen on for communication from other Consul members. It must be accessible by all other nodes in the datacenter. If you don't set a bind address Consul will try to listen on all IPv4 interfaces and will fail to start if it finds multiple private IPs. Since production servers often have multiple interfaces, you should always provide a bind address. In this case it is172.20.20.10
, which you specified as the address of the first VM in your Vagrantfile.data-dir
- This flag tells Consul agents where they should store their state, which can include sensitive data like ACL tokens for both servers and clients. In production deployments you should be careful about the permissions for this directory.config-dir
- This flag tells consul where to look for its configuration. You will set it to a standard location:/etc/consul.d
.
Start Consul server
Once the environment is up, ssh into the first node, n1
, to begin the
configuration of your datacenter.
Start your first Consul agent by running the following command. Consul will start up in the foreground of your terminal window.
Note
Do not copy the vagrant@n1:~$
when copying the command. This
special prompt reminds you that you are ssh-ed into the first virtual machine.
Start Consul client
Open a new terminal window and move into consul-getting-started-join
directory.
Then ssh into your second virtual machine.
Now start up your second Consul agent in client mode. You'll set the bind
address to the IP address of the second VM (172.20.20.11
, specified in the
Vagrantfile) and the name to agent-two
. Don't include the -server
flag and
the agent will start in client mode. Consul will run in the foreground of your
terminal.
Note
Do not copy the vagrant@n2:~$
when copying the command. This
special prompt reminds you that you are ssh-ed into the second virtual machine.
Check datacenter membership
Now you have two Consul agents running: one server and one client. The two agents still don't know about each other and each comprise their own single-node datacenters.
Verify this by ssh-ing into each VM and checking each agent's membership
information. You'll need to open a new terminal window and change into the
consul-getting-started-join
directory.
Check the membership of agent-two
.
Note
Do not copy the vagrant@n2:~$
when copying the command. This
special prompt reminds you that you are ssh-ed into the second virtual machine.
Open a new terminal window and change directories into consul-getting-started-join
.
Check the membership of agent-one
.
Note
Do not copy the vagrant@n1:~$
when copying the command. This
special prompt reminds you that you are ssh-ed into the first virtual machine.
Join the agents
You're now ready to create your multi-agent datacenter. Open the terminal
window where you are ssh-ed into the second VM, and run the consul join
command on the Consul client, giving it the bind address of the Consul server.
Note
Do not copy the vagrant@n2:~$
when copying the command. This
special prompt reminds you that you are ssh-ed into the second virtual machine.
In the same window, run consul members
again and you should get both agents
listed in the output.
Switch to the terminal window where your Consul server is running on the first VM, and you'll notice some log output indicating that agent two joined it.
Now switch to the terminal where your client is running on the second VM. You'll notice that the client had been throwing warnings and errors indicating that no servers were available. When the client learned about the server, it stopped throwing errors and synced its node information.
Consul clients can not function without a server. All datacenters must have at least one agent running in server mode for Consul to function correctly.
In datacenters with more than one server, more than half of the servers must be in communication with each other at all times for the datacenter to function correctly. This is called maintaining quorum. You can learn more about the quorum requirements of servers in the architecture documentation.
Switch to the window where you are ssh-ed onto the second VM and run
consul members
on the client. The client will also list both agents as members.
Tip
To join a datacenter, a Consul agent only needs to learn about one other existing member, which can be a client or a server. After joining the datacenter, the agents automatically gossip with each other to propagate full membership information.
Notes on auto-join
In production, new Consul agents should automatically join the datacenter without human intervention. You can configure Consul to automatically discover new agents in AWS, Google Cloud or Azure by adding the relevant cloud auto join object to your Consul configuration file. This will allow a new node to join the datacenter without any hard-coded configuration.
Alternatively, you can provide hard-coded addresses of known Consul agents to
new agents using the -retry-join
flag.
Query a node
You can query Consul agents using the DNS interface or HTTP API.
For the DNS API, the structure of the names is NAME.node.consul
or
NAME.node.DATACENTER.consul
. If the datacenter is omitted, Consul will only
search the local datacenter.
From the terminal window where you are ssh-ed into agent-one
, query the DNS
interface for the address of agent-two
.
The ability to look up nodes in addition to services is useful for system administration, in addition to service discovery. For example, knowing the address of the node to SSH into is as easy as making the node a part of the Consul datacenter and querying it.
Stop the agents
Stop both of your agents gracefully by either typing Ctrl-c
in the terminal
windows where they are running, or issuing the consul leave
command.
Clean up the environment
Vagrant will automatically stop and power down the virtual machines it created, remove their hard disks from your machine, and free up all of the disk space and RAM they consume.
It will not get rid of the directory you created or the Vagrant file it
contains, so if you would like to re-run this tutorial, all you need to do is issue
vagrant up
again from inside the consul-getting-started-join
directory.
Clean up your virtual environment by running the following command from within
the consul-getting-started-join
directory.
Next steps
In this tutorial, you set up a multi-agent Consul datacenter by joining two Consul agents, a server and a client. Get more hands-on experience with Consul's features with the following tutorials: