Forward DNS for Consul Service Discovery
By default, DNS is served from port 53. On most operating systems, this requires elevated privileges. Rather than running Consul with an administrative or root account, you can forward appropriate queries to Consul (running on an unprivileged port) from another DNS server or port redirect.
In this tutorial, you will learn how to configure the following DNS servers for DNS forwarding.
After configuring forwarding, you will test the configuration. Finally, you will learn how to troubleshoot some common errors.
Prerequisites
To complete this tutorial, you will need a running Consul agent. Consul should be running with default settings and serving DNS on port 8600.
By default, Consul does not resolve DNS records outside the .consul.
zone unless the
recursors configuration option
has been set. For example, if a Consul DNS reply includes a CNAME record pointing outside the .consul
top level domain (TLD), then the DNS reply will only include CNAME records by default. When recursors
is set and the upstream resolver is functioning correctly, however, Consul will try to resolve CNAMEs and include any records (e.g., A, AAAA, PTR) for them in its DNS reply.
For BIND, dnsmasq and Unbound, you will need to configure /etc/resolv.conf
with a nameserver entry poining to localhost
. This configures the operating system to forward the request to the chosen DNS daemon.
systemd-resolved setup
You can use the systemd-resolved
service to resolve local application names in your network. To use the service, configure systemd-resolved
to send .consul
domain queries to Consul by creating consul.conf
file located in the /etc/systemd/resolved.conf.d/ directory.
Add the following settings to your resolved configuration file:
The main limitation with this configuration is that systemd 245 and older does not support configuring port numbers in the DNS field. You can either configure Consul to listen on port 53
instead of 8600, or map port 53 to 8600 using iptables
.
Binding to port 53 usually requires running systemd-resolved
as a privileged user or running Linux with the
CAP_NET_BIND_SERVICE capability. If you are using the Consul Docker image, then you will need to add the following to the environment to allow Consul to use the port: CONSUL_ALLOW_PRIVILEGED_PORTS=yes
The following iptables
commands are sufficient to map the ports.
The above configuration assumes Consul's DNS server is listening on the loopback address. If Consul is not listening on the loopback IP, replace the references to 'localhost' and '127.0.0.1' with the appropriate IP address for your environment.
Note
PTR record queries will still be sent out to the other configured resolvers, in addition to Consul.
After creating the resolved configuration, restart systemd-resolved
.
Validate the systemd-resolved configuration
Validate that systemd-resolved has restarted and is configured to forward queries to Consul.
Confirm that /etc/resolv.conf
points to the stub-resolv.conf
file managed by
systemd-resolved, and that the IP address for systemd-resolved's stub resolver
is the configured nameserver
.
Ensure that the operating system can resolve DNS queries to the .consul
domain.
Using any local resolver with systemd
By default, the local resolver stub in the resolved.conf
file is configured to listen for UDP and TCP requests at 127.0.0.53:53, but you can set the DNSStubListener
option to false
, which disables the stub. As a result, your system will be able to use any DNS configuration as long as it loads earlier than resolved
.
Disabling the local resolver stub can also solve other DNS configuration issues. See Troubleshooting for additional information.
BIND setup
In this example, BIND and Consul are running on the same machine.
Note
If your distribution uses systemd, you may need to disable systemd-resolved
prior to following this example. Refer to the guidance in the systemd-resolved instructions for additional information.
First, disable DNSSEC so that Consul and BIND can communicate. The following example shows a BIND configuration with DNSSEC disabled.
Zone file
Next, set up a zone for your Consul managed records in consul.conf
.
Dnsmasq setup
Note
If your distribution uses systemd, you may need to disable systemd-resolved
prior to following this example. Refer to the guidance in the systemd-resolved instructions for additional information.
Dnsmasq is typically configured via a dnsmasq.conf
or a series of files in
the /etc/dnsmasq.d
directory. Add the following settings to your dnsmasq configuration file
(e.g., /etc/dnsmasq.d/10-consul
):
Restart the dnsmasq
service after creating the configuration.
You can configure additional settings in dnsmasq
that may be useful in your environments. See
dnsmasq(8)
for additional details.
Unbound setup
Note
If your distribution uses systemd, you may need to disable systemd-resolved
prior to following this example. Refer to the guidance in the systemd-resolved instructions for additional information.
Unbound is typically configured in a unbound.conf
file or a series of files in
the /etc/unbound/unbound.conf.d
directory. Add the following settings to your Unbound configuration file
(e.g., /etc/unbound/unbound.conf.d/consul.conf
):
You may have to add the following line to the bottom of your
/etc/unbound/unbound.conf
file for the new configuration to be included.
iptables setup
The rules must be set on the same host as the Consul instance. Relay hosts should not be on the same host, otherwise the redirects will intercept the traffic.
On Linux systems that support it, incoming requests and requests to
the local host can use iptables
to forward to ports on the same machine
without using a secondary service. The recursors
flag is especially important for resolving other domains with iptables
.
This is because Consul only resolves the .consul
TLD by default.
Recursors should not include the local host address because the iptables
redirects would intercept the requests.
The iptables
method is suitable for situations where an external DNS
service is already running in your infrastructure and is used as the
recursor. It is also a suitable method if you want to use an existing DNS server as your query
endpoint and forward requests for the consul
domain to the Consul
server. In both cases, you may want to query the Consul server without the overhead of running a separate service on the Consul host.
macOS setup
On macOS systems, you can use the macOS system resolver to point all .consul
requests to Consul.
Documentation for this feature is available via: man 5 resolver
.
You must add a resolver entry in /etc/resolver/
to point at Consul. If you do not have this folder, create it.
To configure a resolver entry, you will need sudo/root access. Create a new file /etc/resolver/consul
,
and add the following text to the file:
The configuration informs the macOS resolver daemon to forward all .consul
TLD requests to 127.0.0.1 on port 8600.
Query Consul DNS and BIND
First, add a service registration to your Consul agent. You can use the following example service configuration file and registration command.
To ensure that Consul DNS and your DNS server are configured correctly, use a dig
query for the service you've registered.
Next, run the same query against your BIND instance and make sure you get a valid result:
If desired, verify reverse DNS using the same method:
Troubleshooting
If you don't get an answer from your DNS server (e.g., BIND, Dnsmasq) but you do get an answer from Consul, turn on your DNS server's query log to check for errors.
For BIND:
The log may show errors like this:
This indicates that DNSSEC is not disabled properly.
If you see errors about network connections, verify that there are no firewall or routing problems between the servers running BIND and Consul.
For Dnsmasq, see the log-queries
configuration option and the USR1
signal.
DNS forwarding may fail if you use the default systemd-resolved
configuration and attempt to bind to 0.0.0.0. The default configuration uses a DNS stub that listens for UDP and TCP requests at 127.0.0.53. As a result, attempting to bind to 127.0.0.53 conflicts with the running stub. You can disable the stub as described in the Using any local resolver with systemd section to troubleshoot this problem.
Next steps
In this tutorial, you configured DNS forwarding for Consul DNS and your DNS server. Now that you have configured your environment, you can start using Consul for service discovery or service mesh.
- Ensure only healthy service instances are available in Consul DNS.
- Deploy services into a zero trust network with Consul service mesh.