Connect External Services to Consul With Terminating Gateways

The Understand Terminating Gateways tutorial provides an overview of Consul's terminating gateways and their use cases.

In this tutorial you will configure a terminating gateway to secure and route communication to an external service. The external service will be in the same logical network as the service deployed in the Consul service mesh.

This use case represents scenarios where there are services that cannot be deployed with sidecar proxies to the Consul service mesh (e.g. services running on an OS not supported by Consul or managed services, such as Amazon RDS) but are registered within Consul's catalog as external services. The service must be registered with Consul so that it is discoverable with Consul DNS. Once registered, you can create Consul intentions for it like any other service.

Prerequisites

In order to deploy and test Consul terminating gateways, you will need the following resources deployed in a non-production environment.

• A node hosting a legacy service. Consul does not need to be installed on this node.

• Consul service mesh. A Consul datacenter with Consul service mesh functionality enabled. Check Secure Service-to-Service Communication for configuration help. To test the terminating gateways functionality, you will need at least two client nodes inside the service mesh:

  • A node hosting a dashboard service that needs to communicate with a legacy service.
  • A node for the terminating gateway. This node requires unrestricted and verified network connectivity between its host and the legacy service’s host.

The following binaries are required on the different nodes:

Configure a terminating gateway for your service mesh

In this tutorial you will make an external service available inside your service mesh by performing the following steps:

1. Register the external node and service with the Consul catalog.
2. Register a client service inside the service mesh and starting an Envoy proxy as sidecar.
3. Enable zero-trust networking with intentions. This is not strictly required for the functionality to work but is a recommended best practice.
4. Configure and start the terminating gateway.
5. Start the destination service and checking connectivity.

Register the external counting service

External services require you to also register the external node to be properly handled by Consul. The external node and service must be first registered in Consul before they can be reached through the terminating gateway.

Create a file named external.json, on the Consul client, with the following contents.

{
  "Node": "legacy_node",
  "Address": "10.20.10.22",
  "NodeMeta": {
    "external-node": "true",
    "external-probe": "true"
  },
  "Service": {
    "ID": "counting1",
    "Service": "counting",
    "Port": 9003
  }
}

This configuration defines a node named legacy_node available at address 10.20.10.22 that provides a counter service with an ID of counting1 running on port 9003.

Register the external node and service using a PUT request to Consul’s /catalog/register HTTP API exposed on any Consul agent.

$ curl --request PUT --data @external.json localhost:8500/v1/catalog/register

Query the /catalog/service/counting endpoint to verify that the external service registered with the catalog.

$ curl localhost:8500/v1/catalog/service/counting

[
 {
   "ID": "",
   "Node": "legacy_node",
   "Address": "10.20.10.22",
   "Datacenter": "dc1",
   ## ...
   "NodeMeta": {
     "external-node": "true",
     "external-probe": "true"
   },
   ## ...
   "ServiceID": "counting1",
   "ServiceName": "counting",
   ## omitted
   "ServicePort": 9003,
   ## ...
 }
]
## ...

Register the dashboard service and start its Envoy sidecar

Register the dashboard service on the Consul client. Create a file named dashboard.hcl with the following content.

services {
 name = "dashboard"
 port = 9002
 connect {
   sidecar_service {
     proxy {
       upstreams {
         destination_name = "counting"
         local_bind_port = 5000
       }
     }
   }
 }
}

This configuration defines a service named dashboard running on port 9002. The dashboard service is enabled for Consul service mesh, and has an upstream dependency on the destination service counting. The local sidecar proxy will make this destination available on localhost (127.0.0.1) at port 5000.

Register the service with the local Consul client agent.

$ consul services register dashboard.hcl

Registered service: dashboard

Start the local sidecar proxy for the client service.

$ consul connect envoy -sidecar-for dashboard > dashboard-proxy.log &

Finally, if you did not start the dashboard service yet, start it now.

$ PORT=9002 COUNTING_SERVICE_URL="http://localhost:5000" ./dashboard-service_linux_amd64 &

Enable zero-trust networking with intentions

The first intention you will create changes from the default-allow approach where all traffic is allowed unless denied in specific rules to a default-deny approach where all traffic is denied and only specific connections are enabled.

Kind = "service-intentions"
Name = "*"
Sources = [
  {
    Name   = "*"
    Action = "deny"
  }
]

$ consul config write intention-config.hcl
Config entry written: service-intentions/*

Register and start the terminating gateway

On the host where you will deploy the terminating gateway, create a file named terminating-gateway.hcl with the following contents.

Kind = "terminating-gateway"
Name = "legacy-services-gateway"
Services = [
 {
   Name = "counting"
 }
]

This file defines configuration for a terminating gateway named legacy-services-gateway which acts as an egress proxy for traffic destined to the service named counting.

Apply the terminating gateway config entry in Consul.

$ consul config write terminating-gateway.hcl

Now that you have configured your terminating gateway, start it with the Consul CLI.

$ consul connect envoy -gateway=terminating -register -service legacy-services-gateway -address '{{ GetInterfaceIP "eth0" }}:8443' >  terminating_gateway.log &

Start the legacy service and allow connectivity

You can now start the legacy counting service on the legacy node. In a more production oriented scenario this could represent your network team opening connectivity on the firewall from the terminating gateway host(s).

$ PORT=9003 ./counting-service_linux_amd64 &

Open a browser and navigate to http://dashboard.node.dc1.consul:9002.

Because of the zero-trust networking approach the connection indicator in the dashboard ui will show the word "Disconnected". This means you need to explicitly enable the connection across the two services.

Terminating gateways are not considered services and do not need to be considered when defining intentions. Destination services need to be defined in the intention instead of the gateway.

Modify your intentions to allow communication to the counting service.

First, create a file for a config entry definition named intention-counting.hcl.

Kind = "service-intentions"
Name = "counting"
Sources = [
  {
    Name   = "dashboard"
    Action = "allow"
  }
]

From the same directory where you saved this file, run the following command on your terminal to initialize these intention rules.

$ consul config write intention-counting.hcl
Config entry written: service-intentions/counting

Refresh your browser, the indicator in the top right that should turn green and show the word "Connected" when the dashboard service is in communication with the counting service.

Next steps

In this tutorial you enabled a service inside your service mesh to reach an external service using the terminating gateway functionality. You registered the external service and node using the /catalog/register API endpoint, you used global configuration to add a terminating-gateway configuration entry, and started the terminating gateway sidecar proxy. Finally you enabled connectivity to the external service using intentions.

Please check the Register External Services with Consul Service Discovery tutorial to learn how to monitor your external service from Consul using checks and Consul ESM.

Check the Consul documentation for more info on terminating gateways.