Understand Terminating Gateways

Consul service mesh provides encrypted communication over mTLS and provides security between applications using Consul intentions. As you migrate applications to the service mesh, applications in the mesh need to securely connect to services still outside the mesh.

Terminating gateways are egress proxies that provide connectivity to external destinations by terminating mTLS connections, enforcing Consul intentions, and forwarding requests to appropriate destination services.

By using terminating gateways you will only need to permit traffic originating from the gateway on your local network, and not manually authorize every service that needs connectivity to destinations outside the mesh.

This tutorial will provide an overview of the terminating gateways functionality and their use cases. In each use case section, you will gain an understanding of deployment best practices and examples of common deployment scenarios.

Prerequisites

Each terminating gateway needs:

• A local Consul client agent to manage its configuration.
• General network connectivity to services within its local Consul datacenter.

Terminating gateways also require that Consul datacenters meet the following minimum requirements:

• You must use Consul version 1.13.1 or newer.
• Consul Connect service mesh must be enabled on the datacenter's Consul servers.
• gRPC must be enabled on all client agents.

Currently, Envoy is the only proxy with terminating gateway capabilities in Consul. Terminating gateway proxies receive their configuration through Consul, which automatically generates the proxy configuration based on the gateway's registration. Consul can only translate terminating gateway registration information into Envoy's xDS configuration, therefore the proxies acting as terminating gateways must be Envoy.

Connect proxies that send upstream traffic through a gateway aren't affected when you deploy terminating gateways. If you are using non-Envoy proxies as service mesh sidecar proxies they will continue to work for traffic directed at services linked to a terminating gateway as long as the services upstreams can be discovered with the /health/connect endpoint.

Service mesh and terminating gateway architecture

Inside the mesh network, services are deployed with sidecar proxies. Proxies provide authentication and encryption for the services they represent. You can control service-to-service connections using Consul intentions. Consul intentions are enforced by the proxies and determine whether one service can connect to another.

Due to the central role of proxies in the service mesh, you need to deploy them on all service nodes or Kubernetes pods to take full advantage of the mesh functionalities. In most scenarios, the transition to a service mesh architecture will happen over time by incrementally moving services into the service mesh.

There are cases where it might not be possible to deploy a sidecar proxy for a service. For example, the service might be running on an unsupported operating system or might be running somewhere where access to the host is not available, as in the case of a managed database.

Terminating gateways are a solution to uniformly discover and connect to services residing outside the service mesh, while still providing some of the benefits of a service mesh such as observability, encrypted communication, and access policies governed by intentions. When using terminating gateways, you can introduce Consul service mesh to hybrid deployments that would otherwise require alternate security and policy enforcement mechanisms.

Use case: one or more services outside the service mesh

The simplest use case for terminating gateways is routing to external services in the same logical network as the Consul service mesh.

This use case represents scenarios where there are still services that cannot be added to the Consul service mesh, but are located on a node with a local Consul client or are registered with a local Consul client agent in the same datacenter. The service must be registered with Consul so that it can be routed to, and secured with Consul intentions.

Other supported use cases include:

• A managed service, such as Amazon RDS or a third-party service, that needs to be integrated into the service mesh. In this scenario, you will not be able to install Consul on the machine and you will only be able to access the service via the endpoint(s) it exposes.
• A legacy service running on an unsupported OS, or an embedded device that is still present in your infrastructure and where hardware or OS limitations do not permit you to pair it with a sidecar proxy.
• Hybrid applications transitioning towards the service mesh that require integration testing or are required to maintain compatibility with other legacy systems.

In these situations, given the heterogeneous nature or location of these services, the gateways and the external services would be registered in the same Consul datacenter catalog. Traffic from services in the mesh would flow through their sidecar to the terminating gateway. The gateway will then forward the traffic, potentially unencrypted, to the destination service.

To terminate TLS connections, the terminating gateway will present leaf certificates for the services it represents. Internal mesh traffic to the gateway will be encrypted with mTLS and controlled by intentions as expected with Consul service mesh. Intentions reference the source and destination services and do not require knowledge of the gateway itself.

The routing between gateways and external services is determined by the terminating-gateway configuration entry. The configuration is associated with the name of a gateway service and will apply to all instances of gateways with that name across all federated Consul datacenters.

Gateways are registered by name, and a config entry determines which external services each gateway service is responsible for. Spinning up multiple gateway instances with the same name will add redundancy with no additional configuration.

Use case: many services in a shared legacy datacenter

Terminating gateways can also be useful when you have multiple logical network segments residing in multiple Consul service meshes that still depend on having access to services residing in a shared legacy datacenter. These services might have the same limitations listed in previous scenarios or might be subject to internal policies that do not permit them to be migrated or for the configuration of the nodes they run on to be modified.

In these cases, you will be able to register services from the legacy datacenter with the service mesh by configuring a terminating gateway in each Consul datacenter that acts as a bridge to the shared legacy datacenter.

In this scenario the on-premises environment does not contain Consul clients or servers. Connectivity from the cloud environments to the on-premises datacenter is provided by a Virtual Private Network (VPN), direct connect, or equivalent networking solution.

The external, on-premises services are registered in each Consul datacenter so that the service mesh is aware the services are reachable through the gateways. The gateways terminate mTLS connections originated from services in the mesh, and rely on separate TLS connections initiated from the gateways, or encryption provided by the underlying network, in order to secure the connection to the on-premises environment.

An alternative deployment model is to deploy Consul servers to each legacy datacenter, making them their own Consul datacenter, and use Consul's mesh gateways to establish connectivity from the cloud to the terminating gateway which is deployed in the on-premises environment.

Intentions with terminating gateways

Gateways are not considered services and do not need to be considered when defining intentions. Intentions should reference the destination service rather than the gateway.

When creating intentions, the source will be the service in the mesh, and the destination will be the external service.

First, create a file for a config entry definition named intention-config.hcl.

Kind = "service-intentions"
Name = "legacy-backend"
Sources = [
  {
    Name   = "api"
    Action = "allow"
  }
]

From the same directory where you saved this file, run the following command on your terminal to initialize these intention rules.

$ consul config write intention-config.hcl
Config entry written: service-intentions/legacy-backend

Security considerations

Traffic inside the Consul service mesh is automatically mTLS encrypted. This is not true for traffic from a terminating gateway to an external service. By default traffic originating from the terminating gateway that is directed towards the legacy services is unencrypted. Consul will only encrypt connections to the gateway and not from the gateway to the destination service. This simplifies monitoring upstream traffic leaving the service mesh.

It is possible to configure the gateway to establish one-way or mutual TLS with external services if the external service supports it. Check the full documentation for all available options. By specifying a path to a CA file connections from the terminating gateway will be encrypted using one-way TLS authentication. If a path to a client certificate and private key are also specified, connections from the terminating gateway will be encrypted using mutual TLS authentication.

From the network point of view, you will have to ensure that the terminating gateway is able to reach all the external services it needs to communicate with, and make sure firewall rules are in place to only allow traffic from the gateway nodes to the external services.

ACLs with terminating gateways

Service mesh proxies in Consul require a token with service:write on the service they represent in order to be able to request leaf certificates from the Consul CA. Since terminating gateways may represent many services, they need a policy that grants service:write on all the services toward which they are expected to forward traffic. Following the principle of least privilege, ACL tokens used should only grant write on the services represented.

For Consul Enterprise, if the terminating gateway targets services in multiple namespaces, then the policy needs to be updated instead. This is because a role’s service identities can only target services within the role’s namespace.

Lastly, since terminating gateways can target all services in a namespace, Consul Enterprise users can also provide a token that grants service:read to all services in that namespace.

Redundancy considerations

The routing between terminating gateways and external services will be determined by centralized configuration. Terminating gateways will be registered by name, and a config entry will determine the set of external services which are accessible through each gateway. Spinning up multiple gateway instances with the same name will add redundancy with no additional configuration.

Health checks for external services

For many external services there may not be a Consul agent running on the same node. This means that you must manually register the external node, service, and checks with the /catalog/register endpoint. Since an agent isn’t present, you must register the nodes themselves, and then register one or more services on those nodes.

Next steps

In this tutorial you learned the main use cases for Consul terminating gateways. You also learned some best practices that should be applied when securing their configuration and preparing them for production.

Read more about terminating gateways in the Consul documentation.

More info on the configuration entry for the terminating gateways are also available in the Consul documentation.

Follow Connect External Services to Consul Service Mesh With Terminating Gateways tutorial to learn how to use terminating gateways to secure and route communication to an external service.