Best Practices for Service Mesh Security
Consul service mesh enables secure service-to-service communication over mutual TLS. This provides both in-transit data encryption as well as authorization. This page will document how to secure the service mesh. To try service mesh locally, complete the Getting Started guide or for a full security model reference, see the dedicated Consul security model page. When setting up service mesh in production, review this tutorial.
Consul service mesh will function in any Consul configuration. However, unless the checklist below is satisfied, the service mesh is not providing the security guarantees it was built for. The checklist below can be incrementally adopted towards full security if you prefer to operate in less secure models initially.
Warning: The checklist below should not be considered exhaustive. Please read and understand the Consul security model in depth to assess whether your deployment satisfies the security requirements of Consul.
ACLs Enabled with Default Deny
Consul must be configured to use ACLs with a default deny policy. This forces all requests to have explicit anonymous access or provide an ACL token. The configuration also forces all service-to-service communication to be explicitly allowed via an allow intention.
To learn how to enable ACLs, please see the tutorial on ACLs.
If ACLs are enabled but are in default allow mode, then services will be able to communicate by default. Additionally, if a proper anonymous token is not configured, this may allow anyone to edit intentions. We do not recommend this. If ACLs are not enabled, deny intentions will still be enforced, but anyone may edit intentions. This renders the security of the created intentions effectively useless.
TCP and UDP Encryption Enabled
TCP and UDP encryption must be enabled to prevent plaintext communication
between Consul agents. At a minimum,
verify_outgoing should be enabled
to verify server authenticity with each server having a unique TLS certificate.
verify_incoming provides additional agent verification, but doesn't directly
affect service mesh since requests must also always contain a valid ACL token.
Clients calling Consul APIs should be forced over encrypted connections.
See the Consul agent encryption page to learn more about configuring agent encryption.
If encryption is not enabled, a malicious actor can sniff network traffic or perform a man-in-the-middle attack to steal ACL tokens, always authorize connections, etc.
Prevent Unauthorized Access to the Config and Data Directories
The configuration and data directories of the Consul agent on both clients and servers should be protected from unauthorized access. This protection must be done outside of Consul via access control systems provided by your target operating system.
The full Consul security model explains the risk of unauthorized access for both client agents and server agents. In general, the blast radius of unauthorized access for client agent directories is much smaller than servers. However, both must be protected against unauthorized access.
Prevent Non-Mesh Traffic to Services
For services that are using proxies (are not natively integrated), network access via their unencrypted listeners must be restricted to only the proxy. This requires at a minimum restricting the listener to bind to loopback only. More complex solutions may involve using network namespacing techniques provided by the underlying operating system.
For scenarios where multiple services are running on the same machine without isolation, these services must all be trusted. We call this the trusted multi-tenancy deployment model. Any service could theoretically connect to any other service via the loopback listener, bypassing the service mesh completely. In this scenario, all services must be trusted or isolation mechanisms must be used.
For developer or operator access to a service, we recommend using a local service mesh proxy. This is documented in the development and debugging guide.
If non-proxy traffic can communicate with the service, this traffic will not be encrypted or authorized via service mesh.
Restrict Access to Envoy's Administration Interface
Envoy exposes an unauthenticated administration interface that can be used to query and modify the proxy. This interface allows potentially sensitive information to be retrieved, such as:
- Envoy configuration
- TLS certificates
- List of upstream services and endpoints
We strongly advise only exposing the administration interface on a loopback address (default configuration) and restricting access to a subset of users.
If the administration interface is exposed externally, for
example by specifying a routable
address, it may be possible for a malicious actor to gain access to Envoy's
configuration, or impact the service's availability within the cluster.