HashiConf 2025 Don't miss the live stream of HashiConf Day 2 happening now View live stream
Consul Home

Configure multi-port services on Kubernetes

Enterprise

Multi-port service registration on Kubernetes requires Consul Enterprise. On Consul Community Edition, Consul registers only the first port as a single-port service.

This page describes how to register a Kubernetes Pod that exposes multiple ports as a single Consul multi-port service, and how to connect to it from other services in the mesh.

To add a Pod with multiple ports to the service mesh, create a single Kubernetes Service that exposes the named ports and register it with one service account. Consul registers the Pod as a single multi-port service that routes mesh traffic to each named port.

Requirements

  • Consul Enterprise
  • A Kubernetes Service that exposes each port on the workload with a unique name

Register the multi-port service

To register a Pod as a multi-port service, complete the following steps:

  1. Create a single service account for the service.
  2. Create a Kubernetes Service that exposes each port with a unique name.
  3. Create a Deployment that exposes the matching named container ports and enables injection with the consul.hashicorp.com/connect-inject annotation.

When you do not set the consul.hashicorp.com/connect-service-port annotation, Consul registers all exposed container ports as named ports on a single multi-port service and uses the first port as the default port.

The following manifest defines the web service account, Kubernetes Service, and Deployment. The web service exposes three named ports: api-port on 9090, metrics on 9091, and admin-port on 9092. The Deployment runs a single container that serves all three ports. A static-client service then connects to each port through the service mesh.

web.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: web-config
data:
  nginx.conf: |
    events {}
    http {
      server {
        listen 9090;
        location / {
          default_type text/plain;
          return 200 'Response from api-port 9090: Hello there!\n';
        }
      }
      server {
        listen 9091;
        location / {
          default_type text/plain;
          return 200 'Response from metrics port 9091: Hello again!\n';
        }
      }
      server {
        listen 9092;
        location / {
          default_type text/plain;
          return 200 'Response from admin port 9092: Hello again!\n';
        }
      }
    }
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: web
---
apiVersion: v1
kind: Service
metadata:
  name: web
spec:
  selector:
    app: web
  ports:
    - name: api-port
      port: 9090
      targetPort: 9090
    - name: metrics
      port: 9091
      targetPort: 9091
    - name: admin-port
      port: 9092
      targetPort: 9092
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: web
  name: web
spec:
  replicas: 1
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      annotations:
        'consul.hashicorp.com/connect-inject': 'true'
      labels:
        app: web
    spec:
      serviceAccountName: web
      volumes:
        - name: config-volume
          configMap:
            name: web-config
      containers:
        - name: nginx
          image: nginx:alpine
          volumeMounts:
            - name: config-volume
              mountPath: /etc/nginx/nginx.conf
              subPath: nginx.conf
          ports:
            - name: api-port
              containerPort: 9090
            - name: metrics
              containerPort: 9091
            - name: admin-port
              containerPort: 9092

Connect to the multi-port service

The way you connect to a multi-port service depends on whether transparent proxy mode is enabled or disabled.

Transparent proxy enabled

When transparent proxy mode is enabled, you address a specific port on the upstream service using the virtual DNS format <port-name>.<service-name>.virtual.consul. The following static-client manifest is an example of a service that connects to a multiport upstream via transparent proxy. Note that the only enabled Consul related K8s annotations for this service are connect-inject and transparent-proxy.

static-client.yaml

apiVersion: v1
kind: Service
metadata:
  # This name will be the service name in Consul.
  name: static-client
spec:
  selector:
    app: static-client
  ports:
    - port: 80
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: static-client
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: static-client
spec:
  replicas: 1
  selector:
    matchLabels:
      app: static-client
  template:
    metadata:
      name: static-client
      labels:
        app: static-client
      annotations:
        'consul.hashicorp.com/connect-inject': 'true'
        'consul.hashicorp.com/transparent-proxy': 'true'
    spec:
      containers:
        - name: static-client
          image: curlimages/curl:latest
          # Spin and wait forever; connect with `kubectl exec` to demonstrate the upstreams.
          command: ['/bin/sh', '-c', '--']
          args: ['while true; do sleep 30; done;']
      # If ACLs are enabled, the serviceAccountName must match the Consul service name.
      serviceAccountName: static-client

After you deploy static-client, connect to each port on web through its virtual address:

$ kubectl exec deploy/static-client -- curl --silent http://api-port.web.virtual.consul:9090
Response from api-port 9090: Hello there!

$ kubectl exec deploy/static-client -- curl --silent http://metrics.web.virtual.consul:9091
Response from metrics port 9091: Hello again!

$ kubectl exec deploy/static-client -- curl --silent http://admin-port.web.virtual.consul:9092
Response from admin port 9092: Hello again!

By default, Consul uses the first registered port as the service's default port to maintain backward compatibility for clients that dial the upstream service without a port-name prefix. To override this behaviour and mark a specific port as the default, use the consul.hashicorp.com/connect-service-default-port annotation on the upstream web Pod. The following annotations mark api-port as the default port:

annotations:
  'consul.hashicorp.com/connect-inject': 'true'
  'consul.hashicorp.com/connect-service-default-port': 'api-port'

Here is an example on how the service default port functions. If a service attempts to connect to web.virtual.consul, notably missing a port-name prefix in the hostname, it will be routed to the port defined in connect-service-default-port, which in this case is the api-port:

$ kubectl exec deploy/static-client -- curl --silent http://web.virtual.consul:9090
Response from api-port 9090: Hello there!

For more details about the annotations and labels that Consul on Kubernetes supports, refer to the annotations and labels reference.

Transparent proxy disabled

When transparent proxy mode is disabled, you must define each upstream in the consul.hashicorp.com/connect-service-upstreams annotation and provide the destination_port parameter per upstream. Applications then reach the upstream service by connecting to localhost on the desired destination port. The following static-client manifest has transparent proxy disabled and defines the three upstreams.

static-client.yaml

apiVersion: v1
kind: Service
metadata:
  # This name will be the service name in Consul.
  name: static-client
spec:
  selector:
    app: static-client
  ports:
    - port: 80
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: static-client
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: static-client
spec:
  replicas: 1
  selector:
    matchLabels:
      app: static-client
  template:
    metadata:
      name: static-client
      labels:
        app: static-client
      annotations:
        'consul.hashicorp.com/connect-inject': 'true'
        'consul.hashicorp.com/transparent-proxy': 'false'
        'consul.hashicorp.com/connect-service-upstreams': 'web.svc.default.ns.default.ap:9090:destination_port=api-port,web.svc.default.ns.default.ap:9091:destination_port=metrics,web.svc.default.ns.default.ap:9092:destination_port=admin-port'
    spec:
      containers:
        - name: static-client
          image: curlimages/curl:latest
          # Spin and wait forever; connect with `kubectl exec` to demonstrate the upstreams.
          command: ['/bin/sh', '-c', '--']
          args: ['while true; do sleep 30; done;']
      # If ACLs are enabled, the serviceAccountName must match the Consul service name.
      serviceAccountName: static-client

After you deploy static-client, connect to each upstream on its local port:

$ kubectl exec deploy/static-client -- curl --silent http://localhost:9090
Response from api-port 9090: Hello there!

$ kubectl exec deploy/static-client -- curl --silent http://localhost:9091
Response from metrics port 9091: Hello again!

$ kubectl exec deploy/static-client -- curl --silent http://localhost:9092
Response from admin port 9092: Hello again!

Caveats for multi-port services

Consider the following caveats when you register a multi-port service:

  • Multi-port service registration requires Consul Enterprise. On Consul Community Edition, Consul registers only the first port as a single-port service.
    Enterprise
  • All ports of a multi-port service must use the same protocol. Consul does not support a combination of protocols for a single multi-port service.
  • When transparent proxy mode is disabled, the Kubernetes annotation character limit constrains the number of upstreams you can define for a service.
  • Consul performs health checks for each Pod. When a health check fails for any container in the Pod, Consul marks the entire multi-port service as unhealthy.
  • Consul cluster peering is not supported.

For more information about multi-port service limitations and non-Kubernetes configuration options, refer to multi-port services. For the current cross-version, cross-platform support status of specific multiport capabilities, refer to multiport feature support.

Next steps

Edit this page on GitHub