Configure multi-port services

This page describes the process for integrating a service that uses multiple ports in a single container when running Consul on Kubernetes deployments. It includes example configurations to demonstrate an end-to-end deployment test of Consul's multi-port features.

Requirements

Registering multi-port services with Consul requires Kubernetes. Multi-port services are not supported on VM deployments.

Version requirements

Consul deployments that use the v2 catalog enabled must meet the following minimum version requirements:

• consul v1.18.0
• consul-k8s CLI v1.4.0 or hashicorp/consul Helm chart release v1.4.0
• consul-dataplane v1.4.0

To install or update the consul-k8s CLI, refer to install the latest version or upgrade the CLI.

The required version of Consul dataplanes deploy automatically when using the latest version of consul-k8s. Dataplane version is configured manually when you modify imageConsulDataplane in the Helm chart.

For more information about upgrading Helm charts, refer to Upgrade Helm chart version.

Annotation requirements

The examples on this page include the following required annotations in Kubernetes Deployment resources:

spec:
  template:
    metadata:
      annotations:
        "consul.hashicorp.com/mesh-inject": "true"
        "consul.hashicorp.com/transparent-proxy": "true"

These annotations inject Consul dataplanes and enable transparent proxy mode so that the services can curl each other on ports configured in the Kubernetes Service resource. Refer to annotations and labels for more information.

Enable the v2 catalog

To enable the v2 catalog and its support for multi-port services, set global.experiments: ["resource-apis"] and ui.enabled: false. The following example includes these parameters in a Helm chart with additional configurations for the Consul installation:

global:
  enabled: true
  name: consul
  image: hashicorp/consul:1.18.0
  datacenter: dc1
  tls:
    enabled: true
  acls:
    manageSystemACLs: true
  experiments: ["resource-apis"]
server:
  enabled: true
  replicas: 1
connectInject:
  enabled: true
ui:
  enabled: false

Then install Consul to your Kubernetes cluster using either the consul-k8s CLI or Helm.

$ consul-k8s install -config-file=values.yaml

$ helm install consul hashicorp/consul --create-namespace --namespace consul --version 1.4.0 --values values.yaml

Define the multi-port service

Consul's v2 catalog supports two new methods for registering services on the mesh in Kubernetes:

• Method 1: Register a Kubernetes service that select workloads which expose multiple ports
• Method 2: Register multiple Kubernetes Services that direct traffic to an explicit port on the same workload

These methods affect how the Services are addressed in Kubernetes.

Each tab in the following example contains a configuration that defines an api service using one of these methods. Both definitions schedule a Pod running two containers that each support traffic to one of the ports exposed by the Service. In Method 1, both services are addressed using api because both services are exposed by a single service. In Method 2, api and api-admin are defined as separate services and can be addressed using distinct names.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: api
---
apiVersion: v1
kind: Service
metadata:
  name: api
spec:
  selector:
    app: api
  ports:
    - name: api
      port: 80
      targetPort: api
    - name: admin
      port: 90
      targetPort: admin
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
spec:
  replicas: 1
  selector:
    matchLabels:
      app: api
  template:
    metadata:
      labels:
        app: api
      annotations:
        "consul.hashicorp.com/mesh-inject": "true"
        "consul.hashicorp.com/transparent-proxy": "true"
    spec:
      containers:
        - name: api
          image: hashicorp/http-echo:latest
          args:
            - -text="hello world"
            - -listen=:8080
          ports:
            - containerPort: 8080
              name: api
        - name: api-admin
          image: hashicorp/http-echo:latest
          args:
            - -text="hello world from 9090 admin"
            - -listen=:9090
          ports:
            - containerPort: 9090
              name: admin
      serviceAccountName: api

apiVersion: v1
kind: ServiceAccount
metadata:
  name: api
---
apiVersion: v1
kind: Service
metadata:
  name: api
spec:
  selector:
    app: api
  ports:
    - name: api
      port: 80
      targetPort: api
---
apiVersion: v1
kind: Service
metadata:
  name: api-admin
spec:
  selector:
    app: api
  ports:
    - name: admin
      port: 90
      targetPort: admin
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
spec:
  replicas: 1
  selector:
    matchLabels:
      app: api
  template:
    metadata:
      labels:
        app: api
      annotations:
        "consul.hashicorp.com/mesh-inject": "true"
        "consul.hashicorp.com/transparent-proxy": "true"
    spec:
      containers:
        - name: api
          image: hashicorp/http-echo:latest
          args:
            - -text="hello world"
            - -listen=:8080
          ports:
            - containerPort: 8080
              name: api
        - name: api-admin
          image: hashicorp/http-echo:latest
          args:
            - -text="hello world from 9090 admin"
            - -listen=:9090
          ports:
            - containerPort: 9090
              name: admin
      serviceAccountName: api

For testing purposes, the following example defines a Service to function as a static client that you can use to verify that the multi-port services function correctly.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: web
---
apiVersion: v1
kind: Service
metadata:
  name: web
spec:
  selector:
    app: web
  ports:
    - port: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 1
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
      annotations:
        "consul.hashicorp.com/mesh-inject": "true"
        "consul.hashicorp.com/transparent-proxy": "true"
    spec:
      containers:
        - name: web
          image: curlimages/curl:latest
          # Just spin & wait forever, we'll use `kubectl exec` to demo
          command: ['/bin/sh', '-c', '--']
          args: ['while true; do sleep 30; done;']
      serviceAccountName: web

To apply these services to your Kubernetes deployment and register them with Consul, run the following command:

$ kubectl apply -f api.yaml -f web.yaml

Configure traffic permissions

Consul uses traffic permissions to validate communication between services based on L4 identity. When ACLs are enabled for the service mesh, traffic permissions deny all services by default. In order to allow traffic between the static client and the multi-port service, create CRDs that allow traffic to each port.

The following examples create Consul CRDs that allow traffic to only one port of the multi-port service. Each resource separately denies web permission when it is a source of traffic to one of the services. These traffic permissions work with either method for defining a multi-port service. When following the instructions on this page, apply these permissions individually when you validate the ports.

apiVersion: auth.consul.hashicorp.com/v2beta1
kind: TrafficPermissions
metadata:
  name: web-to-api
spec:
  destination:
    identityName: api
  action: ACTION_ALLOW
  permissions:
    - sources:
        - identityName: web
      destinationRules:
        - portNames: ["api"]

apiVersion: auth.consul.hashicorp.com/v2beta1
kind: TrafficPermissions
metadata:
  name: web-to-admin
spec:
  destination:
    identityName: api
  action: ACTION_ALLOW
  permissions:
    - sources:
        - identityName: web
      destinationRules:
        - portNames: ["admin"] 

Validate multi-port connection

To open a shell to the web container, you need the name of the Pod it currently runs on. Run the following command to return a list of Pods:

$ kubectl get pods 
NAME                                           READY   STATUS    RESTARTS   AGE
api-5784b54bcc-tp98l                           3/3     Running   0          6m55s
web-6dcbd684bc-gk8n5                           2/2     Running   0          6m55s

Set environment variables to remember the pod name for the web workload for use in future commands.

$ export WEB_POD=web-6dcbd684bc-gk8n5

Apply traffic permissions

Use the web Pod's name to open a shell session and test the api service on both ports. When ACLs are enabled, these commands fail until you apply a traffic permissions resource.

$ kubectl exec -it ${WEB_POD} -c web -- curl api:80

$ kubectl exec -it ${WEB_POD} -c web -- curl api:90

$ kubectl exec -it ${WEB_POD} -c web -- curl api:80

$ kubectl exec -it ${WEB_POD} -c web -- curl api-admin:90

Apply the CRD to allow traffic to port 80:

$ kubectl apply -f allow-80.yaml 

$ kubectl exec -it ${WEB_POD} -c web -- curl api:80
hello world

$ kubectl exec -it ${WEB_POD} -c web -- curl api:80
hello world

Apply the CRD to allow traffic to port 90:

$ kubectl apply -f allow-90.yaml 

$ kubectl exec -it ${WEB_POD} -c web -- curl api:90
hello world from 9090 admin

$ kubectl exec -it ${WEB_POD} -c web -- curl api-admin:90
hello world from 9090 admin

Next steps

After applying traffic permissions and validating service-to-service communication within your service mesh, you can manage traffic between multi-port services, filter traffic between ports based on L7 header information, or direct match HTTP query parameters to a specific port.

Refer to the following pages for more information:

• Split traffic between services
• gRPC route example: route traffic by matching header
• HTTP route example: route traffic by matching header
• HTTP route example: route traffic by matching header and query parameter