Upgrade Consul to v2.1.x

Review the version-specific requirements for upgrading to Consul 2.1.x. Follow the General Upgrade Process for the standard server and client upgrade sequence. On Kubernetes, also follow the Kubernetes upgrade process.

This page collects the upgrade notes for Consul 2.1.x. Each note in the following table links to a section with the required actions. The version-specific upgrade details page contains the full configuration examples and verification steps.

Upgrade notes at a glance

New experimental features in Consul 2.1.x are controlled by feature gates and are disabled by default. Upgrading does not enable them. The following changes can affect an existing deployment without any configuration change.

From Consul 2.0.x to 2.1.x

Requirements

Peered multiport routing is available in Consul Enterprise 2.1.0 and later. Before you enable it, upgrade every Consul server in both peered clusters, and the mesh gateways, client agents, and sidecar proxies that serve the exporting and consuming services, to Consul Enterprise 2.1.0 or later. Each cluster must have a mesh gateway, and the exporting service must be registered as multiport, exported to the peer, and allowed by a service intention.

Kubernetes multiport registration has a separate Helm chart switch. Set connectInject.multiportServiceRegistration.enabled to true; it defaults to false. Enabling the server feature gate does not enable multiport registration in the Kubernetes injector.

Pre-upgrade checks

1. Review the General Upgrade Process and plan the server and client upgrade sequence.
2. For peered multiport routing, confirm that both clusters can run Consul Enterprise 2.1.0 or later on every server, mesh gateway, client agent, and sidecar proxy involved in the traffic path. The feature gate remains disabled until all servers in each cluster meet the minimum version.
3. For Kubernetes workloads, set connectInject.multiportServiceRegistration.enabled to true in the Helm values before deploying Pods that register multiple ports. When a Pod exposes named ports across multiple containers, set consul.hashicorp.com/connect-service-port to a comma-separated list such as http,metrics. The injector derives its default from the first container only.
4. If you run API gateways that receive their configuration directly from Consul servers (Consul Dataplane without a client agent) and route to services that use service-router or service-resolver subsets, plan to enable the api-gateway-upstream-routing feature gate after the upgrade. Refer to API Gateway upstream routing.
5. If you use hybrid post-quantum TLS settings or a TLS 1.3 minimum, inventory incoming and outgoing mesh settings, agent defaults, and protocol overrides. Prepare compatible components before the upgrade can supply hybrid groups. Do not write ECDHCurves or tls_ecdh_curves to releases that do not support them. Refer to Hybrid post-quantum TLS.
6. For transparent proxy on VMs, Kubernetes, or ECS, confirm that hosts meet the nftables requirements.
7. Identify services with long-lived connections and plan a maintenance window. Refer to Proxies close established connections once.
8. Find API gateways that set TLS.Enabled. Refer to API gateway TLS.Enabled enables HTTPS termination.
9. On Kubernetes with API gateways, check for TCPRoute resources and the installed Gateway CRD version. Refer to Kubernetes Gateway API v1.6 CRDs.

Enable peered multiport routing

The peering-multiport-upstreams feature gate is experimental and disabled by default. It enables named-port routing to services imported from cluster peers. Multiport services within one cluster and peer services that use only their default port do not require this gate.

After all servers in both clusters run Consul Enterprise 2.1.0 or later, enable the gate in both the exporting and consuming clusters:

$ consul operator feature set peering-multiport-upstreams enabled

Check the requested and effective state in each cluster:

$ consul operator feature get peering-multiport-upstreams

The feature_gates.bootstrap server configuration sets initial gate state only when you create a datacenter. For existing clusters, use the consul operator feature set command. For upstream and DNS configuration, refer to Multiport services across cluster peers.

For Kubernetes, enable the Helm chart switch separately:

connectInject:
  multiportServiceRegistration:
    enabled: true

When this value is false, the injector starts with -disable-multiport-registration=true and rejects Pods that select multiple service ports. Check the injector logs for multi-port service registrations will remain disabled when registration does not take effect.

Post-upgrade checks

1. For deployments affected by hybrid post-quantum TLS, confirm that proxies accept the TLS configuration and verify negotiated groups on fresh mesh and agent connections. Follow Verify negotiated key exchange; a successful request or TLS 1.3 handshake alone does not prove hybrid negotiation.
2. Confirm that every server, mesh gateway, client agent, and sidecar proxy in both clusters reports Consul Enterprise 2.1.0 or later.
3. Check the peering-multiport-upstreams gate in both clusters with consul operator feature get.
4. Inspect the exported service registration and confirm that its named ports appear in Ports. For a Kubernetes Pod with ports in different containers, confirm that the connect-service-port annotation lists every port name.
5. Test a peer-qualified DNS name for each port. For example, query http.api.virtual.dc2-peer.peer.consul. for the http port of the api service imported from dc2-peer. The name http.api.virtual.consul does not query the peer and returns no answer when no matching local service exists. Both http.api.virtual.dc2-peer.consul. and http.api.virtual.dc2-peer.peer.consul. resolve in the tested configuration. When you use curlimages/curl, include the trailing dot so the resolver treats the name as absolute.

When the gate is disabled, per-port virtual IPs can remain allocated and per-port DNS names can still resolve, but connections to those addresses reset. A successful lookup alone does not confirm that the feature gate is enabled or that traffic can pass.

Multiport registration and lookup considerations

• The injector derives its default port list from the first container in the Pod. If named ports are split across containers, explicitly set consul.hashicorp.com/connect-service-port to a comma-separated list such as http,metrics.
• Do not set both the single-port Port field and multiport Ports field in one service registration. The conflicting fields produce Cannot set both Port and Ports.
• A removed or nonexistent named port does not always produce an error. A DNS lookup for a removed port can return the service's default virtual IP after the service is no longer registered as multiport. A non-transparent upstream with a removed or nonexistent destination_port can also route to the default port and return a successful HTTP response.
• A single-port service can resolve a <port-name>.<service>.virtual... name to its base virtual IP. Do not rely on an NXDOMAIN response or a failed request to detect a misspelled or removed port name.
• Removing and re-adding ports can allocate new per-port virtual IPs. A previously observed address can later identify a different port; refresh cached addresses after changing a service's port registration.
• The warning that names a service, affected ports, and protocol is not present in the tested Consul binary. The related upstream warning is emitted on the consumer sidecar; do not use it to infer exporter-side discovery-chain behavior.

During validation, disabling the gate took about 5 seconds and enabling it took about 19 seconds. After an intention change, each route path converged separately in 0 to 9 seconds. Treat these as observed approximate timings, not service-level guarantees.

From Consul 1.21.x to 2.1.x

First, follow Upgrade Consul to v2.0.x. The 2.0.x documentation also describes the version-specific changes in 2.0.x, including the IBM Consul Enterprise license prerequisite. Then complete the applicable checks and upgrade steps in the 2.0.x to 2.1.x section.

Behavior changes

Feature gates

Consul 2.1 adds a datacenter-wide feature gate framework for experimental features. All gates are disabled by default, so upgrading does not change behavior until you enable a gate. Gates are per datacenter, and a gate takes effect only after every server in the datacenter runs Consul 2.1.0 or later.

During a rolling server upgrade, the feature gate API reports that the policy is not initialized until the last server runs 2.1.0. The consul operator feature get command and the HTTP API return an error with status code 503, and consul operator feature list returns no gates. This is expected. Retry after the upgrade completes.

On Kubernetes, the Helm chart runs a feature-gate job after each install and upgrade that sets the gates your Helm values enable.

For the CLI, API, and bootstrap configuration, refer to consul operator feature and Feature gates.

Hybrid post-quantum TLS

Hybrid post-quantum key exchange is not controlled by a feature gate. It changes TLS key exchange, not certificate signatures or certificate verification.

Consul 2.1 can automatically select hybrid key-exchange groups when TLS 1.3 is the effective minimum and the curve list is unset. A mesh direction with a minimum of TLS 1.3 and omitted or empty ECDHCurves selects X25519MLKEM768 and X25519. Agent protocols with an effective minimum of TLS 1.3 and no explicit or inherited tls_ecdh_curves select the same pair. An explicitly empty per-protocol agent string instead leaves group selection to Go TLS defaults, which can also include hybrid groups. Hybrid key exchange can therefore become active during the upgrade without a separate opt-in change. Actual negotiation depends on both peers; a connection that selects X25519 remains classical.

Review your TLS settings and prepare compatible Envoy builds, Consul Dataplane images, and agent clients before you upgrade. A classical fallback group does not make an unsupported hybrid group acceptable to an older TLS provider. Follow Prepare compatible components to plan the rollout order. On Kubernetes, also prepare the Helm chart, controller, and installed Mesh CRD; upgrading the server image alone is insufficient. Confirm that the installed schema exposes both spec.tls.incoming.ecdhCurves and spec.tls.outgoing.ecdhCurves.

Hybrid key exchange requires a TLS 1.3 handshake, not necessarily a TLS 1.3 minimum. The mesh TLS example sets both TLSMinVersion and TLSMaxVersion to TLSv1_3 in both directions. Raising the minimum rejects TLS 1.2-only peers, so confirm their compatibility first and remove explicit CipherSuites lists from directions whose minimum you raise to TLS 1.3. If you must retain TLS 1.2 compatibility, the mesh reference permits an explicit hybrid-plus-classical group list with a TLS 1.2 minimum; set the maximum to TLS 1.3 to allow hybrid handshakes. Consul does not promote an unspecified maximum, and setting only the maximum does not trigger automatic group selection.

To defer hybrid negotiation on components that support these fields, use explicit approved classical groups in both mesh directions and each affected agent protocol. Mesh ECDHCurves takes an array, such as ["X25519"]; agent tls_ecdh_curves takes a comma-separated string, such as "X25519". Inspect protocol overrides as well as tls.defaults. Removing or emptying a field does not reliably disable hybrid selection.

After upgrading, verify new, full TLS handshakes using Verify negotiated key exchange. Confirm X25519MLKEM768 where hybrid negotiation is required and test the intended classical fallback separately. Verify mesh traffic and agent HTTPS, gRPC, and internal RPC independently. Existing connections retain their negotiated parameters until they close.

Follow Migrate to post-quantum TLS for staged adoption and Roll back post-quantum TLS to return to classical key exchange. For upgrade behavior, refer to version-specific upgrade details.

API Gateway upstream routing

In Consul 2.0.x, an API gateway always builds each route from the backend service's discovery chain. Service-router rules are composed with the HTTPRoute matches, and service-resolver subsets are resolved. Consul 2.0.x has no feature gate for this behavior.

Consul 2.1.x adds the experimental api-gateway-upstream-routing feature gate. The gate is disabled by default and applies only to API gateways that receive their configuration directly from Consul servers, such as gateways that run with Consul Dataplane and no Consul client agent. API gateways that run with a Consul client agent always compose upstream routing, so the gate does not change their behavior.

When the gate is disabled, a gateway that receives its configuration directly from Consul servers does not compose backend service-router rules with HTTPRoute matches and does not preserve service-resolver subset definitions. Routes to services that use service-router entries can return HTTP 503 until you enable the gate. If your gateways routed to such services before the upgrade, enable the gate as soon as the upgrade completes.

To enable upstream routing for these gateways:

1. Upgrade every Consul server in the datacenter to Consul 2.1.0 or later. The gate stays ineffective, with the reason below-minimum-version, until every server meets the minimum version.

2. Enable the gate in each datacenter that has affected gateways:

   $ consul operator feature set api-gateway-upstream-routing enabled
   

3. Confirm that Effective is true:

   $ consul operator feature get api-gateway-upstream-routing
   

Consul rebuilds the configuration for affected gateways automatically when the effective gate state changes. You do not need to restart the gateways. To return to the legacy shape, set the gate to disabled.

The feature_gates.bootstrap server configuration sets initial gate state only when you create a datacenter. For existing clusters, use the consul operator feature set command. For more information, refer to Feature gates.

The api-gateway-upstream-routing gate is separate from the peering-multiport-upstreams feature gate. The latter controls named-port routing to services imported from cluster peers and does not affect API gateways.

API gateway options that require a completed upgrade

Consul 2.0 servers and client agents do not recognize the following API gateway options added in 2.1. They reject the options, ignore them, or apply them incorrectly.

Before you use these options:

1. Upgrade all Consul servers to 2.1.
2. If your API gateways run with Consul client agents, upgrade the client agent on each node that runs an API gateway. Gateways that run with Consul Dataplane receive their configuration from the servers.
3. Confirm that the Build column of consul members reports 2.1 for every server and gateway client agent.

On Kubernetes, the corresponding resources and annotations have the same requirement: the api-gateway.consul.hashicorp.com/listener-<sectionName>-protocol annotation, the RouteHeaderMatchInvertFilter resource, and the RouteUpstreamLimitsFilter passiveHealthCheck field. Apply them only after you upgrade Consul.

Before you downgrade, change http2 and grpc listeners back to http, remove Invert from routes, and expect passive health checks to stop being applied. Circuit-breaker limits such as MaxConnections continue to apply. For configuration examples and verification steps, refer to the version-specific sections on header match inversion, HTTP/2 and gRPC listener protocols, and passive health checks.

API gateway TLS.Enabled enables HTTPS termination

In Consul 2.0, TLS.Enabled = true in an api-gateway configuration entry has no effect unless a listener also references a certificate, an SDS certificate, or TLS parameters. The listener serves plain HTTP. In Consul 2.1, TLS.Enabled = true turns on TLS termination for every listener on the gateway, and listeners without a certificate terminate HTTPS with the gateway's service mesh leaf certificate.

After the upgrade, a listener that sets TLS.Enabled = true without a certificate serves HTTPS instead of plain HTTP, and plain HTTP clients fail to connect. For listeners that reference certificates, TLS connections whose SNI matches none of the certificates now complete with the gateway leaf certificate instead of being rejected.

Before you upgrade, find the gateways that set TLS.Enabled. For each one, either remove TLS.Enabled to keep plain HTTP, or keep it and prepare clients to connect over HTTPS and trust the Consul service mesh CA. On Kubernetes, only Gateway resources with the consul.hashicorp.com/tls-enabled annotation set TLS.Enabled. For the discovery command and examples, refer to API gateway TLS.Enabled enables HTTPS termination.

Proxies close established connections once

Consul 2.1 delivers the Connect leaf certificate and CA roots to sidecar proxies and terminating gateways over the Envoy secret discovery service (SDS) instead of in the listener configuration. API gateways with TLS listeners also receive a new leaf certificate that includes *.api-gateway.<domain> DNS names. Each affected proxy rebuilds its listener once when it first receives configuration from Consul 2.1. Envoy drains the old listener and closes the established connections on it when the drain time ends. Short-lived HTTP requests are not affected. Long-lived connections, such as AMQP, database, or gRPC streaming connections, close once and must reconnect.

The change happens when you upgrade the client agent that a proxy started with consul connect envoy connects to, or when Consul Dataplane connects to an upgraded server. The drain time is 600 seconds for proxies that you start with consul connect envoy, and 30 seconds for Consul Dataplane, unless you override it. Schedule the upgrade for affected services in a maintenance window and confirm that their clients reconnect automatically.

The xds_fetch_timeout_ms Envoy setting now also applies to SDS configuration. Consider a higher value if proxies with many upstreams time out while they wait for configuration. For verification steps, refer to Proxies close established connections once during the upgrade.

API gateway DNS registration

Consul 2.1 registers each service that a bound route targets in Consul DNS at <service>.api-gateway.<domain>. No action is required. After you upgrade all servers, the leader reconciles existing API gateways and builds the gateway-services mapping that DNS lookups use. Refer to API gateway service lookups.

Transparent proxy uses nftables

Consul 2.1 redirects transparent proxy traffic with nftables instead of iptables and ip6tables. Consul does not fall back to iptables, so hosts that do not meet the following requirements fail to set up traffic redirection:

• The nft binary is installed.
• The Linux kernel supports stateful NAT in nftables inet chains. Upstream Linux provides this support starting with version 5.2, and some distributions backport it to earlier kernel versions, such as RHEL 8 and derivatives.
• On Kubernetes, the nft and nsenter binaries are installed on every worker node, because the CNI plugin runs on the node rather than in a container.
• On ECS, container instances run Linux kernel 5.2 or later, and custom images that bundle the consul-ecs binary include the nftables package.
• Scripts that run iptables inside the official Consul container image use nft instead. The image includes nftables and no longer includes iptables.

The consul connect redirect-traffic command does not remove iptables rules that earlier Consul versions created. Before you enable transparent proxy on an existing VM, remove those rules to avoid double NAT or inconsistent traffic redirection. Refer to Transparent proxy on VMs for the cleanup script and Transparent proxy on Kubernetes for the Kubernetes requirements.

OIDC IdP logout

Consul Enterprise 2.1 can end a user's session at the OIDC identity provider (IdP) when the user logs out of a token that an oidc auth method created. IdP logout turns on automatically, without a configuration change, for auth methods whose provider advertises an https end_session_endpoint in its discovery document. After the upgrade, logging out of the Consul UI also opens the provider logout page in a new browser tab, and consul logout prints the provider logout URL. Sessions that started before the upgrade are not affected.

Consul 2.1 also adds the OIDCLogoutURL and OIDCPostLogoutRedirectURIs auth method parameters. Consul 2.0 servers do not recognize them and cannot load an auth method that sets them, so OIDC logins that use that auth method fail.

1. Upgrade all Consul servers before you set OIDCLogoutURL or OIDCPostLogoutRedirectURIs.
2. Upgrade the client agents that serve the Consul UI or that consul login connects to, and the consul CLI that users run. Consul 2.0 agents and CLIs do not return or save the provider logout URL, so logout destroys only the Consul token.
3. Ask CLI users to log in again with consul login -type=oidc. Consul saves the provider logout URL only for logins made with a Consul 2.1 CLI.

Before you downgrade to Consul 2.0, remove both parameters from your auth methods. Refer to OIDC logout.

Localized DNS

The localized-dns feature gate lets Consul Dataplane and the Envoy sidecar resolve *.virtual.consul domains locally. It is disabled by default, and until you enable it all DNS queries resolve through Consul. The gate stays disabled on servers that run a version earlier than 2.1.0 even when you enable it.

Upgrade the Consul servers before Consul Dataplane, and then roll out workloads with a standard rolling update. To roll back, roll back Consul Dataplane first and then the servers. Keep the servers on 2.1 until all Consul Dataplane instances are rolled back, because rolling back the servers first removes the listeners that Consul Dataplane routes to. Consul Dataplane falls back to the servers when a listener is absent. For requirements and steps, refer to Localized virtual DNS resolution.

Kubernetes changes

Kubernetes Gateway API v1.6 CRDs

Starting in Consul 2.1, the Standard API gateway uses the Kubernetes Gateway API v1.6 CRDs in the gateway.networking.k8s.io API group, which include TCPRoute. Before you upgrade, check whether you use TCPRoute and which Gateway CRD version your cluster has:

$ kubectl get tcproutes -A
$ kubectl get crd gateways.gateway.networking.k8s.io -o jsonpath='{.metadata.annotations.gateway\.networking\.k8s\.io/bundle-version}'

On OpenShift, use oc instead of kubectl. Then choose the flow for your platform:

• If the platform manages Gateway CRDs v1.6.0 or later and you use TCPRoute, keep the Standard API gateway. Set global.installK8sNetworkingCRDs to false and keep global.crds.enableTcpRoute set to true.
• If the platform manages Gateway CRDs earlier than v1.6.0 and you require TCPRoute, use the Consul API gateway in the consul.hashicorp.com API group. Set global.installK8sNetworkingCRDs and global.crds.enableTcpRoute to false, and global.crds.consulapi.enabled to true.
• On OCP 4.19 and later, the Standard API gateway controller does not watch TCPRoute even when the platform provides v1.6 or later CRDs. Use the Consul API gateway if the cluster uses TCPRoute.

The first helm upgrade in each flow sets global.generateManifests to true. After the manifests are generated and you back them up, set it to false for later upgrades. For the Helm values for every platform and OpenShift version combination, refer to Upgrade Gateway resource types.

New route filter CRDs

Consul on Kubernetes 2.1 adds the RouteUpstreamLimitsFilter and RouteHeaderMatchInvertFilter CRDs. When connectInject.enabled is true, the Helm chart installs and upgrades them. If you manage CRDs outside of the chart, install them before you create these resources. Verify the CRDs after the upgrade with kubectl get crd routeupstreamlimitsfilters.consul.hashicorp.com routeheadermatchinvertfilters.consul.hashicorp.com. Both filters require Consul and Consul Helm chart v2.1.0 or later.

Multiport registration

The Helm chart disables multiport service registration by default. Existing workloads that expose several container ports and do not set the consul.hashicorp.com/connect-service-port annotation keep running after the upgrade. When Kubernetes recreates their Pods, the injector rejects the new Pods. Refer to Enable peered multiport routing and Convert multi-port workloads during an upgrade.

Kubernetes clusters that share external servers

When global.acls.manageSystemACLs is true, the Helm chart creates Kubernetes auth methods on the Consul servers and updates them on every install and upgrade. If multiple Kubernetes clusters connect to the same Consul servers in the same admin partition and use the same auth method names, each Helm installation or upgrade overwrites the auth methods of the other clusters, including their Kubernetes API server address. consul-k8s components and service mesh workloads in the other clusters then cannot log in to Consul. Set global.acls.authMethod.name to a unique value in each cluster and set externalServers.k8sAuthMethodHost to that cluster's API server address. Refer to Configure auth methods for multiple Kubernetes clusters.

Kubernetes feature-gate job

The Helm chart runs a feature-gate job after each install and upgrade. The job waits for a Consul leader and sets only the gates that your Helm values enable. When your values enable no gates, it exits without changes but still runs. When Vault is your secrets backend and you set global.acls.manageSystemACLs or global.acls.bootstrapToken.secretName, you must set global.secretsBackend.vault.featureGateSetRole. Without it, the chart fails to render. Refer to Configure Vault access for the Consul feature-gate job.

Rollback

When you disable peered multiport routing, disable peering-multiport-upstreams in both the exporting and consuming clusters. A one-sided change leaves the peers in different feature states and can interrupt named-port traffic. If you also roll back Kubernetes multiport registration, set connectInject.multiportServiceRegistration.enabled to false after you stop deploying multiport Pods.

During validation, named-port requests failed for up to 35 seconds after disabling the gate and about 18 seconds after enabling it. Per-port route virtual hosts remained in the Envoy config_dump for about 30 seconds after disable, so an immediate configuration dump may still show them. The gate removes peering-specific per-port routing from mesh gateways; local .internal. per-port chains and clusters remain. Treat these timings and observations as approximate, not service-level guarantees.

Disabling the gate does not immediately remove per-port virtual IP allocations or DNS answers. Connections to those addresses reset while the gate is off. Verify gate state in both clusters and test traffic before you conclude that a rollback or re-enable has completed.

Rollback checklist for other 2.1 changes

Complete the applicable steps before you downgrade from Consul 2.1 to 2.0:

• Feature gates: Disabling a gate stops new behavior but does not remove resources that were created while it was on. For example, services with an ai block stay registered. Gate state is preserved across a rolling server downgrade and returns when all servers run 2.1 again.
• API gateway options: Change http2 and grpc listeners back to http, and remove Invert and PassiveHealthCheck from your configuration entries. On Kubernetes, remove the listener protocol annotations, the RouteHeaderMatchInvertFilter references, and the passiveHealthCheck settings.
• OIDC: Remove OIDCLogoutURL and OIDCPostLogoutRedirectURIs from your auth methods. Consul 2.0 servers cannot load an auth method that sets them.
• Localized DNS: Roll back Consul Dataplane first and then the servers.
• Post-quantum TLS: To return to classical key exchange while retaining Consul 2.1, follow Roll back post-quantum TLS and use explicit classical group lists. Preserve TLS and certificate verification, and verify fresh connections after applying the configuration. This configuration rollback does not establish that Consul 2.0 can read the resulting state; review binary downgrade considerations before changing versions.