Custom Resource Definitions (CRDs) for Consul on Kubernetes
This topic describes how to manage Consul configuration entries via Kubernetes Custom Resources. Configuration entries provide cluster-wide defaults for the service mesh.
Requirements
- consul-helm 0.28.0 or later
- consul-k8s 0.22.0 or later
- consul 1.8.4 or later; some configuration entries require a newer version of Consul
Supported Configuration Entries
You can specify the following values in the kind
field. Click on a configuration entry to view its documentation:
Mesh
(requires Consul 1.10.0+)ExportedServices
EnterpriseProxyDefaults
ServiceDefaults
ServiceSplitter
ServiceRouter
ServiceResolver
ServiceIntentions
(requires Consul 1.9.0+)IngressGateway
TerminatingGateway
Installation
Verify that the minimum version of the helm chart (0.28.0
) is installed:
Update your helm repository cache if necessary:
Next, you must configure consul-helm via your values.yaml
to install the custom resource definitions
and enable the controller that acts on them:
Note that:
controller.enabled: true
installs the CRDs and enables the controller.- Configuration entries are used to configure Consul service mesh so it's also
expected that
connectInject
will be enabled.
See Install with Helm Chart for further installation instructions.
Upgrading An Existing Cluster to CRDs
If you have an existing Consul cluster running on Kubernetes you may need to perform extra steps to migrate to CRDs. See Upgrade An Existing Cluster to CRDs for full instructions.
Usage
Once installed, you can use kubectl
to create and manage Consul's configuration entries.
Create
You can create configuration entries via kubectl apply
.
See Configuration Entries for detailed schema documentation.
Get
You can use kubectl get [kind] [name]
to get the status of the configuration entry:
The SYNCED
status shows whether the configuration entry was successfully created
in Consul.
Describe
You can use kubectl describe [kind] [name]
to investigate the status of the
configuration entry. If SYNCED
is false, the status will contain the reason
why.
Edit
You can use kubectl edit [kind] [name]
to edit the configuration entry:
You can then use kubectl get
to ensure the change was synced to Consul:
Delete
You can use kubectl delete [kind] [name]
to delete the configuration entry:
You can then use kubectl get
to ensure the configuration entry was deleted:
Delete Hanging
If running kubectl delete
hangs without exiting, there may be
a dependent configuration entry registered with Consul that prevents the target configuration entry from being
deleted. For example, if you set the protocol of your service to http
via ServiceDefaults
and then
create a ServiceSplitter
, you won't be able to delete the ServiceDefaults
.
This is because by deleting the ServiceDefaults
config, you are setting the
protocol back to the default which is tcp
. Since ServiceSplitter
requires
that the service has an http
protocol, Consul will not allow the ServiceDefaults
to be deleted since that would put Consul into a broken state.
In order to delete the ServiceDefaults
config, you would need to first delete
the ServiceSplitter
.
Kubernetes Namespaces
Consul OSS
Consul Open Source (Consul OSS) ignores Kubernetes namespaces and registers all services into the same
global Consul registry based on their names. For example, service web
in Kubernetes namespace
web-ns
and service admin
in Kubernetes namespace admin-ns
will be registered into
Consul as web
and admin
with the Kubernetes source namespace ignored.
When creating custom resources to configure these services, the namespace of the
custom resource is also ignored. For example, you can create a ServiceDefaults
custom resource for service web
in the Kubernetes namespace admin-ns
even though
the web
service is actually running in the web-ns
namespace (although this is not recommended):
NOTE: If two custom resources of the same kind and the same name are attempted to be created in different Kubernetes namespaces, the last one created will not be synced.
ServiceIntentions Special Case
ServiceIntentions
are different from the other custom resources because the
name of the resource doesn't matter. For other resources, the name of the resource
determines which service it configures. For example, this resource configures
the service web
:
For ServiceIntentions
, because we need to support the ability to create
wildcard intentions (e.g. foo => * (allow)
meaning that foo
can talk to any service),
and because *
is not a valid Kubernetes resource name, we instead use the field spec.destination.name
to configure the destination service for the intention:
NOTE: If two ServiceIntentions
resources set the same spec.destination.name
, the
last one created will not be synced.
Consul Enterprise Enterprise
Consul Enterprise supports multiple configurations for how Kubernetes namespaces are mapped to Consul namespaces. The Consul namespace that the custom resource is registered into depends on the configuration being used but in general, you should create your custom resources in the same Kubernetes namespace as the service they're configuring and everything will work as expected.
The details on each configuration are:
Mirroring - The Kubernetes namespace will be "mirrored" into Consul, i.e. service
web
in Kubernetes namespaceweb-ns
will be registered as serviceweb
in the Consul namespaceweb-ns
. In the same vein, aServiceDefaults
custom resource with nameweb
in Kubernetes namespaceweb-ns
will configure that same service.This is configured via
connectInject.consulNamespaces
:Mirroring with prefix - The Kubernetes namespace will be "mirrored" into Consul with a prefix added to the Consul namespace, i.e. if the prefix is
k8s-
then serviceweb
in Kubernetes namespaceweb-ns
will be registered as serviceweb
in the Consul namespacek8s-web-ns
. In the same vein, aServiceDefaults
custom resource with nameweb
in Kubernetes namespaceweb-ns
will configure that same service.This is configured via
connectInject.consulNamespaces
:Single destination namespace - The Kubernetes namespace is ignored and all services will be registered into the same Consul namespace, i.e. if the destination Consul namespace is
my-ns
then serviceweb
in Kubernetes namespaceweb-ns
will be registered as serviceweb
in Consul namespacemy-ns
.In this configuration, the Kubernetes namespace of the custom resource is ignored. For example, a
ServiceDefaults
custom resource with the nameweb
in Kubernetes namespaceadmin-ns
will configure the service with nameweb
even though that service is running in Kubernetes namespaceweb-ns
because theServiceDefaults
resource ends up registered into the same Consul namespacemy-ns
.This is configured via
connectInject.consulNamespaces
:NOTE: In this configuration, if two custom resources of the same kind and the same name are attempted to be created in two Kubernetes namespaces, the last one created will not be synced.
ServiceIntentions Special Case (Enterprise)
ServiceIntentions
are different from the other custom resources because the
name of the resource doesn't matter. For other resources, the name of the resource
determines which service it configures. For example, this resource configures
the service web
:
For ServiceIntentions
, because we need to support the ability to create
wildcard intentions (e.g. foo => * (allow)
meaning that foo
can talk to any service),
and because *
is not a valid Kubernetes resource name, we instead use the field spec.destination.name
to configure the destination service for the intention:
In addition, we support the field spec.destination.namespace
to configure
the destination service's Consul namespace. If spec.destination.namespace
is empty, then the Consul namespace used will be the same as the other
config entries as outlined above.