Manage Consul with Kubernetes Custom Resource Definitions (CRDs)
Configuration entries are Consul resources used to define datacenter-wide configuration
defaults for various aspects of the service mesh, or to create service specific
configurations. As of Consul 1.9, configuration entries can be created as Custom Resource Definitions (CRDs), and managed using kubectl
.
In this tutorial you will:
- Learn how to work with config entries as Custom Resource Definitions (CRDs)
- Download the latest helm chart
- Install or upgrade Consul (with Helm or Consul K8S CLI) to enable CRDs
- Deploy an application workload
- Configure a service intention
- Modify a service intention
- Delete a service intention
Prerequisites
To complete this tutorial you will need:
- Access to a Kubernetes cluster (Minikube v1.10.1+, kind v0.8.1+, or cloud-based k8s)
- kubectl
- helm v3.2.1+ or Consul-K8S-CLI
Working with config entry Custom Resource Definitions (CRDs)
Prior to Consul 1.9, when using configuration entries with Consul in Kubernetes,
an operator would either need to exec
into a running container, or configure
a host that could interact with the datacenter using a local Consul binary.
In versions older than 1.9, configuration entries have to be managed with the Consul CLI,
the HTTP API, or provided to agents during startup as configuration files.
As of Consul 1.9, most configuration entries can be managed as Kubernetes Custom
Resource Definitions (CRDs). You can now define most configuration entries as YAML,
and register them with Consul using the familiar kubectl apply
command.
The configuration entries currently available as CRDs for Consul on Kubernetes are:
proxy-defaults
- controls proxy configurationservice-defaults
- configures defaults for all the instances of a given serviceservice-resolver
- matches service instances with a specific Connect upstream discovery requestsservice-router
- defines where to send layer 7 traffic based on the HTTP routeservice-splitter
- defines how to divide requests for a single HTTP route based on percentagesservice-intentions
- defines restrictions for specific service to service interactions
Download Helm chart
If you have not already done so, download the latest official consul-helm chart now.
Example output:
Verify chart version
To ensure you have version 0.40.0
of the Helm chart, search your local repo.
Example output:
If you do not see that you have the correct version of the chart locally, try updating your Helm repo.
Download sample code
This tutorial comes with sample code you can use to test CRDs. From the command line, clone the GitHub repository that contains the configuration you will use with this tutorial.
Next, change into the directory containing the repository you just cloned.
Now, checkout the tagged version verified for this tutorial.
Enable Consul CRDs
CRDs can be enabled in a Consul on Kubernetes datacenter by adding the following top level stanza to the Helm chart configuration file, and then installing or upgrading the datacenter using the chart with the updated configuration file.
No other changes to the Helm chart configuration is required.
Note
This feature is only available using the official consul-helm chart versions 0.25 and higher.
Create a values file
You can use the config.yaml
included in the git repo to create a minimal, unsecured,
development datacenter for testing this tutorial. You can review this configuration
below. Note, that this configuration is not suitable for production use. See the Secure Consul and Registered Services on Kubernetes tutorial
for instructions on how to configure a production datacenter.
Install Consul in your cluster
You can now deploy a complete Consul datacenter in your Kubernetes cluster using the official Consul Helm chart or the Consul K8S CLI.
Deploy a demo application workload
Now that you have a datacenter with CRDs enabled, you will deploy an
demo application to test the features. Deploy the demo application using the
kubectl apply
command.
HashiCups may take a minute or more to deploy and start. Run the command kubectl get pods --all-namespaces
to verify all resources were successfully created.
You can test the application by viewing the user interface. Do this by forwarding the frontend deployment's port 80 to your development host.
Note
Since the port forwarding process is running in the foreground, open a new terminal window, and navigate to the same directory to complete the rest of the tutorial.
Navigate to http://localhost:8081
in a browser window.
You should observe the following screen.
Configure a service intention
Also, new as of Consul 1.9, is the ability to define permissions based on application-layer (i.e. layer 7) request attributes such as the HTTP method, path, or the presence/absence of a header.
To create the service intention for this tutorial, you first need to define the protocols for the frontend and public-api services as HTTP because application aware intentions can only operate on HTTP services. To do so, you need to create ServiceDefaults config entries for the frontend and public-api services:
Now you will define a ServiceIntentions config entry that manages the traffic between the frontend and public-api services. ServiceIntentions can have only a single destination, but can have multiple sources, and each source can have its own permissions specified. For example, this configuration denies all traffic from the frontend service to the health route. The health route is only for internal use and could be abused by external users, so traffic from the frontend service to this exact path is denied.
Next, the permissions for the primary allow action are specified. In the HashiCups
application, it is required that the client passes an Authorization
header. Also,
the application only allows the GET, PUT, POST, and DELETE HTTP methods.
Finally, a catch-all deny action is specified. In this particular datacenter, there is no default deny policy defined, so you must define a deny action for this specific destination service.
By placing this at the end of the list, any traffic that doesn't match either of the previous actions will trigger the action, and be denied.
Putting it all together, your manifest should resemble the following:
Use kubectl
along with the file named service-intentions.yaml
in the repository
you downloaded earlier to register the service intention resource with Kubernetes.
Now you can interact with the service intention using kubectl
just like you can
any other kube-native resource. For example, you can get a list of all service
intentions with this command:
You can also inspect details about the service intention using kubectl describe
like so:
This configuration should allow traffic, because the frontend service passes
the required header and uses the GET HTTP method. Revisit the application in
a browser tab at http://localhost:8081
, and verify you can still visit the web page.
Modify a service intention
Next, modify the service intention you created a moment ago.
This script updates the service intention. It changes the name of the required header to a value the application does not provide. Review the file and notice the "allow" action has been altered. The header key "Authorization" has been changed to "api-token".
Use kubectl apply
to apply the modified configuration.
Now, navigate to http://localhost:8081
in a browser window. You should observe that you are no longer able to successfully visit the page. If you inspect the network call in the browser's developer tools, you should observe that the request was denied with the following error message:
View the service intention in the Consul UI
To access the Consul UI, forward the consul-server-0
pod's port 8500
to the
development host, so that you can access the Consul UI in a browser.
Now visit http://localhost:8500
in a new browser tab. Click on "Intentions" in the left navigation pane, and you will observe a list of intentions. The screen
will have one entry as illustrated in this screenshot.
Notice there is a badge next to the frontend
entry indicating that the service is App Aware
. Also, notice that there is a badge that says Managed by CRDs
. Click anywhere on the frontend
entry, and you will be redirected to a page that contains the service intentions configuration details.
Delete a service intention
Now, use kubectl delete
to remove the service intention.
If you navigate back to the "Intentions" screen in the Consul UI, you will see that the list is now empty.
Navigate to http://localhost:8081
in a browser window. You should once again be able to visit the page.
Next steps
In this tutorial you:
- Learned how to work with config entries as Custom Resource Definitions (CRDs)
- Downloaded the latest helm chart
- Installed/upgraded a cluster to enable Consul CRDs
- Deployed an application workload
- Configured a service intention
- Modified a service intention
- Deleted a service intention
To learn more about how to manage Consul config entries using Consul on Kubernetes visit the documentation.
To learn more about securing your service mesh, check out our Securing Agents and Registered Services tutorial.