Install Consul Snapshot agent on Kubernetes

This topic describes how to install the Consul Snapshot agent on Kubernetes using the official Helm chart. The Consul Snapshot agent runs as a service and takes snapshots of the state of the Consul servers. The snapshots are atomic and point-in-time. The Consul Snapshot agent can save these snapshots locally or push to optional remote storage services. Consul snapshots include, but are not limited to, the following:

Key/Value store entries
Service catalog registrations
Prepared queries
Sessions
Access Control Lists (ACLs)
Namespaces

You need the Consul Snapshot agent to automate taking the snapshots. You do not need the agent to restore a snapshot.

Enterprise

This feature requires Consul Enterprise(opens in new tab).

Requirements

You must run the Consul Snapshot agent within the Consul cluster, co-located with the Consul servers and inheriting the servers’ nodeSelector, tolerations, and affinity.

If the Consul ACL system is bootstrapped and active, you need to create a token for your Consul Snapshot agent. The following is a permissive example policy for a Consul Snapshot agent token. To learn how to create a token with this configuration, refer to Create a snapshot agent token.

# used for snapshot agent leader election
key_prefix "" {
  policy = "write"
}

# used for registering the snapshot agent service
service_prefix "" {
  policy = "write"
}

# used for snapshot agent leader election
session_prefix "" {
  policy = "write"
}

# used for registering the snapshot agent node
node_prefix "" {
  policy = "write"
}

acl = "write"

Configuration

The following is an empty example configuration for the Consul Snapshot agent. All fields are optional. For more information on each field, refer to the Consul Snapshot agent configuration file options.

consul-snapshot-agent.json

{
  "snapshot_agent": {
    "token": "",
    "datacenter": "",
    "ca_file": "",
    "ca_path": "",
    "cert_file": "",
    "key_file": "",
    "license_path": "",
    "tls_server_name": "",
    "login": {
      "auth_method": "",
      "bearer_token": "",
      "bearer_token_file": "",
      "meta": {}
    },
    "log": {
      "level": "INFO",
      "enable_syslog": false,
      "syslog_facility": "LOCAL0"
    },
    "snapshot": {
      "interval": "1h",
      "retain": 30,
      "stale": false,
      "service": "consul-snapshot",
      "deregister_after": "72h",
      "lock_key": "consul-snapshot/lock",
      "max_failures": 3,
      "local_scratch_path": ""
    },
    "backup_destinations": {
      "local_storage": [
        {
          "path": "."
        }
      ],
      "aws_storage": [
        {
          "access_key_id": "",
          "secret_access_key": "",
          "session_token": "",
          "iam_endpoint": "",
          "role_arn": "",
          "role_session_name": "",
          "web_identity_token_file": "",
          "sts_endpoint": "",
          "s3_region": "",
          "s3_endpoint": "",
          "s3_bucket": "",
          "s3_key_prefix": "consul-snapshot"
        }
      ],
      "azure_blob_storage": [
        {
          "account_name": "",
          "account_key": "",
          "container_name": ""
        }
      ],
      "google_storage": [
        {
          "bucket": ""
        }
      ]
    }
  }
}

Apply configuration as a Kubernetes Secret

The Consul Snapshot agent configuration may contains sensitive information, such as access keys for various cloud storage backends, as well as Consul ACL tokens. Therefore we recommend keeping this configuration in a Kubernetes secret. Use the following command to create a Kubernetes secret from a local configuration file:

Create a K8s secret

kubectl create secret generic <SECRET-NAME> --from-file=<SECRET-KEY>=<PATH-TO-SNAPSHOT-AGENT-CONFIG>

Install the Consul Snapshot agent using Helm

The Consul Snapshot agent runs as a sidecar with the Consul servers. Make sure you have the Consul Snapshot agent configuration present as a Kubernetes secret, and then modify the following Helm chart values accordingly.

If you are upgrading an existing Consul installation with the Consul Snapshot agent, use the helm upgrade command along with the modified Helm chart. If you are performing a new Consul deployment, refer to Deploy Consul Enterprise on Kubernetes.

consul-snapshot-agent.yaml

server:
    # snapshotAgent contains settings for setting up and running snapshot agents
    # within the Consul clusters. They are required to be co-located with Consul
    # clients, so will inherit the clients' nodeSelector, tolerations and affinity.
    # This is an Enterprise feature only. 
    snapshotAgent:
        enabled: true

        # replicas determines how many snapshot agent pods are created
        replicas: 2

        # configSecret references the Kubernetes secret that you manually created to
        # contain the entire config to be used on the snapshot agent.
        # Snapshot agent config details:
        # https://developer.hashicorp.com/consul/commands/snapshot/agent#config-file-options

        configSecret:
            secretName: consul-snapshot-agent
            secretKey: snapshot-agent-config

        # Resource settings for snapshot agent pods.
        resources:
            requests:
                memory: '200Mi'
                cpu: '100m'
            limits:
                memory: '200Mi'
                cpu: '100m'

        # Optional PEM-encoded CA certificate that will be added to the trusted system CAs.
        # Useful if using an S3-compatible storage exposing a self-signed certificate.
        # Example
        #   caCert: |
        #     -----BEGIN CERTIFICATE-----
        #     MIIC7jCCApSgAwIBAgIRAIq2zQEVexqxvtxP6J0bXAwwCgYIKoZIzj0EAwIwgbkx
        #     ...
        caCert: null