• HashiCorp Developer

  • HashiCorp Cloud Platform
  • Terraform
  • Packer
  • Consul
  • Vault
  • Boundary
  • Nomad
  • Waypoint
  • Vagrant
Vault
  • Install
  • Tutorials
  • Documentation
  • API
  • Try Cloud(opens in new tab)
  • Sign up
Consul Storage Backend

Skip to main content
5 tutorials
  • Vault with Consul Storage Reference Architecture
  • Vault Multi-Cluster Architecture Guide
  • Vault with Consul Storage Deployment Guide
  • Production Hardening
  • Vault High Availability with Consul

  • Resources

  • Tutorial Library
  • Certifications
  • Community Forum
    (opens in new tab)
  • Support
    (opens in new tab)
  • GitHub
    (opens in new tab)
  1. Developer
  2. Vault
  3. Tutorials
  4. Consul Storage Backend
  5. Vault with Consul Storage Deployment Guide

Vault with Consul Storage Deployment Guide

  • 11min

  • VaultVault
  • ConsulConsul

This guide applies to Vault versions 1.7 and above and Consul versions 1.8 and above.

This deployment guide outlines the required steps to install and configure a single HashiCorp Vault cluster as defined in the Vault with Consul Storage Reference Architecture on virtual machines (VMs) or bare-metal servers running a Debian- or RedHat-based Linux distribution.

This guide includes general guidance as well as specific recommendations for popular cloud infrastructure platforms.

For production: Integrated Storage native to Vault is now recommended rather than using Consul for Vault storage. Refer to the Vault with Integrated Storage Deployment Guide to deploy Vault with Integrated Storage.

Vault Enterprise: As of version 1.12.0, Vault Enterprise will no longer start up if configured to use a storage backend other than Integrated Storage or Consul. See the Release Notes.

Prerequisites

You must have already set up a Consul cluster to use for Vault storage according to the Consul Deployment Guide including ACL bootstrapping.

It is strongly recommended to deploy a dedicated Consul cluster for this purpose, as described in the Vault with Consul Storage Reference Architecture to minimize resource contentation on the storage layer.

The remaining steps should be followed on each Vault server in your cluster.

Install Vault and Consul

Pre-built Vault and Consul packages are available from the HashiCorp Linux Repository. In addition to the installing the appropriate binaries, the official packages seed Vault and Consul with a baseline configuration, a systemd service unit, and local vault and consul user accounts under which the corresponding services will be run.

Add the HashiCorp GPG key as a trusted package-signing key.

$ curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -

Add the official HashiCorp Linux repository.

$ sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"

Update the package index.

$ sudo apt update

Install Vault and Consul.

$ sudo apt install -y vault-enterprise consul-enterprise

NOTE: To install open source Vault or Consul, replace vault-enterprise with vault and consul-enterprise with consul.

Install yum-config-manager to manage your repositories.

$ sudo yum install -y yum-utils

Add the official HashiCorp Linux repository.

$ sudo yum-config-manager --add-repo https://rpm.releases.hashicorp.com/RHEL/hashicorp.repo

Install Vault and Consul.

$ sudo yum -y install vault-enterprise consul-enterprise

NOTE: To install open source Vault or Consul, replace vault-enterprise with vault and consul-enterprise with consul.

Install dnf config-manager to manage your repositories.

$ sudo dnf install -y dnf-plugins-core

Add the official HashiCorp Linux repository.

$ sudo dnf config-manager --add-repo https://rpm.releases.hashicorp.com/fedora/hashicorp.repo

Install Vault and Consul.

$ sudo dnf -y install vault-enterprise consul-enterprise

NOTE: To install open source Vault or Consul, replace vault-enterprise with vault and consul-enterprise with consul.

Install yum-config-manager to manage your repositories.

$ sudo yum install -y yum-utils

Add the official HashiCorp Linux repository.

$ sudo yum-config-manager --add-repo https://rpm.releases.hashicorp.com/AmazonLinux/hashicorp.repo

Install Vault and Consul.

$ sudo yum -y install vault-enterprise consul-enterprise

NOTE: To install open source Vault or Consul, replace vault-enterprise with vault and consul-enterprise with consul.

Prepare TLS certificates

NOTE: The installation package generates a self-signed TLS certificate for use by the Vault service to secure intra-cluster communication. While these certificates are usable for experimenting with getting Vault up and running, HashiCorp strongly recommends replacing them with certificates generated and signed by an appropriate CA.

You must have three files to configure TLS for Vault. Place them at these paths:

  • /opt/vault/tls/vault-cert.pem - The Vault TLS certificate itself. If this file was signed by an intermediate CA, append the certificate of that CA (and any other chained CAs) to the end of this file.
  • /opt/vault/tls/vault-key.pem - The private key of the Vault TLS certificate.
  • /opt/vault/tls/vault-ca.pem - The certificate of the CA root which signed the Vault TLS certificate.

Set the file ownership of the CA and certificate files to be owned by root.

$ sudo chown root:root /opt/vault/tls/vault-cert.pem /opt/vault/tls/vault-ca.pem

Set the file group ownership of the private key to allow the Vault service to read the file.

$ sudo chown root:vault /opt/vault/tls/vault-key.pem

Set the file permissions of the CA and certificate files to be world-readable.

$ sudo chmod 0644 /opt/vault/tls/vault-cert.pem /opt/vault/tls/vault-ca.pem

Set the file permissions of the private key file to be readable only by the Vault service.

$ sudo chmod 0640 /opt/vault/tls/vault-key.pem

Configure the Consul agent

Consul may be configured by editing the /etc/consul.d/consul.hcl file included with the installation package, or by creating additional .hcl files in the /etc/consul.d directory. A typical configuration file is given below, but the exact settings may vary significantly. Refer to the Consul Deployment Guide for more information on configuring the Consul agent to connect to the cluster deployed for Vault storage.

/etc/consul.d/consul.hcl
datacenter = "<DATACENTER_NAME>"
verify_incoming = true
verify_outgoing = true
verify_server_hostname = true
ca_file = "/etc/consul/tls/agent-ca-cert.pem"
auto_encrypt {
  tls = true
}
retry_join = [<CONSUL_SERVER_ADDRESSES>]

Enable and start the Consul service

Enable the systemd consul.service unit to allow the Consul service to start.

$ sudo systemctl enable consul.service

Start the Consul service.

$ sudo systemctl start consul.service

Check the status of the Consul service to ensure it is running.

$ sudo systemctl status consul.service

Create a Vault service ACL token

In order to coordinate high-availability and write to Consul storage, the Vault service requires a Consul ACL token.

Create a file, vault-service-policy.hcl, with the necessary ACL policy:

vault-service-policy.hcl
service "vault" { policy = "write" }
key_prefix "vault/" { policy = "write" }
agent_prefix "" { policy = "read" }
session_prefix "" { policy = "write" }

Create the ACL policy on the Consul cluster using a management token:

$ consul acl policy create -name vault-service -rules @vault-service-policy.hcl

Create an ACL token using that policy:

$ consul acl token create \
    -description "Vault Service Token" \
    -policy-name vault-service

Copy the SecretID that is output from the previous step, and use it in the Consul storage configuration for Vault described below.

Configure Vault

Vault may be configured by editing the /etc/vault.d/vault.hcl file included with the installation package. Refer to the Vault Configuration Overview for additional details about each setting.

Base configuration

The core required configuration values for Vault are cluster_addr, api_addr, and listener. The configuration below tells vault to advertise its local IPv4 address for internal communication, and to listen to TCP port 8200 on all network interfaces using the TLS configuration files specified.

/etc/vault.d/vault.hcl
cluster_addr  = "https://<LOCAL_IPV4_ADDRESS>:8201"
api_addr      = "https://<LOCAL_IPV4_ADDRESS>:8200"

listener "tcp" {
  address            = "0.0.0.0:8200"
  tls_cert_file      = "/opt/vault/tls/vault-cert.pem"
  tls_key_file       = "/opt/vault/tls/vault-key.pem"
  tls_client_ca_file = "/opt/vault/tls/vault-ca.pem"
}

Storage configuration

To configure Consul storage, use the following Vault configuration stanza using the Vault service ACL token created in a previous step.

/etc/vault.d/vault.hcl
storage "consul" {
  address = "http://localhost:8500"
  path = "vault/"
  token = "<VAULT_SERVICE_TOKEN>"
}

Auto-unseal

If you are using a supported cloud provider or have access to a supported hardware security module (HSM), you can enable auto-unseal using the seal configuration stanza.

To use AWS KMS for auto-unsealing securely, your EC2 instances should have appropriate permissions to a KMS key via their instance profile. See the awskms seal documentation for full details of the configuration options and necessary permissions.

/etc/vault.d/vault.hcl
seal "awskms" {
  region     = "<AWS_REGION>"
  kms_key_id = "<KMS_KEY_ARN>"
}

To use Azure Key Vault for auto-unsealing securely, your VMs should have a Managed Service Identity (MSI) with appropriate permissions to an Azure Key Vault. See the azurekeyvault seal documentation for full details of the configuration options and necessary permissions.

/etc/vault.d/vault.hcl
seal "azurekeyvault" {
  tenant_id  = "<AZURE_TENANT_ID>"
  vault_name = "<AZURE_VAULT_NAME>"
  key_name   = "<AZURE_KEY_NAME>"
}

To use GCP Cloud KMS for auto-unsealing securely, your compute instances should be bound to a service account with appropriate permissions to a GCP Cloud KMS Key Ring. See the gcpckms seal documentation for full details of the configuration options and necessary permissions.

/etc/vault.d/vault.hcl
seal "gcpckms" {
  project    = "<GCP_PROJECT_NAME>"
  region     = "<GCP_REGION>"
  key_ring   = "<GCP_KMS_KEY_RING>"
  crypto_key = "<GCP_KMS_KEY_NAME>"
}

NOTE: This section applies only to Vault Enterprise.

See the pkcs11 seal documentation for full details of the configuration options and necessary permissions.

/etc/vault.d/vault.hcl
seal "pkcs11" {
  lib            = "<PATH_TO_HSM_SHARED_LIBRARY>"
  slot           = "<HSM_SLOT_NUMBER>"
  pin            = "<HSM_PIN>"
  key_label      = "<HSM_KEY_LABEL>"
  hmac_key_label = "<HSM_HMAC_KEY_LABEL>"
}

Vault Enterprise license

NOTE: This section applies only to Vault Enterprise.

Beginning with Vault 1.8, an Enterprise license file must exist on disk and be readable by the Vault service.

Copy your Vault Enterprise license file to /opt/vault/vault.hclic. Then set the file's group ownership to allow the Vault service to read the file.

$ sudo chown root:vault /opt/vault/vault.hclic

Set the file permissions to allow the Vault service to read the file, but no other users to access the file.

$ sudo chmod 0640 /opt/vault/vault.hclic

Add the path to the Enterprise license file to /etc/vault.d/vault.hcl.

/etc/vault.d/vault.hcl
license_path = "/opt/vault/vault.hclic"

Vault web UI

To enable the Vault web UI, set the ui configuration value to true.

/etc/vault.d/vault.hcl
ui = true

Enable and start the Vault service

Enable the systemd vault.service unit to allow the Vault service to start.

$ sudo systemctl enable vault.service

Start the Vault service.

$ sudo systemctl start vault.service

Check the status of the Vault service to ensure it is running.

$ sudo systemctl status vault.service

Check the status of Vault

To ensure Vault is running, you can use the vault status command. To ensure TLS connection can be validated, first set the VAULT_CACERT environment variable to the path of the CA root certificate.

$ export VAULT_CACERT=/opt/vault/tls/vault-ca.pem

Then use vault status to see the status of the local Vault server.

$ vault status
Key                     Value
---                     -----
[...]
Storage Type            raft
[...]

Initialize Vault

If this is the first time you are running this cluster, Vault will need to be initialized on one host before Raft clients can be joined.

$ vault operator init
Unseal Key 1: <omitted>
[...]
Initial Root Token: s.AJ8gE4odM92zW1clYbPPXpLu
[...]

Take note of these values and store them securely. You will need them to manually unseal Vault and to perform initial setup.

Unseal Vault

If you configured auto-unseal, Vault should unseal automatically. Use the vault status command to check whether Vault is sealed.

$ vault status
Key                     Value
---                     -----
[...]
Sealed                  false
[...]

If Vault is sealed, use the vault operator unseal command on each host in the cluster with the unseal keys you received when initializing Vault to unseal it.

$ vault operator unseal
Unseal Key (will be hidden): ******************************
Key                Value
---                -----
[...]
Unseal Progress    1/3
[...]

Repeat the unseal command with new keys until the unseal progress is complete and Vault is unsealed.

Additional configuration

Audit Logging

Vault provides audit logging via Audit devices which serialize details of each request and response Vault handles. These logs may be stored to a file on disk, or sent to syslog or any socket endpoint.

Learn more about using audit logs to monitor and secure your Vault deployment with the Monitor Telemetry & Audit Device Log Data tutorial.

Replication

Vault Enterprise supports replication functionality to support performance and disaster recovery use cases. There are a number of tutorials and other documentation to explain the concepts and provide details about configuring these replication modes.

  • Vault Enterprise Replication
  • Vault Replication Internals
  • Setting Up Performance Replication
  • Disaster Recovery Repication Setup
  • Monitoring Vault Replication

Resource Quotas

Vault provides tools for establishing lease count limits (Enterprise only) and request rate limits to avoid systemwide denial-of-service. Learn more about the concepts and capabilities behind Vault resource quotas in the Resource Quotas concepts documentation, and discover how to enable and manage resource quotas in the Protecting Vault with Resource Quotas tutorial.

Telemetry

The telemetry stanza specifies various configurations for Vault to publish metrics to upstream systems. Vault supports sending telemetry to a number of systems including Circonus, DogStatsD, Prometheus, Stackdriver, Statsite, and any statsd-compatible endpoint.

Review the telemetry configuration section of our documentation for details on configuring Vault telemetry for your provider, and learn more about Vault's telemetry collection and which metrics are collected in the telemetry internals documentation.

You can also learn more about using telemetry to monitor Vault on our Monitor Telemetry & Audit Device Log Data tutorial.

Next Steps

  • Vault Production Hardening Guide
  • Vault Application Integration
  • Vault Upgrade Standard Procedure
 Previous
 Next

On this page

  1. Vault with Consul Storage Deployment Guide
  2. Prerequisites
  3. Install Vault and Consul
  4. Prepare TLS certificates
  5. Configure the Consul agent
  6. Enable and start the Consul service
  7. Create a Vault service ACL token
  8. Configure Vault
  9. Vault web UI
  10. Enable and start the Vault service
  11. Check the status of Vault
  12. Initialize Vault
  13. Unseal Vault
  14. Additional configuration
  15. Next Steps
Give Feedback(opens in new tab)
  • Certifications
  • System Status
  • Terms of Use
  • Security
  • Privacy
  • Trademark Policy
  • Trade Controls
  • Give Feedback(opens in new tab)