Deploy HCP Vault Performance Replication with Terraform

15min
HCP
Terraform
Vault

This tutorial will demonstrate how to deploy a HashiCorp Cloud Platform (HCP) Vault cluster with performance replication enabled.

Because Vault provides the critical services of identity management, secrets storage, and policy management, this functionality is expected to be highly available. With HCP Vault you can deploy a performance replica in a separate region to achieve high availability.

NOTE: Performance replication is a HCP Vault Plus feature.

If you are not familiar with HCP Vault, please review the Getting Started with HCP Vault tutorials before you begin.

This tutorial builds off the concepts introduced Deploy HCP Vault with Terraform, Codify Management of HCP Vault, and HCP Vault Performance Replication.

Scenario introduction

In this tutorial, you are going to create an HCP Vault primary cluster in the US East region, and the secondary cluster in the US West region.

Prerequisites

An HCP account with appropriate permissions to initiate this change.
Install git on your local machine.
Install Vault on your local machine.
Install Terraform on your local machine.

Create service principal and key

For Terraform to interact with HCP, a service principal must be created with a role of Contributor, and have a key associated with it.

Launch the HCP portal and login.
From the navigation menu, click Access control (IAM).
Click Service principals, and then click Create service principal.
Enter the name you prefer in the Name field.
For example, learn-hcp-vault for this tutorial.
Select Contributor from the Role select field.
Click Save. The Service principal is created and displays the overview page.
Click Create service principal key.
A dialog like the following example appears:
Copy the Client ID.
In a terminal, export the variable HCP_CLIENT_ID to the Client ID.
```
$ export HCP_CLIENT_ID=<client id value previously copied>
```
Terraform authenticates with HCP using this ID.
Copy the Client secret.
In a terminal, export the variable HCP_CLIENT_SECRET to the Client secret.
```
$ export HCP_CLIENT_SECRET=<client secret value previously copied>
```
Terraform authenticates with HCP using this secret.
Click <- Back to Overview to return to the Cloud dashboard page.

Clone the tutorial repository

Clone the learn-hcp-vault-replication-terraform repository to get the necessary Terraform configuration for the example scenarios.

Ensure that you are in a directory that you wish to work through this tutorial in, and clone the repository.
```
$ git clone https://github.com/hashicorp/learn-hcp-vault-replication-terraform
```
Change into the directory containing the base example scenario Terraform configuration.
```
$ cd learn-hcp-vault-replication-terraform
```
Verify that you are in the correct directory before proceeding.
```
$ ls -1

README.md
deploy-hcp-vault
configure-hcp-vault
```
The deploy-hcp-vault directory contains the minimum configuration necessary to deploy HCP Vault using the Terraform HCP provider as an example for this tutorial.
The configure-hcp-vault directory contains an example configuration to configure HCP Vault using the Terraform HCP provider for this tutorial.
You will need to significantly expand on these examples to build a more advanced configurations for actual use cases.

Review deploy-hcp-vault configuration

Since the Terraform Vault provider relies on a Vault cluster to be available before it can be configured, you will first review the Terraform configuration needed to deploy an HCP Vault cluster with performance replication.

Review variables.tf

Switch to the deploy-hcp-vault directory.
```
$ cd deploy-hcp-vault
```
Examine the main.tf file. This file defines required providers and any specific provider configuration.
main.tf
```
1 2 3 4 5 6 7 8 9 101112terraform {
  required_providers {
    hcp = {
      source = "hashicorp/hcp"
      version = ">=0.26.0"
    }
  }
}

provider "hcp" {
 # Configuration options
}
```
Since performance replication was introduced in version 0.26.0 of the HCP provider, this is the minimum version required. By including the "greater-than or equal to operator" >= before the version, this will ensure Terraform uses the latest version of the provider, with a minimum version of at least 0.26.0.

Examine the variables.tf file. This file defines the variables used by the Terraform HCP provider which can be referenced in various configurations.

variables.tf

1 2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859variable "primary_cluster_hvn" {
  description = "The ID of the HCP HVN."
  type        = string
  default     = "hvn-us-east-1"
}

variable "primary_cluster_hvn_cidr" {
  description = "The ID of the HCP HVN."
  type        = string
  default     = "172.25.16.0/20"
}

variable "secondary_cluster_hvn" {
  description = "The ID of the HCP HVN."
  type        = string
  default     = "hvn-us-west-2"
}

variable "secondary_cluster_hvn_cidr" {
  description = "The ID of the HCP HVN."
  type        = string
  default     = "172.24.16.0/20"
}

variable "primary_cluster_id" {
  description = "The ID of the HCP Vault cluster."
  type        = string
  default     = "vault-cluster-primary"
}

variable "secondary_cluster_id" {
  description = "The ID of the HCP Vault cluster."
  type        = string
  default     = "vault-cluster-secondary"
}

variable "primary_region" {
  description = "The region of the primary cluster HCP HVN and Vault cluster."
  type        = string
  default     = "us-east-1"
}

variable "secondary_region" {
  description = "The region of the secondary cluster HCP HVN and Vault cluster."
  type        = string
  default     = "us-west-2"
}

variable "cloud_provider" {
  description = "The cloud provider of the HCP HVN and Vault cluster."
  type        = string
  default     = "aws"
}

variable "tier" {
  description = "Tier of the HCP Vault cluster. Valid options for tiers."
  type        = string
  default     = "plus_small"
}

Lines 1-23 provide the string and CIDR for each regions HVN. HVN CIDR ranges cannot overlap.

Lines 25-35 define the name of each of the clusters that will be created by Terraform.

Lines 37-53 define the cloud provider and regions from the cloud provider that will be used.

Lines 55-59 specify the cluster tier and size. Performance replication is available on any size within the Plus tier. The primary and secondary clusters must be the same tier and size. A list of available tiers and sizes is available in the Terraform registry.

Review hcpvault.tf

Examine the hcpvault.tf file. This file defines the configuration used by the Terraform HCP provider to deploy the HCP HVN and Vault clusters. Values here are referenced from the variables.tf file you reviewed earlier in this tutorial.

hcvault.tf

1 2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829resource "hcp_hvn" "primary_cluster_hvn" {
  hvn_id         = var.primary_cluster_hvn
  cloud_provider = var.cloud_provider
  region         = var.primary_region
  cidr_block     = var.primary_cluster_hvn_cidr
}

resource "hcp_vault_cluster" "primary_cluster" {
  hvn_id     = hcp_hvn.primary_cluster_hvn.hvn_id
  cluster_id = var.primary_cluster_id
  tier       = var.tier
  public_endpoint = true
}

resource "hcp_hvn" "secondary_cluster_hvn" {
  hvn_id         = var.secondary_cluster_hvn
  cloud_provider = var.cloud_provider
  region         = var.secondary_region
  cidr_block     = var.secondary_cluster_hvn_cidr
}

resource "hcp_vault_cluster" "secondary_cluster" {
  hvn_id          = hcp_hvn.secondary_cluster_hvn.hvn_id
  cluster_id      = var.secondary_cluster_id
  tier            = var.tier
  primary_link    = hcp_vault_cluster.primary_cluster.self_link
  paths_filter    = ["do-not-replicate-namespace", "replicate-namespace/do-not-replicate-secrets"]
  public_endpoint = true
}

Lines 1-6 create the HVN for the primary cluster in the region defined in variables.tf.

Lines 8-13 create the cluster, sets the tier defined in variables.tf, and sets the public address to true for purposes of completing this tutorial.

Lines 15-20 create the HNV for the secondary cluster in the region defined in variables.tf.

Lines 22-29 create the secondary cluster. The primary_link parameter here is required to enable performance replication. This parameter identifies which cluster is the primary cluster in the replication topology. Line 27 includes the paths_filter parameter. This optional parameter allows you to define which paths will not be replicated from the primary cluster to the secondary cluster.

Deploy HCP Vault

Now that you have reviewed the Terraform configuration for HCP Vault, you are ready to deploy the Vault clusters.

Initialize Terraform.

$ terraform init

Initializing the backend...

Initializing provider plugins...
...snip...

Terraform has been successfully initialized!

After Terraform is initialized, you can verify that the resources will be created using terraform plan.

$ terraform plan

Terraform used the selected providers to generate the following execution plan.
Resource actions are indicated with the following symbols:
...snip...

Plan: 4 to add, 0 to change, 0 to destroy.

Review the planned changes from Terraform and deploy the clusters using terraform apply.
```
$ terraform apply

...snip...

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

Enter a value:
```
Confirm the run by entering yes.
Once you confirm, it will take up to 10 minutes for each cluster to complete the deployment. If the deployment was successful, you should observe output at the end resembling this example.
```
Apply complete! Resources: 4 added, 0 changed, 0 destroyed.
```

Review the HCP Vault deployment

Switch back to the the HCP Portal.
From the navigation menu, click Vault.
You will see two clusters: vault-cluster-primary and vault-cluster-secondary. The arrow to the left of vault-cluster-secondary signifies it is a replica of the cluster above.
Click vault-cluster-primary then click Replication from the navigation menu.
From the Replication page you can manage replication settings such deleting the secondary cluster or editing paths filters.
Click the ellipses to the right of the vault-cluster-secondary card and select Edit path filters.
The path filters listed in the paths_filter parameter of the hcpvault.tf Terraform configuration are listed.
Click Cancel to return to the Replication page.

Using the supplied Terraform configuration, you were able to create two HVNs, two HCP Vault clusters, and configure performance replication by linking the designated secondary cluster to the primary cluster.

Review configure-hcp-vault configuration

Now that you have an HCP Vault cluster up and running, you can now use Terraform to configure your cluster with namespaces, policies, auth methods, and secrets engines.

To demonstrate the replication capabilities of HCP Vault you will create the following resources:

Type	Name	Description
namespace	replicate-namespace	A namespace with path: `admin/replicate-namespace`
namespace	do-not-replicate-namespace	A namespace with path: `admin/do-not-replicate-namespace
secrets engine	do-not-replicate-secrets	Enable kv-v2 secrets engine at `do-not-replicate-secrets` in `replicate-namespace`

To learn more about managing additional Vault configurations such as auth methods or policies, try the Codify Management of HCP Vault tutorial.

Review main.tf

Switch to the configure-hcp-vault directory.
```
$ cd ../configure-hcp-vault
```

Examine the main.tf file. This file defines required providers and any specific provider configuration.

main.tf

1 2 3 4 5 6 7 8 9 10111213141516171819terraform {
  required_providers {
    hcp = {
      source = "hashicorp/hcp"
      version = ">=0.26.0"
    }
    vault = {
      source = "hashicorp/vault"
      version = ">=3.4.1"
    }
  }
}

provider "vault" {
 # Configuration options
   # Configuration options
   alias = "admin"
   namespace = "admin"
}

To configure HCP Vault clusters, you use the Vault terraform provider. Because all HCP Vault clusters operate from the admin namespace, you define this in the vault provider configuration options section.

Review namespace.tf

Examine the namespace.tf file. This file is used to define the namespaces Terraform will create.

namespace.tf

123456789resource "vault_namespace" "replicate-namespace" {
  provider = vault.admin
  path = "replicate-namespace"
}

resource "vault_namespace" "do-not-replicate-namespace" {
  provider = vault.admin
  path = "do-not-replicate-namespace"
}

The resource block defines the provider to be used, and the path where the namespace will be created.

Review secrets.tf

Examine the secrets.tf file. This file is used to define the secrets engines, the path, and namespace Terraform will create them in.
secrets.tf
```
123456resource "vault_mount" "kv-v2" {
  depends_on = [vault_namespace.replicate-namespace]
  provider = vault.replicate-namespace
  path = "do-not-replicate-secrets"
  type = "kv-v2"
}
```
The vault_mount resource allows you to define configuration for the desired secrets engine. Because you will create this in a yet to be created namespace, you can use the depends_on parameter. This parameter lets Terraform know it must complete the configuration listed - in this example creating the replicate-namespace namespace, before taking any action on this resource.

Configure HCP Vault

Now that you have reviewed the Terraform configuration for HCP Vault, you are ready to configure the primary Vault cluster. The secondary cluster will have the configuration replicated from the primary, without the paths defined in the path_filter parameter.

Switch back to the the HCP Portal.
Click vault-cluster-primary.
Under Cluster URLs, click the Public Cluster URL.
In a terminal, set the VAULT_ADDR environment variable to the copied address.
```
$ export VAULT_ADDR=<Public_Cluster_URL>
```
Return to the Overview page and click Generate token.
Within a few moments, a new token will be generated.
Copy the Admin Token.
In a terminal, set the VAULT_TOKEN environment variable to store the token value.
```
$ export VAULT_TOKEN=<Admin_Token>
```
The admin namespace is the top-level namespace automatically created by HCP Vault. All CLI operations default to use the namespace defined in this environment variable.
Set the VAULT_NAMESPACE environment variable to admin.
```
$ export VAULT_NAMESPACE="admin"
```

Verify that no namespaces exist.

$ vault namespace list

No namespaces found

Initialize Terraform.

$ terraform init

Initializing the backend...

Initializing provider plugins...
...snip...

Terraform has been successfully initialized!

After Terraform is initialized, you can verify that the resources will be created using terraform plan.

$ terraform plan

Terraform used the selected providers to generate the following execution plan.
Resource actions are indicated with the following symbols:
...snip...

Plan: 3 to add, 0 to change, 0 to destroy.

Review the planned changes from Terraform and deploy the clusters using terraform apply.
```
$ terraform apply

...snip...

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

Enter a value:
```
Confirm the run by entering yes.
Once you confirm, it will take a few seconds to complete the configuration. If the configuration was successful, you should observe output at the end resembling this example.
```
Apply complete! Resources: 3 added, 0 changed, 0 destroyed.
```

Validation

You already have your shell session configured to authenticate with the primary Vault cluster. You can now validate that the namespaces and secrets engines were created.

Run vault namespace list to verify the namespaces were created.
```
$ vault namespace list

Keys
----
do-not-replicate-namespace/
replicate-namespace/
```
Both the do-not-replicate-namespace and replicate-namespace namespaces were created.

View the secrets engines available in the replicate-namespace namespace.

$ vault secrets list -namespace="admin/replicate-namespace"

Path                         Type            Accessor                 Description
----                         ----            --------                 -----------
cubbyhole/                   ns_cubbyhole    ns_cubbyhole_773fac29    per-token private secret storage
do-not-replicate-secrets/    kv              kv_056e611d              n/a
identity/                    ns_identity     ns_identity_92d21333     identity store
sys/                         ns_system       ns_system_a2b7452a       system endpoints used for control, policy and debugging

They KV v2 secret engine was created with a path of do-not-replicate-secrets.

Switch back to the HCP portal and click Replication in the navigation menu.
Click vault-cluster-secondary.
Under Cluster URLs, click the Public Cluster URL.
In a terminal, set the VAULT_ADDR environment variable to the copied address.
```
$ export VAULT_ADDR=<Public_Cluster_URL>
```
Return to the Overview page and click Generate token.
Within a few moments, a new token will be generated.
Copy the Admin Token.
In a terminal, set the VAULT_TOKEN environment variable to store the token value.
```
$ export VAULT_TOKEN=<Admin_Token>
```
Your shell session is now configured to authenticate with the secondary cluster.
View the namespaces available on the secondary cluster.
```
$ vault namespace list

Keys
----
replicate-namespace/
```
The do-not-replicate-namespace namespace was not replicated because you added the path to the path_filter parameter when deploying HCP Vault.

View the secrets engines available in the replicate-namespace namespace.

$ vault secrets list -namespace="admin/replicate-namespace"

Path                         Type            Accessor                 Description
----                         ----            --------                 -----------
cubbyhole/                   ns_cubbyhole    ns_cubbyhole_773fac29    per-token private secret storage
identity/                    ns_identity     ns_identity_92d21333     identity store
sys/                         ns_system       ns_system_a2b7452a       system endpoints used for control, policy and debugging

The do-not-replicate-secrets K/V secret engine was not replicated, even though the namespace was replicated because you added the secret engine path to the path_filter parameter when deploying HCP Vault.

Clean up

Switch to the deploy-hcp-vault directory.
```
$ cd ../deploy-hcp-vault
```

Run terraform destroy to delete the clusters.

$ terraform destroy

...snip...
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:

...snip...
Plan: 0 to add, 0 to change, 4 to destroy.

Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value:

Confirm the destroy by entering yes.

Once you confirm, it will take a few minutes to complete the the destroy. When successful, you should observe output resembling the following example.

Destroy complete! Resources: 4 destroyed.

Remove the state files.
```
$ rm *.tfstate.*
```
Remove the state files under the configure-hcp-vault folder.
```
$ rm ../configure-hcp-vault/*.tfstate.*
```
Unset the VAULT_TOKEN and VAULT_ADDR environment variables.
```
$ unset VAULT_TOKEN VAULT_ADDR
```

In this tutorial you learned how to automate the deployment of an HCP Vault cluster with performance replication and path filters using the Terraform HCP provider. You also learned about some of the resources available with the provider.

Help and reference

Terraform Vault Provider documentation page
Terraform Provider GitHub repository
Learn Terraform

13 tutorials

HashiCorp Product Integrations
Vault can manage secrets associated with other HashiCorp products.
- Vault
12 tutorials

HCP Vault Operations
Learn how to provision and connect to HCP Vault clusters.
- Vault