Deploy HCP Vault Dedicated with Terraform
HashiCorp Cloud Platform (HCP) portal features a web user interface to deploy and manage resources, including HCP Vault Dedicated deployments. If you prefer to automate the Vault Dedicated deployment, the recommended approach is to use HashiCorp Terraform with the HCP provider.
You can use the Terraform CLI and the HCP provider with your HCP account credentials from a terminal session to successfully deploy a Vault Dedicated cluster. This enables you to leverage HCP to rapidly and reliably deploy Vault clusters, while offloading the operations burden of running Vault to SRE experts at HashiCorp.
The tutorial example scenario automatically deploys an AWS VPC, and peers the provisioned HashiCorp Virtual Network (HVN).
Warning
This tutorial provisions resources that qualify under the AWS free-tier. If your account doesn't qualify under the AWS free-tier, HashiCorp is not responsible for any charges that you may incur.
Prerequisites
To complete the steps listed in this tutorial, you need:
Terraform CLI installed
An HCP account
An AWS account
git
installedAWS IAM credentials configured locally
Example:
$ export AWS_ACCESS_KEY_ID="<YOUR_AWS_ACCESS_KEY_ID>" AWS_SECRET_ACCESS_KEY="<YOUR_AWS_SECRET_ACCESS_KEY>"
Tip
If you don't have access to IAM user credentials, use another authentication method described in the AWS provider documentation.
Create a service principal and key
Log in to the HCP portal to create a service principal and associated key that you will use with Terraform to deploy the Vault Dedicated cluster.
From the navigation menu, select your organization, then select Access control (IAM).
Select Service principals, and then click Create service principal.
Enter
learn-hcp-vault
in the Service principal name field.Choose
Contributor
from the Role pull down menu, and click Create service principal.The Service principal is created and displays the overview page.
Click Create service principal key.
Copy the Client ID.
Open a terminal and export an environment variable for the
HCP_CLIENT_ID
.$ export HCP_CLIENT_ID=<client id value previously copied>
Terraform authenticates with HCP using this ID.
Return to the HCP Portal and copy the Client secret.
Return to the terminal and export an environment variable for the
HCP_CLIENT_SECRET
.$ export HCP_CLIENT_SECRET=<client secret value previously copied>
Terraform authenticates with HCP using this secret.
Clone learn-terraform-hcp-vault repository
Clone the learn-terraform-hcp-vault repository to get the necessary Terraform configuration for the example scenarios.
Ensure that you are in a directory that you wish to work through this tutorial in, and clone the repository.
$ git clone https://github.com/hashicorp-education/learn-terraform-hcp-vault
Change into the directory containing the base example scenario Terraform configuration.
$ cd learn-terraform-hcp-vault
Verify that you are in the correct directory before proceeding.
$ ls -1 variables.tf vault.tf vpc-peering.tf
Tip
The commands you use for the rest of the tutorial should all be executed from within this directory.
Review the Terraform configuration
The configuration in hcp-vault-vpc
is the minimum configuration necessary to deploy Vault Dedicated using the
Terraform HCP provider with peered VPC, as an example for this tutorial. You will need to significantly
expand on this example to build a more advanced configurations for production use cases.
Review variables
The variables.tf
file defines all Terraform variables for the project expressed in HashiCorp Configuration Language (HCL).
You can edit the variables.tf
file to change attributes for the Vault Dedicated cluster. You can learn more about the available options in
the Terraform HCP provider documentation.
Examine the
variables.tf
file. Note the description values to understand each variable.variables.tf
variable "hvn_id" { description = "The ID of the HCP HVN." type = string default = "learn-hcp-vault-hvn" } variable "cluster_id" { description = "The ID of the Vault Dedicated cluster." type = string default = "learn-hcp-vault-cluster" } variable "peering_id" { description = "The ID of the HCP peering connection." type = string default = "learn-peering" } variable "route_id" { description = "The ID of the HCP HVN route." type = string default = "learn-hvn-route" } variable "region" { description = "The region of the HCP HVN and Vault cluster." type = string default = "us-west-2" } variable "cloud_provider" { description = "The cloud provider of the HCP HVN and Vault cluster." type = string default = "aws" } variable "tier" { description = "Tier of the Vault Dedicated cluster. Valid options for tiers." type = string default = "dev" }
Line 37 through 41 defines the values for Vault Dedicated cluster tier. This is an optional parameter and the default value is
dev
; therefore, you can omit this parameter. However, if you wish to create a cluster other than a dev tier cluster, overwrite this value by creating aterraform.tfvars
file.Example:
$ tee terraform.tfvars -<<EOF tier = "starter_small" EOF
Note
The only supported
cloud_provider
types for the HCP provider areaws
orazure
.Make the necessary changes for your environment before proceeding.
Define the HashiCorp Virtual Network
Before you can deploy the Vault Dedicated cluster, you need to first configure a HashiCorp Virtual Network (HVN).
An HVN is defined by the hcp_hvn
resource.
Examine the
vault.tf
file. Lines 1 through 5 define the HVN used in this tutorial.vault.tf
resource "hcp_hvn" "learn_hcp_vault_hvn" { hvn_id = var.hvn_id cloud_provider = var.cloud_provider region = var.region } resource "hcp_vault_cluster" "learn_hcp_vault" { hvn_id = hcp_hvn.learn_hcp_vault_hvn.hvn_id cluster_id = var.cluster_id tier = var.tier #public_endpoint = true }
The configuration creates an HVN resource named
learn_hcp_vault_hvn
that uses the variable values from thevariables.tf
to declare the HVN ID, the cloud provider, and cloud provider region.HCP project considerations
By default, Terraform will create the HVN defined in the default or oldest HCP project. If you are using multiple projects in your HCP account, you can control where Terraform will create the HVN and cluster using the
project_id
parameter.For more information on how to work with multiple projects, refer to the HCP provider documentation.
Define HCP Vault Dedicated cluster
In this tutorial you will deploy a single Vault cluster.
Examine the
vault.tf
file. Lines 7 through 12 define the Vault Dedicated cluster used in this tutorial.vault.tf
resource "hcp_hvn" "learn_hcp_vault_hvn" { hvn_id = var.hvn_id cloud_provider = var.cloud_provider region = var.region } resource "hcp_vault_cluster" "learn_hcp_vault" { hvn_id = hcp_hvn.learn_hcp_vault_hvn.hvn_id cluster_id = var.cluster_id tier = var.tier #public_endpoint = true }
The configuration specifies an Vault Dedicated cluster resource named
learn_hcp_vault
that uses the variable values fromvariables.tf
to declare the HVN ID, and set the cluster ID tolearn-hcp-vault-cluster
.The
public_endpoint
parameter is commented out at line 11. See the Enable public cluster address section for more details about this optional parameter.Performance Replication
You can enable performance replication for Vault Dedicated by adding an additional
hcp_hvn
andhcp_vault_cluster
block, then set theprimary_link
parameter. For more information please visit the Configure Vault performance replication - HCP Provider guide.
Metric and audit log streaming
You can configure your Vault Dedicated cluster to stream audit logs and metrics to supported providers.
Configuring metric and audit log streaming is beyond the scope of this tutorial, but provided as a reference.
Metrics are configured using the metrics_config
parameter for the hcp_vault_cluster
resource.
Example:
resource "hcp_vault_cluster" "example" {
cluster_id = "vault-cluster"
hvn_id = hcp_hvn.example.hvn_id
tier = "standard_large"
metrics_config {
datadog_api_key = "test_datadog"
datadog_region = "us1"
}
}
Audit logs are configured using the audit_log_config
parameter for the hcp_vault_cluster
resource.
Example:
resource "hcp_vault_cluster" "example" {
cluster_id = "vault-cluster"
hvn_id = hcp_hvn.example.hvn_id
tier = "standard_large"
audit_log_config {
datadog_api_key = "test_datadog"
datadog_region = "us1"
}
}
Refer to the hcp_vault_cluster
documentation in the Terraform Registry for more information.
Define VPC peering
Examine the
vpc-peering.tf
file.vpc-peering.tf
provider "aws" { region = var.region } resource "aws_vpc" "peer" { cidr_block = "172.31.0.0/16" } data "aws_arn" "peer" { arn = aws_vpc.peer.arn } resource "hcp_aws_network_peering" "peer" { hvn_id = hcp_hvn.learn_hcp_vault_hvn.hvn_id peering_id = var.peering_id peer_vpc_id = aws_vpc.peer.id peer_account_id = aws_vpc.peer.owner_id peer_vpc_region = data.aws_arn.peer.region } resource "hcp_hvn_route" "peer_route" { hvn_link = hcp_hvn.learn_hcp_vault_hvn.self_link hvn_route_id = var.route_id destination_cidr = aws_vpc.peer.cidr_block target_link = hcp_aws_network_peering.peer.self_link } resource "aws_vpc_peering_connection_accepter" "peer" { vpc_peering_connection_id = hcp_aws_network_peering.peer.provider_peering_id auto_accept = true }
After making any necessary changes to
vpc-peering.tf
, proceed to deploy the infrastructure.
Deploy infrastructure
After you have cloned the repository and defined the environment variables for HCP (and optionally AWS), you are ready to deploy the HCP resources and peer them with your AWS account.
Initialize terraform; this downloads the necessary providers and initializes the backend.
$ terraform init Initializing the backend... Initializing provider plugins... ...snip... Terraform has been successfully initialized!
Verify the resources that will be created using
terraform plan
.$ terraform plan An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: ...snip... Plan: 6 to add, 0 to change, 0 to destroy.
You should note resources listed in the output, and at the end a summary that lists 5 resources to add.
Deploy the resources with
terraform apply
.$ terraform apply ...snip... Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value:
Confirm the run by entering
yes
.Once confirmed, it will take a few minutes to complete the deployment. If the deployment was successful, you should observe output at the end resembling this example.
Apply complete! Resources: 6 added, 0 changed, 0 destroyed.
Access Vault
Return to the HCP portal to inspect the newly created Vault cluster.
From the navigation menu within the Resources section, select Vault.
Click learn-hcp-vault-cluster to access cluster details.
The Vault Dedicated cluster overview is shown and the State is
Running
. You can interact with the cluster from this overview to perform a range of operational tasks.To confirm the HVN to VPC peering status, return to the main menu, and select HashiCorp Virtual Network.
Click learn-hcp-vault-hvn to access the HVN details.
Click Peering connections.
Select the learn-peering link.
The details for the VPC peering connection you deployed are shown. The HVN and VPC are peered and ready to use.
Additional discussion
VPC peering
When you establish a VPC peering connection from your AWS VPC and the HCP HVN you ensure only trusted clients (users, applications, containers, etc.) running inside the VPC connect to Vault and prevent systems outside of your selected network from connecting.
![Vault Dedicated Architecture Diagram](/_next/image?url=https%3A%2F%2Fcontent.hashicorp.com%2Fapi%2Fassets%3Fproduct%3Dtutorials%26version%3Dmain%26asset%3Dpublic%252Fimg%252Fcloud%252Fhcp-vault-arch.png%26width%3D1230%26height%3D479&w=3840&q=75&dpl=dpl_Cfraazb7cTY3kxdHoe7jRFiYgXDj)
Follow the Peering an AWS VPC with HashiCorp Cloud Platform
(HCP) tutorial to
create and establish a VPC peering. Vault clients running in the VPC can connect
to learn-hcp-vault-cluster
using the private cluster URL.
Updating AWS security groups
Note
Configuring, managing, and securing access to your AWS workloads is a critical component of the shared responsibility model. Terraform changes the default AWS behavior of managing security groups. This section is provided to assist with understanding how you can manage access using Terraform. Refer to the Terraform AWS Provider documentation for a full list of available resources for AWS.
Once your AWS VPC is peered with the HVN, you will need to consider how to manage access from your AWS workloads to connect to Vault Dedicated.
When creating a new security group through the AWS console or AWS CLI, the default behavior is to create an egress or outbound
rule that permits all traffic from the AWS workload to any destination, represented by the 0.0.0.0/0
CIDR notation.
However, if you manage your AWS security groups using Terraform, the default Terraform behavior is to remove the default egress rule. Once the default egress rule is removed, you need to explicitly add a rule to permit traffic from your AWS workloads to the desired destination.
How you utilize security groups in your organization will guide you on the best way to approach managing access to your Vault Dedicated cluster.
If you are already using Terraform to manage your AWS security groups, one option is to add an egress rule that will permit TCP traffic
on port 8200
using the aws_security_group_rule
resource to an existing security group.
Note
If you are already using the aws_security_group
resource with in-line rules, you will need to update the existing Terraform configuration with a rule to allow port 8200
. You cannot
use both the aws_security_group and the aws_security_group_rule resources together.
resource "aws_security_group_rule" "example" {
type = "egress"
from_port = 8200
to_port = 8200
protocol = "tcp"
cidr_blocks = ["172.25.16.0/20"]
security_group_id = "sg-123456"
}
- Line 2: The
type
parameter allows you to define whether the rule is foringress
(inbound) oregress
(outbound). - Line 3-4: The
from_port
andto_port
parameters allow you to define one or more port numbers. For Vault, clients will connect on port8200
. - Line 5: The
protocol
field specifies the type of traffic allowed. - Line 6: The
cidr_blocks
field contains the IP address the rule will permit access to, or from and should match the CIDR of your HVN. - Line 7: The
security_group_id
is the ID of the security group you want to update.
If you are not familiar with AWS security groups, refer to the AWS documentation on Control traffic to resources using security groups.
Enable public cluster address
If you want to enable the public cluster address, update the vault.tf
file with
public_endpoint
set to true
for the hcp_vault_cluster
resource.
vault.tf
resource "hcp_hvn" "learn_hcp_vault_hvn" {
hvn_id = var.hvn_id
cloud_provider = var.cloud_provider
region = var.region
}
resource "hcp_vault_cluster" "learn_hcp_vault" {
hvn_id = hcp_hvn.learn_hcp_vault_hvn.hvn_id
cluster_id = var.cluster_id
public_endpoint = true
}
Security consideration
When an HCP Vault Dedicated cluster has public access enabled, you can connect to Vault from any internet connected device. If your use case requires public access to be enabled, we recommend configuring the IP allow list to limit which IPv4 public IP addresses or CIDR ranges can connect to Vault to limit the attack surface.
When the HCP Vault Dedicated cluster has private access enabled you will need to access the cluster from a connected cloud provider such as AWS with a VPC peering connection, a AWS transit gateway connection, or Azure with a Azure Virtual Network peering connection. For the purposes of this tutorial, your cluster should have public access enabled.
Re-run the terraform apply
command.
$ terraform apply
When prompted, enter yes
to continue.
Plan: 0 to add, 1 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
When it completes, you can access your Vault Dedicated cluster using its public address.
Clean up
Use terraform destroy
to clean up all of the resources that you created.
$ terraform destroy
Terraform will prompt you for confirmation.
Do you really want to destroy all resources?
Terraform will destroy all your managed infrastructure, as shown above.
There is no undo. Only 'yes' will be accepted to confirm.
Enter a value:
Confirm by entering yes
.
Once you confirm, it will take a few minutes to complete the the destroy. When successful, you should observe output resembling the following example.
Destroy complete! Resources: 6 destroyed.
In this tutorial you learned how to automate the deployment of an Vault Dedicated cluster using the Terraform HCP provider. You also learned about some of the resources available with the provider.
You can find the full documentation for the HashiCorp Cloud Platform Terraform provider in the Terraform registry documentation.
Next steps
From here you can learn more about HCP Vault Dedicated including Vault Operation Tasks specific to Vault Dedicated.
With Vault in your HVN, and your HVN peered to an AWS VPC, you can deploy further AWS resources into the same VPC and access Vault with them through the Vault HTTP API or CLI.