HCP Vault Secrets with Terraform

8min
|
Beta
HCP
Terraform
Vault

In addition to using the command line interface (CLI) or application programming interface (API), you can also use the HCP Terraform provider to retrieve secrets from HCP Vault Secrets.

Prerequisites

An existing HCP account
Completed the previous HCP Vault Secrets tutorials
HCP service principal with HCP_CLIENT_ID and HCP_CLIENT_SECRET available
Terraform installed
Terraform Cloud account (Terraform Cloud workflow only)

Retrieve secrets with Terraform

The HCP Terraform provider includes a data source for HCP Vault Secrets. This data source allows you to retrieve secrets and use them in a Terraform configuration.

Retrieve the HCP_CLIENT_ID and HCP_CLIENT_SECRET environment variables set during the Install HCP Vault Secrets CLI tutorial.

$ printenv | grep HCP_
HCP_CLIENT_ID=aBcDefGx2ruWptclGVpIE7MX7uxkU4
HCP_CLIENT_SECRET=S3sAm3stxKjziNhjTWWe9QxEffKyTdiGlxE_suNsh1n3

Log into the Terraform Cloud portal.
Click Projects & workspaces then click Settings.
Click Variable sets. Variables in Terraform Cloud allow you to store items similar to setting an environment variable for a shell session or in a .tfvars file.
Click Create variable set.
Enter HCP_SERVICE_PRINCIPAL in the Name textbox.
Select the Apply globally radio button.
Click + Add variable.
Enter HCP_CLIENT_ID in the Key field, and the actual client ID in the Value field.
Click the Sensitive checkbox, then click Add variable.
Click + Add variable again.
Enter HCP_CLIENT_SECRET in the Key field, and the actual client secret in the Value field.
Click the Sensitive checkbox, then click Add variable.
Click Create variable set.
Click < Workspaces then click Projects & workspaces
Click New and select Workspace.
Select CLI-driven workflow and enter HCPVaultSecretsLab in the Workspace name field.
Scroll to the bottom of the form and click Create workspace.
Copy the name of your Terraform Cloud organization and return to your terminal session.
Set an environment variable with the name of your Terraform Cloud organization.
```
$ export TFC_ORG=<actual-org-name>
```

Create an example Terraform configuration with the required provider and Terraform Cloud configuration block. The HCP Vault Secrets data source requires the hcp provider at version 0.63.0 or greater.

$ tee example.tf <<EOF
terraform {
  cloud {
    organization = "$TFC_ORG"

    workspaces {
      name = "HCPVaultSecretsLab"
    }
  }
  required_providers {
    hcp = {
      source = "hashicorp/hcp"
      version = "0.63.0"
    }
  }
}

provider "hcp" {
  # Configuration options
  client_id = var.HCP_CLIENT_ID
  client_secret = var.HCP_CLIENT_SECRET
}

EOF

Declare the HCP_CLIENT_ID and HCP_CLIENT_SECRET variables in the Terraform configuration.

$ tee -a example.tf <<EOF
variable "HCP_CLIENT_ID" {
  type = string
}

variable "HCP_CLIENT_SECRET" {
  type = string
}

EOF

Add the hcp_vault_secrets_app data block to the Terraform configuration to read the WebApplication application you created in an earlier tutorial.
```
$ tee -a example.tf <<EOF
data "hcp_vault_secrets_app" "WebApplication" {
  app_name = "WebApplication"
}

EOF
```
Add an output block to allow Terraform to display the retrieved secret. This step is not necessary for production configurations, but used in this tutorial to validate that Terraform successfully retrieved the secret.
Because HCP Vault Secrets is used to store sensitive information, you must add sensitive = true to the output block.
```
$ tee -a example.tf <<EOF
output "secrets" {
  value = data.hcp_vault_secrets_app.WebApplication.secrets
  sensitive = true
}
EOF
```

$ terraform login
---------------------------------------------------------------------------------

Terraform must now open a web browser to the tokens page for app.terraform.io.

If a browser does not open this automatically, open the following URL to proceed:
    https://app.terraform.io/app/settings/tokens?source=terraform-login

---------------------------------------------------------------------------------

Generate a token using your browser, and copy-paste it into this prompt.

Terraform will store the token in plain text in the following file
for use by subsequent commands:
    /Users/username/.terraform.d/credentials.tfrc.json

Token for app.terraform.io:
  Enter a value:

Retrieved token for user username

---------------------------------------------------------------------------------

                                          -
                                          -----                           -
                                          ---------                      --
                                          ---------  -                -----
                                           ---------  ------        -------
                                             -------  ---------  ----------
                                                ----  ---------- ----------
                                                  --  ---------- ----------
   Welcome to Terraform Cloud!                     -  ---------- -------
                                                      ---  ----- ---
   Documentation: terraform.io/docs/cloud             --------   -
                                                      ----------
                                                      ----------
                                                       ---------
                                                           -----
                                                               -

   New to TFC? Follow these steps to instantly apply an example configuration:

   $ git clone https://github.com/hashicorp/tfc-getting-started.git
   $ cd tfc-getting-started
   $ scripts/setup.sh

Run terraform init to initialize the configuration and install the necessary providers.

$ terraform init

Initializing Terraform Cloud...

Initializing provider plugins...
- Finding hashicorp/hcp versions matching "0.63.0"...
- Installing hashicorp/hcp v0.63.0...
- Installed hashicorp/hcp v0.63.0 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform Cloud has been successfully initialized!

You may now begin working with Terraform Cloud. Try running "terraform plan" to
see any changes that are required for your infrastructure.

If you ever set or change modules or Terraform Settings, run "terraform init"
again to reinitialize your working directory.

Run terraform apply to read the WebApplication values from HCP Vault Secrets.

$ terraform apply -auto-approve

Running apply in Terraform Cloud. Output will stream here. Pressing Ctrl-C
will cancel the remote apply if it's still pending. If the apply started it
will stop streaming the logs, but will not stop the apply running remotely.

Preparing the remote apply...

To view this run in a browser, visit:
https://app.terraform.io/app/your-org/HCPVaultSecretsLab/runs/run-ps3UvBSumbg7c91C

Waiting for the plan to start...

Terraform v1.5.2
on linux_amd64
Initializing plugins and modules...
data.hcp_vault_secrets_app.WebApplication: Refreshing...
data.hcp_vault_secrets_app.WebApplication: Refresh complete after 1s [id=WebApplication]

Changes to Outputs:
  + secrets = (sensitive value)

You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure.

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:
secrets = (sensitive value)

Return to the Terraform Cloud portal.
On the Overview page you can see the Latest Run is in a stage of Planned and finished.
Click States.
Click Triggered via CLI.
View the state.
The username secret you previously added is displayed.

Verify the HCP_CLIENT_ID and HCP_CLIENT_SECRET environment variables for your HCP service principal are set.

$ printenv | grep HCP_
HCP_CLIENT_ID=aBcDefGx2ruWptclGVpIE7MX7uxkU4
HCP_CLIENT_SECRET=S3sAm3stxKjziNhjTWWe9QxEffKyTdiGlxE_suNsh1n3

Create an example Terraform configuration with the required provider. The HCP Vault Secrets data source requires the hcp provider at version 0.63.0 or greater.

$ tee example.tf <<EOF
terraform {
  required_providers {
    hcp = {
      source = "hashicorp/hcp"
      version = "0.63.0"
    }
  }
}

provider "hcp" {
  # Configuration options
}

EOF

Add the hcp_vault_secrets_app data block to the Terraform configuration to read the WebApplication application you created in an earlier tutorial.
```
$ tee -a example.tf <<EOF
data "hcp_vault_secrets_app" "WebApplication" {
  app_name = "WebApplication"
}

EOF
```
Add an output block to allow Terraform to display the retrieved secret. This step is not necessary for production configurations, but used in this tutorial to validate that Terraform successfully retrieved the secret.
Because HCP Vault Secrets is used to store sensitive information, you must add sensitive = true to the output block.
```
$ tee -a example.tf <<EOF
output "secrets" {
  value = data.hcp_vault_secrets_app.WebApplication.secrets
  sensitive = true
}
EOF
```

Run terraform init to initialize the configuration and install the necessary providers.

$ terraform init

Initializing the backend...

Initializing provider plugins...
- Finding hashicorp/hcp versions matching "0.63.0"...
- Installing hashicorp/hcp v0.63.0...
- Installed hashicorp/hcp v0.63.0 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Run terraform apply to read the WebApplication values from HCP Vault Secrets.

$ terraform apply -auto-approve
data.hcp_vault_secrets_app.WebApplication: Reading...
data.hcp_vault_secrets_app.WebApplication: Read complete after 1s [id=WebApplication]

Changes to Outputs:
  + secrets = (sensitive value)

You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure.

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

secrets = <sensitive>

Run terraform output secrets to display the secret values.
```
$ terraform output secrets
tomap({
  "username" = "db-user"
})
```
The username secret you previously added is displayed.

Help and reference

Retrieve a secret from HCP Vault Secrets

Next Collection

UI quick start