• HashiCorp Developer

  • HashiCorp Cloud Platform
  • Terraform
  • Packer
  • Consul
  • Vault
  • Boundary
  • Nomad
  • Waypoint
  • Vagrant
Vault
  • Install
  • Tutorials
  • Documentation
  • API
  • Integrations
  • Try Cloud(opens in new tab)
  • Sign up
HCP Vault

Skip to main content
3 tutorials
  • HCP Vault with Amazon Elastic Kubernetes Service
  • HCP Vault with AWS EKS and JWT Auth Method
  • Vault Agent with Amazon Elastic Container Service

  • Resources

  • Tutorial Library
  • Certifications
  • Community Forum
    (opens in new tab)
  • Support
    (opens in new tab)
  • GitHub
    (opens in new tab)
  1. Developer
  2. Vault
  3. Tutorials
  4. HCP Vault
  5. HCP Vault with AWS EKS and JWT Auth Method

HCP Vault with AWS EKS and JWT Auth Method

  • 12min

  • HCPHCP
  • VaultVault

HashiCorp Cloud Platform (HCP) is a fully managed platform offering HashiCorp Products as a Service (HPaaS) to automate infrastructure on any cloud.

In this tutorial, you will learn the process required to authenticate an AWS EKS cluster to HCP Vault with the JWT auth method.

Prerequisites

The following prerequisites are required:

  • An HCP HashiCorp Virtual Network (HVN).
  • A public HCP Vault deployment.
  • AWS CLI installed.
  • kubectl installed.
  • helm installed.
  • jq installed.
  • git installed.
  • An EKS Cluster deployed in the VPC associated with your HVN.

TIP: Ensure that you have authenticated with the AWS CLI, and that the CLI is targeting the region where you created your EKS cluster. Review the AWS documentation for instructions on how to configure the AWS CLI.

Complete the steps detailed in the manual deployment tutorial to enable communication between your HCP Vault cluster servers and the Vault Agent running in your EKS cluster.

Your EKS cluster security group must also allow traffic from the HCP Vault CIDR range. If your EKS cluster endpoint uses port 443, create a security group rule to allow ingress traffic from HCP Vault to the primary EKS cluster security group.

The HashiCorp Virtual Network (HVN) section of the HCP UI can assist you in composing the correct eksctl commands to do this when you establish the peering connection between your HVN and VPC.

You will configure Vault from your development host. As a result, the HCP Vault cluster needs to be publicly available. You should instead configure HCP Vault over a bastion host using the Vault private interface instead when working in production.

Scenario environments

The scenario consists of 3 distinct environments:

  • Your local host: This is where you will use the terminal along with eksctl, vault, and kubectl CLI commands to perform the tasks which make up the scenario.

  • Your HCP account: This is where you have deployed your HCP Vault cluster. You can use Terraform to deploy the HCP Vault cluster, or do so manually with the HCP web UI.

  • Your AWS account: This is where you have deployed your EKS cluster and its VPC that is peered to your HVN with appropriate subnet, route table, and security group configuration.

Prepare local scenario environment

On your local host, prepare the scenario environment.

For ease of clean up and simplicity, create a temporary directory that contains all required configurations for the scenario, and assign that directory name value to the environment variable HC_LEARN_LAB.

$ export HC_LEARN_LAB="$(mktemp -d -t hc-learn-XXXX)"

Now change into this directory; you will execute all scenario commands on the local host from this directory or one of its descendants.

$ cd ${HC_LEARN_LAB?}

Kubernetes stores cluster connection information in a special file called kubeconfig. You can retrieve the Kubernetes configuration settings for your EKS cluster, and merge them into your local kubeconfig file.

  1. Use the AWS CLI to retrieve the kubeconfig.

    $ aws eks --region <your-region> update-kubeconfig --name <your-cluster-name>
    
  2. You can use the HCP Portal to retrieve the client configuration information you need to connect the Vault Agents in your EKS cluster to HCP Vault. Navigate to the Vault resource page in the HCP portal, and then select the Vault cluster.

  3. Click "Generate Token". Copy the administrator token and set it in your terminal to an environment variable named VAULT_TOKEN.

    $ export VAULT_TOKEN=hvs.********************************************************************************************************
    
  4. Click the clipboard next to "Public". This will copy the public Vault address to your clipboard. Set it as the VAULT_ADDR environment variable.

    $ export VAULT_ADDR=https://learn.vault.**************.aws.hashicorp.cloud:8200
    
  5. Click the clipboard next to "Private". This will copy the private Vault address to your clipboard. Set the Vault private address as the VAULT_PRIVATE_ADDR environment variable. You will use this later in the tutorial to enable the EKS cluster to access HCP Vault over the HVN peering connection.

    $ export VAULT_PRIVATE_ADDR=https://learn.private.vault.**************.aws.hashicorp.cloud:8200
    
  6. Since HCP Vault uses the Enterprise Namespace feature with a default namespace named admin, you need to set the VAULT_NAMESPACE environment variable value to admin.

    $ export VAULT_NAMESPACE=admin
    

Install Vault Agents on EKS

Use the official vault-helm chart to install Vault Agents in your EKS cluster.

  1. Retrieve the Helm chart from the HashiCorp Helm repository.

    $ helm repo add hashicorp https://helm.releases.hashicorp.com && \
        helm repo update
    

    Example output:

    "hashicorp" has been added to your repositories
    Hang tight while we grab the latest from your chart repositories...
    ...Successfully got an update from the "hashicorp" chart repository
    Update Complete. ⎈ Happy Helming!⎈
    
  2. Create a values.yaml file that sets the external servers to HCP Vault over the private address. This will deploy a Vault Agent injector into the EKS cluster.

    $ cat > values.yaml << EOF
    injector:
       enabled: true
       externalVaultAddr: "${VAULT_PRIVATE_ADDR?}"
    EOF
    
  3. Validate that the values file is correctly populated with your HCP Vault private address value (which will differ from the example shown).

    $ cat values.yaml
    injector:
    enabled: true
    externalVaultAddr: "https://vault-cluster-brian.private.vault.11eb3a47-8920-4714-ba99-0242ac11000e.aws.hashicorp.cloud:8200"
    
  4. Install the HashiCorp Vault Helm chart.

    $ helm install vault -f values.yaml hashicorp/vault
    
    NAME: vault
    LAST DEPLOYED: Thu Jun  9 08:54:48 2022
    NAMESPACE: default
    STATUS: deployed
    REVISION: 1
    TEST SUITE: None
    NOTES:
    Thank you for installing HashiCorp Vault!
    
    Now that you have deployed Vault, you should look over the docs on using
    Vault with Kubernetes available here:
    
    https://www.vaultproject.io/docs/
    
    Your release is named vault. To learn more about the release, try:
    
      $ helm status vault
      $ helm get manifest vault
    
  5. Once the helm install command completes, verify the Vault Agent injector pod deploys by issuing kubectl get pods.

    $ kubectl get pods
    

    Example output:

    NAME                                   READY   STATUS    RESTARTS   AGE
    vault-agent-injector-c8fd9fc5f-jhhw9   1/1     Running   0          2m11s
    

Kubernetes configuration

Create a namespace, learn-vault.

$ kubectl create namespace learn-vault

Create a cluster role binding.

$ kubectl create clusterrolebinding oidc-reviewer \
--clusterrole=system:service-account-issuer-discovery \
--group=system:unauthenticated

Create a service account named product in the learn-vault namespace.

$ kubectl create sa product -n learn-vault

Configure JWT auth method on HCP Vault

Your services need a service account token to authenticate to Vault, which they can obtain from the Vault JWT/OIDC Auth Method.

  1. Enable the JWT/OIDC auth method in HCP Vault using your terminal.

    $ vault auth enable jwt
    

    Output:

    Success! Enabled jwt auth method at: jwt/
    
  2. Fetch the service account signing public key from your cluster's JWKS URI.

    Start a proxy.

    $ kubectl proxy &
    

    Assign its value to the environment variable ISSUER.

    $ ISSUER="$(curl --fail --silent --show-error 127.0.0.1:8001/.well-known/openid-configuration | jq -r '.issuer')"
    

    Stop the proxy by killing its process.

    $ kill %%
    
  3. Create the JWT auth method configuration, specifying the ISSUER environment variable as value of the OIDC Discovery URL parameter.

    $ vault write auth/jwt/config oidc_discovery_url="${ISSUER?}"
    

    Successful output example:

    Success! Data written to: auth/jwt/config
    

Deploy an example workload

Now that the clients have been deployed, it is time to deploy an application workload. This tutorial will use the HashiCups demo application.

Issue the following command to clone the repository to the development host.

$ git clone https://github.com/hashicorp/hcp-vault-eks-jwt-auth

Change into the project directory.

$ cd hcp-vault-eks-jwt-auth

Deploy a database to Kubernetes

NOTE: This tutorial deploys a database into Kubernetes and exposes it to HCP Vault using a public LoadBalancer service type. In a production configuration, you should deploy this as a private load balancer restricting access to only HCP Vault.

  1. Deploy a PostgreSQL database. This contains data for various coffees related to a demo application, all hosted in the products database.

    $ kubectl apply -f postgresql.yaml
    

    Output:

    service/postgres created
    deployment.apps/postgres created
    
  2. Verify you've deployed the PostgreSQL database in your Kubernetes cluster.

    $ kubectl get pods
    

    Example output:

    NAME                                   READY   STATUS    RESTARTS   AGE
    postgres-5bbfc8bb5c-9px77              1/1     Running   0          31s
    vault-agent-injector-c8fd9fc5f-jhhw9   1/1     Running   0          2m34s
    

Add the database role to Vault

The product API needs to read the database username and password from Vault. Create the role for the product service account to generate database credentials.

  1. Enable the database secrets engine.

    $ vault secrets enable database
    

    Output:

    Success! Enabled the database secrets engine at: database/
    
  2. Set the POSTGRES_IP environment variable to the load balancer DNS hostname.

    $ export POSTGRES_IP=$(kubectl get service -o jsonpath='{.status.loadBalancer.ingress[0].hostname}' \
       postgres)
    
  3. Create the database configuration that allows HCP Vault to configure Postgres.

    $ vault write database/config/products \
        plugin_name=postgresql-database-plugin \
        allowed_roles="*" \
        connection_url="postgresql://{{username}}:{{password}}@${POSTGRES_IP}:5432/products?sslmode=disable" \
        username="postgres" \
        password="password"
    
  4. Create a database role for product that allows Vault to issue database passwords.

    $ vault write database/roles/product \
        db_name=products \
        creation_statements="CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}'; \
            GRANT SELECT ON ALL TABLES IN SCHEMA public TO \"{{name}}\";" \
        revocation_statements="ALTER ROLE \"{{name}}\" NOLOGIN;"\
        default_ttl="1h" \
        max_ttl="24h"
    

    Output:

    Success! Data written to: database/roles/product
    
  5. Request a new set of PostgreSQL database credentials for the product role.

    $ vault read database/creds/product
    

    Example output:

    Key                Value
    ---                -----
    lease_id           database/creds/product/zeXVyjlT550wHYEDOhF5ckDC.8fmet
    lease_duration     1h
    lease_renewable    true
    password           gT8FvhyMtLuF-SCbw4mF
    username           v-token-hc-product-Qcg6fDERVvya93uwvbCy-1652888425
    

Configure Vault policy for database credentials

Create a new product ACL policy that allows the product service to read the database credentials specific to product.

$ vault policy write product - << EOF
path "database/creds/product" {
  capabilities = ["read"]
}
EOF

Output:

Success! Uploaded policy: product

Create the vault-jwt-product JWT auth method role to associate the product. This allows the product service account in Kubernetes to get a Vault token with the JWT auth method.

$ vault write auth/jwt/role/vault-jwt-product \
    role_type="jwt" \
    bound_audiences="https://kubernetes.default.svc" \
    user_claim="sub" \
    bound_subject="system:serviceaccount:learn-vault:product" \
    policies="product" \
    ttl="1h"

NOTE: The bound_audiences value in this example is for EKS, but could differ for other platforms. If you are using a platform other than EKS, be sure to consult the documentation for that platform for the correct value.

Deploy the product API

  1. Make sure that you include your HCP Vault namespace in the deployment that will access database credentials.

    $ grep ${VAULT_NAMESPACE} product.yaml
    

    Output:

    vault.hashicorp.com/namespace: "admin"
    
  2. Deploy the product service.

    $ kubectl apply -f product.yaml
    

    Output:

    service/product created
    serviceaccount/product created
    deployment.apps/product created
    
  3. The product deployment should initialize.

    $ kubectl get pods
    

    Example output:

    NAME                                    READY   STATUS    RESTARTS   AGE
    postgres-5bd8c648fd-q6jhr               1/1     Running   0          25m
    product-6f7ff64c75-zrcwd                2/2     Running   0          27s
    vault-agent-injector-7d4ccf785f-dkt7c   1/1     Running   0          29m
    
  4. Port forward the web service locally to port 9090.

    $ kubectl port-forward service/product 9090
    

    Output:

    Forwarding from 127.0.0.1:9090 -> 9090
    Forwarding from [::1]:9090 -> 9090
    
  5. Open another terminal and make a request to localhost:9090/coffees to check if the web service can pull coffee information from the database.

    $ curl -s localhost:9090/coffees | jq .
    

    Output:

    [
      {
        "id": 1,
        "name": "Packer Spiced Latte",
        "teaser": "Packed with goodness to spice up your images",
        "description": "",
        "price": 350,
        "image": "/packer.png",
        "ingredients": [
          {
            "ingredient_id": 1
          },
          {
            "ingredient_id": 2
          },
          {
            "ingredient_id": 4
          }
        ]
      },
      {
        "id": 2,
        "name": "Vaulatte",
        "teaser": "Nothing gives you a safe and secure feeling like a Vaulatte",
        "description": "",
        "price": 200,
        "image": "/vault.png",
        "ingredients": [
          {
            "ingredient_id": 1
          },
          {
            "ingredient_id": 2
          }
        ]
      },
      {
        "id": 3,
        "name": "Nomadicano",
        "teaser": "Drink one today and you will want to schedule another",
        "description": "",
        "price": 150,
        "image": "/nomad.png",
        "ingredients": [
          {
            "ingredient_id": 1
          },
          {
            "ingredient_id": 3
          }
        ]
      },
      {
        "id": 4,
        "name": "Terraspresso",
        "teaser": "Nothing kickstarts your day like a provision of Terraspresso",
        "description": "",
        "price": 150,
        "image": "/terraform.png",
        "ingredients": [
          {
            "ingredient_id": 1
          }
        ]
      },
      {
        "id": 5,
        "name": "Vagrante espresso",
        "teaser": "Stdin is not a tty",
        "description": "",
        "price": 200,
        "image": "/vagrant.png",
        "ingredients": [
          {
            "ingredient_id": 1
          }
        ]
      },
      {
        "id": 6,
        "name": "Connectaccino",
        "teaser": "Discover the wonders of our meshy service",
        "description": "",
        "price": 250,
        "image": "/consul.png",
        "ingredients": [
          {
            "ingredient_id": 1
          },
          {
            "ingredient_id": 5
          }
        ]
      }
    ]
    

When you are done, return to the terminal with the port-forward command and type Ctrl + C to exit.

Clean up

  1. Delete the product API.

    $  kubectl delete -f product.yaml
    
  2. Delete the product role.

    $ vault delete auth/kubernetes/role/product
    
  3. Delete the product policy.

    $ vault policy delete product
    
  4. Delete the product database role.

    $ vault delete database/roles/product
    
  5. Revoke all leases for database credentials.

    $ vault lease revoke -prefix database/creds/product
    
  6. Delete the PostgreSQL database.

    $ kubectl delete -f postgres.yaml
    
  7. Delete the database secrets engine configuration.

    $ vault delete database/config/product
    
  8. Delete the Helm installation for HCP Vault.

    $ helm delete vault
    
  9. Disable the database secrets engine in HCP Vault.

    $ vault secrets disable database
    
  10. Disable the Kubernetes auth method in HCP Vault.

    $ vault auth disable kubernetes
    
  11. Delete your HCP Vault cluster and the associated HVN.

  12. Remove the scenario directory and all of its contents.

    $ rm -rf ${HC_LEARN_LAB?}
    

Next steps

In this tutorial, you connected Vault clients on Amazon EKS to HCP Vault and retrieved PostgreSQL database credential dynamically. To learn more about the Vault features introduced in this tutorial, refer to the following tutorials.

  • Dynamic Secrets: Database Secrets Engine
  • Integrate a Kubernetes Cluster with an External Vault
  • Provision an EKS Cluster (AWS)
 Previous
 Next

This tutorial also appears in:

  •  
    12 tutorials
    HCP Vault Operations
    Learn how to provision and connect to HCP Vault clusters.
    • Vault

On this page

  1. HCP Vault with AWS EKS and JWT Auth Method
  2. Prerequisites
  3. Prepare local scenario environment
  4. Install Vault Agents on EKS
  5. Kubernetes configuration
  6. Configure JWT auth method on HCP Vault
  7. Deploy an example workload
  8. Clean up
  9. Next steps
Give Feedback(opens in new tab)
  • Certifications
  • System Status
  • Terms of Use
  • Security
  • Privacy
  • Trademark Policy
  • Trade Controls
  • Give Feedback(opens in new tab)