Generate cloud provider credentials with Vault

23min
|
Vault

Deployment

Dynamic secrets are a core feature in Vault. A class of dynamic secrets is on-demand, revocable, time-limited access credentials for cloud providers and database platforms. For example, the Dynamic secrets foundations tutorial demonstrates the database secrets engine to dynamically generate credentials.

Challenge

To consume cloud provider services (for example, Azure Kubernetes service or GCP Kubernetes engine), the client must have valid credentials. Each cloud provider has a unique process to manage the full lifecycle of credentials, adding to the complexity of multi-cloud environments.

Solution

Automate the full lifecycle of credential management by integrating your applications with Vault's dynamically generated credentials. The applications request credential from Vault with a time-to-live (TTL) enforcing its validity so that the credentials are automatically revoked when they are no longer used.

In this tutorial, you will deploy Vault and configure Vault's Azure secrets engine to dynamically generate Azure service principals. Vault supports multiple different cloud providers, including Azure, AWS, and GCP.

Benefits

Each app instance can request unique, short-lived credentials. Unique credentials ensures isolated, auditable access and enable revocation of a single client. While short-lived reduces the time in which they are valid.

Personas

The end-to-end scenario described in this tutorial involves two personas:

admin with privileged permissions to configure secrets engines
apps read the secrets from Vault

Prerequisites

To perform the tasks described in this tutorial, you need to have:

You have a Microsoft Azure account
Vault installed

Policy requirements

Each persona requires a different set of capabilities. Vault policies allow you to limit the capabilities of each persona following the principle of least privilege. If you are not familiar with policies, complete the policies tutorial.

Required Vault policies

The admin tasks require these capabilities.

# Mount secrets engines
path "sys/mounts/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Configure the azure secrets engine and create roles
path "azure/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Write ACL policies
path "sys/policies/acl/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Manage tokens for verification
path "auth/token/create" {
  capabilities = [ "create", "read", "update", "delete", "list", "sudo" ]
}

The apps tasks require these capabilities.

path "azure/creds/edu-app" {
  capabilities = [ "read" ]
}

Lab setup

Open a new terminal window and start a Vault dev server with root as the root token.
```
$ vault server -dev -dev-root-token-id root -log-level=debug
```
The Vault dev server starts initialized and unsealed, and defaults to running at 127.0.0.1:8200.
The -log-level=debug is optional, and used in Automated root rotation step.
Insecure operation
Do not run a Vault dev server in production. This approach starts a Vault server with an in-memory database and runs in an insecure way.
In a separate window, export an environment variable for the vault CLI to address the Vault server.
```
$ export VAULT_ADDR=http://127.0.0.1:8200
```
Export an environment variable for the vault CLI to authenticate with the Vault server.
```
$ export VAULT_TOKEN=root
```

Note

For these tasks, you can use Vault's root token. However, the recommended practice is to use root tokens for initial setup or in emergencies. As a best practice, use an authentication method or token that meets the policy requirements.

The Vault server is ready.

Create an Azure service principal and resource group

(Persona: admin)

To delegate the credential generation task to Vault, you need to give Vault privileged Azure credentials to perform the task. The following demonstrates the creation of a service principal.

Production

The recommended practice is to dedicate a Azure service principal to the Vault secrets engine.

Invoking the rotate-root command will delete the existing client secret and generate a new secret known only to Vault.

Note

Refer to the online Azure documentation for more details.

Launch the Microsoft Azure Portal and sign in.
Select Microsoft Entra ID and select Properties.
Copy the Tenant ID.
In a terminal, set the variable TENANT_ID to the Tenant ID.
```
$ export TENANT_ID=<Tenant ID>
```
From the side navigation, select App registrations.
Select New registrations.
Enter a desired name in the Name field (vault-education).
Click Register.
Copy the Application (client) ID.
In a terminal, set the variable CLIENT_ID to the Application (client) ID.
```
$ export CLIENT_ID=<Client ID>
```
From the side navigation, select Certificate & secrets.
Under the Client secrets, click New client secret.
Enter a description in the Description field.
Click Add.
Copy the client secret value.
In a terminal, set the variable CLIENT_SECRET to the client secret value.
```
$ export CLIENT_SECRET=<Client secret>
```
From the side navigation, click API permissions.
Under Configured permissions, click Add a permission.
The Azure Secrets Engine documentation lists the required Azure permissions.
Click Microsoft Graph.
Select Application permissions.
Add the following permissions.
Permission Name Type
Application.ReadWrite.OwnedBy Application
GroupMember.ReadWrite.All Application
Note
If you plan to use the rotate root credentials API, you'll need to change Application.ReadWrite.OwnedBy to Application.ReadWrite.All.
The Automated root rotation feature requires this change. If you think you will want to do that section please make this change now.
Click Add permissions.
Click Grant admin consent for azure to grant the permissions.
Click Yes to confirm consent.
Navigate to the Subscriptions blade.
Click the name of your subscription.
Copy the Subscription ID.
In a terminal, set the variable SUBSCRIPTION_ID to the Subscription ID.
```
$ export SUBSCRIPTION_ID=<Subscription ID>
```
From the side navigation, click Access control (IAM).
Click Add > Add a role assignment.
Under the Privileged administrator roles tab, select User Access Administrator and click Next.
In the Members tab, click Select members.
Enter your application name or application id in the Select field.
After the applcation appears on the list, click to add it to the Members list.
Click Next to move to the Conditions tab.
Choose the radio button for Allow user to assign all roles (highly privileged) and click on Next.
Click Review + assign tab, and then Review + assign gain.

Permission Name	Type
Application.ReadWrite.OwnedBy	Application
GroupMember.ReadWrite.All	Application

You created the application with the correct capabilities and you have these identifiers and credentials:

Tenant ID
Client ID
Client Secret
Subscription ID

You created and configured a service principal, app registration, client secret and granted it the required permissions.

Resource group

The secrets engine generates credentials within an Azure resource group.

Navigate to the Resource groups blade.
Click Create.
Choose the subscription from the Subscription select field.
Enter vault-education in the Resource group field.
Click Review + create.
The view changes to display the review page.
Click Create.

You created the vault-education resource group.

Configure Vault

With the necessary resources configured in Azure, you can configure the Azure secrets engine to dynamically generate Azure service principals.

Enable the Azure secrets engine

Enable the azure secrets engine at its default path.

$ vault secrets enable azure

You enabled the secrets engine at the path azure/. To enable the secrets engine at a different path requires that you use the -path parameter and the desired path.

Enable the azure secrets engine at the path named azure.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data '{"type":"azure"}' \
    $VAULT_ADDR/v1/sys/mounts/azure

The data field defines the type, {"type":"azure"}, of secrets engine to enable. The requested URL, /v1/sys/mounts/azure, defines the path of the secrets engine.

Note

You can provide additional configuration parameters to configure the secrets engine. The path is configurable as well to support your needs.

Open a web browser and launch the Vault UI (for example http://127.0.0.1:8200/ui) and then login.
Select Enable new engine.
Select Azure from the list, and then click Next.
Click Enable Engine to complete. This sets the path to be azure.

Note

In this tutorial, the Azure secrets engine uses the default path /azure path in Vault. However, it is possible to enable your secret engines at any location by entering your desired path in the Path text field.

Configure the Azure secrets engine

(Persona: admin)

The Azure secrets engine requires the credentials you generated in the create an Azure service principal and resource group step to communicate with Azure and generate service principals.

Verify that you set Azure subscription ID, client ID, client ID, and tenant ID as environment variables.

$ echo $SUBSCRIPTION_ID; echo $CLIENT_ID; echo $CLIENT_SECRET; echo $TENANT_ID

If any of those variables are missing their value, refer to the previous step and set them before proceeding.

Configure the Azure secrets engine with the Azure credentials.

$ vault write azure/config \
     subscription_id=$SUBSCRIPTION_ID  \
     client_id=$CLIENT_ID \
     client_secret=$CLIENT_SECRET \
     tenant_id=$TENANT_ID \
     use_microsoft_graph_api=true

Create an API request payload containing the Azure credentials.

$ tee payload.json <<EOF
{
  "subscription_id": "$SUBSCRIPTION_ID",
  "tenant_id": "$TENANT_ID",
  "client_id": "$CLIENT_ID",
  "client_secret": "$CLIENT_SECRET",
  "use_microsoft_graph_api": true
}
EOF

Configure the Azure secrets engine with the credentials.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data @payload.json \
    $VAULT_ADDR/v1/azure/config

Create a role

(Persona: admin)

A Vault role lets you configure either an existing service principal or a set of Azure roles.

Create a Vault role named, edu-app mapped to the Azure role named, Contributor in the vault-education resource group.

$ vault write azure/roles/edu-app ttl=1h azure_roles=-<<EOF
    [
      {
        "role_name": "Contributor",
        "scope": "/subscriptions/$SUBSCRIPTION_ID/resourceGroups/vault-education"
      }
    ]
EOF

Create an API request payload specifying the role definition.

$ tee payload.json <<EOF
{
  "azure_roles": "[
    {
      \"role_name\": \"Contributor\",
      \"scope\":  \"/subscriptions/$SUBSCRIPTION_ID/resourceGroups/vault-education\"
    }
  ]",
  "ttl": 3600,
  "max_ttl": "24h"
}
EOF

This payload defines the mapping of the Vault role the Azure role named,

Contributor in the vault-education resource group. Credentials are generated with a time-to-live (TTL) of 1 hour and max TTL of 24 hours.

Create the Vault role with the path edu-app with the role defined within the payload.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data @payload.json \
    $VAULT_ADDR/v1/azure/roles/edu-app

You enabled and configured the azure secrets engine with your subscription ID, client ID, secrets ID ad tenant ID. Then you created the edu-app role.

Request Azure credentials

The role generates credentials with a time-to-live (TTL) of 1 hour and max TTL of 24 hours.

Read credentials from the edu-app azure role.

$ vault read azure/creds/edu-app

Example output:

Key                Value
---                -----
lease_id           azure/creds/edu-app/EA2uTB98qAPR2BRSaasnmnja
lease_duration     1h
lease_renewable    true
client_id          074421c4-60b7-477c-9c5f-07e0925ba6a6
client_secret      Vw17Q~3u3ZRRUd.M4pP-Bl487.i4Fe1~jLpLT

The results display the credentials, its TTL, and the lease ID.

For applications (apps persona) to request credentials, it requires a Vault policy that grants access to this role. Define a policy named apps.
```
$ vault policy write apps - <<EOF
path "azure/creds/edu-app" {
  capabilities = [ "read" ]
}
EOF
```
The apps policy grants the read capability for requests to the path azure/creds/edu-app.
Create a variable named APPS_TOKEN to capture the token created with the apps policy attached.
```
$ APPS_TOKEN=$(vault token create -policy=apps -field=token)
```
Note
AppRole Pull Authentication tutorial demonstrates a more sophisticated way of generating a token for your apps.

Read credentials from the edu-app azure role with the APPS_TOKEN.

$ VAULT_TOKEN=$APPS_TOKEN vault read azure/creds/edu-app

Example output:

Key                Value
---                -----
lease_id           azure/creds/edu-app/W24u6d77acJbBzzf02iq6YHd
lease_duration     1h
lease_renewable    true
client_id          b43b4e84-5568-4efd-8ba5-cbd40936ba12
client_secret      iXK7Q~mFGn-MJjcXhGbevQCQPRhB2Hkg1QGAq

Read credentials from the edu-app azure role.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --request GET \
    $VAULT_ADDR/v1/azure/creds/edu-app | jq -r ".data"

Note

This example uses jq to process the JSON output for readability.

Example output:

{
  "client_id": "0478e592-8492-4ae5-bc16-6c7ba0541112",
  "client_secret": "F-Y7Q~m1fwJ4PscEq-xa84NhMddgNR7b1vRtG"
}

The results display the credentials, its TTL, and the lease ID.

For applications (apps persona) to request credentials require a Vault policy that grants access to this role. Define a stringified version of the policy definition in a file.
```
$ tee payload.json <<EOF
{
  "policy": "path \"azure/creds/edu-app\" {capabilities = [ \"read\" ]}"
}
EOF
```

Create a policy named apps.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --request PUT \
    --data @payload.json \
    $VAULT_ADDR/v1/sys/policies/acl/apps

The apps policy grants the read capability for requests to the path azure/creds/edu-app.

Create a variable named APPS_TOKEN to capture the token created with the apps policy attached.

$ APPS_TOKEN=$(curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data '{"policies": ["apps"]}' \
    $VAULT_ADDR/v1/auth/token/create | jq -r ".auth.client_token")

Note

AppRole Pull Authentication tutorial demonstrates a more sophisticated way of generating a token for your apps.

Read credentials from the edu-app azure role with the APPS_TOKEN.

$ curl --header "X-Vault-Token: $APPS_TOKEN" \
    --request GET \
    $VAULT_ADDR/v1/azure/creds/edu-app | jq -r ".data"

Example output:

{
  "client_id": "d286adc5-53cf-43bd-b36a-6f939c3bc9c9",
  "client_secret": "B0F7Q~COpioWx5FzPrZsp8~VnjH_5FiAxB1eS"
}

Click the Policies tab, and then select Create ACL policy.
Enter apps in the Name field.
Enter this policy in the Policy field.
```
path "azure/creds/edu-app" {
  capabilities = [ "read" ]
}
```
This policy grants the read capability for requests to the path azure/creds/edu-app.
Click Create Policy.
Click the Vault CLI shell icon (>_) to open a command shell.
Execute vault write auth/token/create policies=apps in the CLI shell to create a new token:
Copy and save the generated client token value.
Sign out of the Vault UI.
Now, sign into the Vault using the newly generated token you just copied.
Click the Vault CLI shell icon (>_) to open a command shell. Execute vault read azure/creds/edu-app in the CLI shell.

The results display the credentials, its TTL, and the lease ID. The credentials for this application (service principal) in the Azure Portal searching by its client_id.

Note

Re-run the command and notice that the role returns a new set of credentials. This means that each app instance acquires a unique set of Azure credentials.

Manage leases

(Persona: admin)

The primary use of the lease ID is to manage the credentials. The credentials remain valid for the lease duration (TTL) or until revoked. Once revoked the credentials are no longer valid.

List the existing leases.

$ vault list sys/leases/lookup/azure/creds/edu-app

Example output:

Keys
----
o2F4EA3hU8Fpjgc39XyQpjtU
fRFeCtlMnoPqelTrjf5j5kGA

This command displays a list of valid leases for Azure credentials.

Create a variable that stores the first lease ID.

$ LEASE_ID=$(vault list -format=json sys/leases/lookup/azure/creds/edu-app | jq -r ".[0]")

Renew a lease

If you need to extend the use of the generated Azure credentials, you can renew the lease by passing its lease ID.

$ vault lease renew azure/creds/edu-app/$LEASE_ID

Example output:

Key                Value
---                -----
lease_id           azure/creds/edu-app/EA2uTB98qAPR2BRSaasnmnja
lease_duration     1h
lease_renewable    true

You set the TTL of the renewed lease to 1h.

Revoke the leases

When the Azure credentials are no longer needed, you can revoke the lease without waiting for its expiration.

Revoke the least associated with the $LEASE_ID environment variable.

$ vault lease revoke azure/creds/edu-app/$LEASE_ID

All revocation operations queued successfully!

List the existing leases.

$ vault list sys/leases/lookup/azure/creds/edu-app

fRFeCtlMnoPqelTrjf5j5kGA

The first lease is no longer valid.

Read new credentials from the edu-app role.
```
$ vault read azure/creds/edu-app
```
Revoke all the leases with the prefix azure/creds/edu-app.
```
$ vault lease revoke -prefix azure/creds/edu-app
```
The prefix flag matches all valid leases with the path prefix of azure/creds/edu-app.

List the existing leases.

$ vault list sys/leases/lookup/azure/creds/edu-app
No value found at sys/leases/lookup/azure/creds/edu-app

This verifies the revocation of all the leases with this path as a prefix.

List the existing lease IDs.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   --request LIST \
   $VAULT_ADDR/v1/sys/leases/lookup/azure/creds/edu-app | jq -r ".data.keys"

This command displays a list of valid leases for Azure credentials.

Example output:

[
  "w8KAdJGYmN4ysZVFHPgl6psT",
  "zKl9iK07Yf5qZtmg7qcM5Tgg"
]

Create a variable that stores the first lease ID.

$ LEASE_ID=$(curl --header "X-Vault-Token: $VAULT_TOKEN" \
   --request LIST \
   $VAULT_ADDR/v1/sys/leases/lookup/azure/creds/edu-app | jq -r ".data.keys[0]")

Renew a lease

If you need to extend the use of the generated Azure credentials, you can renew the lease by passing its lease ID.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --request PUT \
    --data "{ \"lease_id\": \"azure/creds/edu-app/$LEASE_ID\" }" \
    $VAULT_ADDR/v1/sys/leases/renew | jq

The TTL of the renewed lease is 1h.

Example output:

{
  "request_id": "7dee4bd8-73b6-f7f0-693f-78d3d38aef5c",
  "lease_id": "azure/creds/edu-app/w8KAdJGYmN4ysZVFHPgl6psT",
  "renewable": true,
  "lease_duration": 3600,
  "data": null,
  "wrap_info": null,
  "warnings": null,
  "auth": null
}

Revoke the leases

When the Azure credentials are no longer needed, you can revoke the lease without waiting for its expiration.

Revoke the lease without waiting for its expiration.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   --request PUT \
   --data "{ \"lease_id\": \"azure/creds/edu-app/$LEASE_ID\" }" \
   $VAULT_ADDR/v1/sys/leases/revoke

List the existing leases.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   --request LIST \
   $VAULT_ADDR/v1/sys/leases/lookup/azure/creds/edu-app | jq -r ".data.keys"

Example output:

[
  "zKl9iK07Yf5qZtmg7qcM5Tgg"
]

The deleted lease is no longer valid and is not displayed.

Read new credentials from the edu-app role.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   --request GET \
   $VAULT_ADDR/v1/azure/creds/edu-app | jq -r ".data"

Revoke all the leases with the prefix azure/creds/edu-app.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   --request PUT \
   $VAULT_ADDR/v1/sys/leases/revoke-prefix/azure/creds/edu-app

List the existing leases.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   --request LIST \
   $VAULT_ADDR/v1/sys/leases/lookup/azure/creds/edu-app | jq

This verifies the revocation of all leases with this path as a prefix; therefore, it returns an empty array.

{
  "errors": []
}

Automated root rotation

Note

Automated root rotation requires Vault community or Enterprise 1.19 or later. Until 1.19 is available on HCP Vault dedicated, this tutorial requires self-hosted Vault.

Rotating static root credentials makes using cloud credentials more efficient and secure. It enforces a finite lifespan for the initial root credentials and in the case of a leaked secret, a limited window of use. The new root credential is only known by Vault, and not viewed by a user.

On the Azure side ensure that the Application permissions for Microsoft Graph include the following.

Permission Name	Type
Application.ReadWrite.All	Application
GroupMember.ReadWrite.All	Application

If the IAM Graph API permissions do not match these, update the secret to include them.

The role configuration for azure/roles/edu-app need not change, but this feature requires adding two fields in azure/config.

Reconfigure the Azure secrets engine with the rotation_schedule and rotaion window parameters.
```
$ vault write azure/config \
   subscription_id=$SUBSCRIPTION_ID  \
   client_id=$CLIENT_ID \
   client_secret=$CLIENT_SECRET \
   tenant_id=$TENANT_ID \
   use_microsoft_graph_api=true \
   rotation_schedule="*/1 * * * *" \
   rotation_window=60
```
The rotation_schedule parameter is a standard cron schedule expression. The fields in the string represent the minute, hour, day of month, month, and day of week.
In this example, "*/1 * * * *" the secret rotates every minute. For "* */1 * * *" the secret would rotate every hour.

Check the window with the Vault server running. The log should include some entries with a [DEBUG] entries that resemble the following.

2025-02-27T12:04:18.351-0600 [DEBUG] secrets.azure.azure_374be79b.azure.vault-plugin-secrets-azure: starting periodic func: timestamp=2025-02-27T12:04:18.351-0600
2025-02-27T12:05:18.350-0600 [DEBUG] secrets.azure.azure_374be79b.azure.vault-plugin-secrets-azure: starting periodic func: timestamp=2025-02-27T12:05:18.349-0600

Disable automated root rotation use the disable_automated_rotation parameter.

$ vault write azure/config \     
   disable_automated_rotation=true
Success! Data written to: azure/config

You configured and validated automated root rotation, then disabled it.

Summary

You enabled and configured the Azure secrets engine and requested a secret. You also learned how to list, renew, and revoke credential leases. Finally, you enabled automated root rotation.

Learn more by exploring the Azure secrets engine documentation and Azure secrets engine API documentation.

Clean up

If they are no longer required, delete the Azure credentials created to configure the secrets engine.

Launch the Microsoft Azure Portal and sign in.
Navigate to Azure Entra ID.
From the side navigation, select App registrations.
Click the vault-education application (or whatever the name you set for the application).
From the application overview, click delete.
Select Yes to delete the application.
Navigate to Resource groups.
Click the vault-education resource group.
From the resource group overview, click Delete resource group.
Enter vault-education in the TYPE THE RESOURCE GROUP NAME: field.
Click Delete to remove the resource group.

Stop the Vault server

Unset the VAULT_TOKEN environment variable.
```
$ unset VAULT_TOKEN
```
Unset the VAULT_ADDR environment variable.
```
$ unset VAULT_ADDR
```
If you ran the automated root rotation section of this tutorial, unset the environment variables use for that section.
```
$ unset CLIENT_ID TENANT_ID SUBSCRIPTION_ID CLIENT_SECRET
```
If you are running Vault locally in dev mode, stop the Vault dev server by pressing Ctrl+C where the server is running. Or, execute the following command.
```
$ pgrep -f vault | xargs kill
```

Help and reference

Store versioned Key/value secrets

Manage LDAP credentials