Acquisition complete HashiCorp officially joins the IBM family. Learn more
Vault
Policies

Write a Vault policy using audit logs

  • 10min
  • |
  • Vault

Every Vault operation performed through the command-line interface (CLI), API, or web UI requires the client to authenticate and get a token with a policy attached. Vault uses paths, like a filesystem, for secret storage. Every policy in Vault has a corresponding path and capability. Policies provide a declarative way to grant the necessary access to each Vault path.

Scenario

In this tutorial, you will learn Vault's policy language and how to query an audit log to define policies.

You need a policy to allow updates to the transit secrets engine.

Prerequisites

  • Vault installed.
  • jq for consuming Vault JSON formatted output and capturing specific fields.

Set up the lab

  1. Open a terminal and start a Vault dev server with the literal string root as the root token value, and enable TLS.

    $ vault server -dev -dev-root-token-id root -dev-tls

    The dev server listens on the loopback interface at 127.0.0.1 on TCP port 8200 with TLS enabled. At runtime, the dev server also automatically unseals, and prints the unseal key and initial root token values to the standard output.

    Root tokens

    The dev mode server starts with an initial root token value set. Root token use should be extremely guarded in production environments because it provides full access to the Vault server. You can supply the root token value to start Vault in dev mode for convenience and to keep the steps here focused on the learning goals of this tutorial.

  2. In a new terminal, export the VAULT_ADDR and VAULT_CACERT environment variables using the commands suggested in your Vault dev server output.

    Copy each command (without the $) from the server output, and paste it into the new terminal session.

    Example:

    export VAULT_ADDR='https://127.0.0.1:8200'

    Example:

    export VAULT_CACERT='/var/folders/qr/zgztx0sj6n1dxy86sl36ntnw0000gn/T/vault-tls3037226588/vault-ca.pem'

    Remember to use your dev server's values, not the examples shown here.

  3. Export an environment variable for the vault CLI to authenticate with the Vault server.

    $ export VAULT_TOKEN=root

    The Vault server is ready.

Enable audit logs

Audit devices keep a detailed log of all requests and responses to Vault. The file audit device appends these events to a file and logs all requests and responses to Vault.

  1. Enable the file audit device at a file named vault-audit.log in the current directory.

    $ vault audit enable file file_path=$PWD/vault-audit.log
Success! Enabled the file audit device at: file/

    The vault audit enable command enables the audit device. The file parameter defines the type of audit device. Once enabled, Vault appends a test message to the audit log file.

  2. Display the contents of the audit log file.

    $ cat vault-audit.log | jq

    Example output:

    {
  "request": {
    "id": "5f079553-9612-6176-fed4-26162fadd6e7",
    "namespace": {
      "id": "root"
    },
    "operation": "update",
    "path": "sys/audit/test"
  },
  "time": "2025-06-04T19:52:19.432701Z",
  "type": "request"
}
{
  "auth": {
    "accessor": "hmac-sha256:b08da96919570945ebb32537d04e879305fe84713d37213830b5e386eeb3f044",
    "client_token": "hmac-sha256:f8c4bb61690d36aacdf170062cf820f3a28a588eb5fa45ab4f23af10affe9782",
    "display_name": "token",
    "policies": [
      "root"
    ],
    "policy_results": {
      "allowed": true,
      "granting_policies": [
        {
          "type": ""
        },
        {
          "name": "root",
          "namespace_id": "root",
          "type": "acl"
        }
      ]
    },
    "token_policies": [
      "root"
    ],
    "token_issue_time": "2025-06-04T15:50:00-04:00",
    "token_type": "service"
  },
  "request": {
    "client_id": "0DHqvq2D77kL2/JTPSZkTMJbkFVmUu0TzMi0jiXcFy8=",
    "client_token": "hmac-sha256:f8c4bb61690d36aacdf170062cf820f3a28a588eb5fa45ab4f23af10affe9782",
    "client_token_accessor": "hmac-sha256:b08da96919570945ebb32537d04e879305fe84713d37213830b5e386eeb3f044",
    "data": {
      "description": "hmac-sha256:344a2b0e7a1c76d5c070c8f44d447be0de238190376d7707423257631e5d877f",
      "local": false,
      "options": {
        "file_path": "hmac-sha256:6d11cb94a896deb5b2386c3d1fc60fd2aec391911fe2f665c42047bbf7b4f85c"
      },
      "type": "hmac-sha256:edef0ebe41163104b7a63580c892e669f9ebd86afa1ae95d3eeffc8ca4db7a14"
    },
    "id": "bdb39f4e-0c74-19d0-f5eb-0e934545554a",
    "mount_accessor": "system_53409ed3",
    "mount_class": "secret",
    "mount_point": "sys/",
    "mount_running_version": "v1.18.4+builtin.vault",
    "mount_type": "system",
    "namespace": {
      "id": "root"
    },
    "operation": "update",
    "path": "sys/audit/file",
    "remote_address": "127.0.0.1",
    "remote_port": 56704
  },
  "time": "2025-06-04T19:52:19.434493Z",
  "type": "response"
}

    The audit log appends JSON objects to the log file. The first object describes an audit test message. The second object describes the operation that enabled the audit log.

Enable transit secrets engine

The transit secrets engine handles cryptographic functions on data in-transit through keys that Vault manages.

  1. Enable the transit secrets engine at the default path.

    $ vault secrets enable transit
Success! Enabled the transit secrets engine at: transit/

    When you enable the transit secrets engine, Vault writes the operation performed to the audit log.

  2. View the last entry in the audit log file.

    $ cat vault-audit.log | jq -s ".[-1]"

    Example output:

    {
  "auth": {
    "accessor": "hmac-sha256:b08da96919570945ebb32537d04e879305fe84713d37213830b5e386eeb3f044",
    "client_token": "hmac-sha256:f8c4bb61690d36aacdf170062cf820f3a28a588eb5fa45ab4f23af10affe9782",
    "display_name": "token",
    "policies": [
      "root"
    ],
    "policy_results": {
      "allowed": true,
      "granting_policies": [
        {
          "type": ""
        },
        {
          "name": "root",
          "namespace_id": "root",
          "type": "acl"
        }
      ]
    },
    "token_policies": [
      "root"
    ],
    "token_issue_time": "2025-06-04T15:50:00-04:00",
    "token_type": "service"
  },
  "request": {
    "client_id": "0DHqvq2D77kL2/JTPSZkTMJbkFVmUu0TzMi0jiXcFy8=",
    "client_token": "hmac-sha256:f8c4bb61690d36aacdf170062cf820f3a28a588eb5fa45ab4f23af10affe9782",
    "client_token_accessor": "hmac-sha256:b08da96919570945ebb32537d04e879305fe84713d37213830b5e386eeb3f044",
    "data": {
      "config": {
        "default_lease_ttl": "hmac-sha256:553eb6c907b5b3c84542696eb3e2ea925545eee554be01bd9ba97a87fb658d77",
        "force_no_cache": false,
        "max_lease_ttl": "hmac-sha256:553eb6c907b5b3c84542696eb3e2ea925545eee554be01bd9ba97a87fb658d77",
        "options": null
      },
      "description": "hmac-sha256:344a2b0e7a1c76d5c070c8f44d447be0de238190376d7707423257631e5d877f",
      "external_entropy_access": false,
      "local": false,
      "options": null,
      "seal_wrap": false,
      "type": "hmac-sha256:3b5d775a808975472dcd878310635d292e5ee3a14d33f24034bc19d1f2854a93"
    },
    "id": "6a37d5f2-9844-dec4-be97-23ab6d821d77",
    "mount_accessor": "system_53409ed3",
    "mount_class": "secret",
    "mount_point": "sys/",
    "mount_running_version": "v1.18.4+builtin.vault",
    "mount_type": "system",
    "namespace": {
      "id": "root"
    },
    "operation": "update",
    "path": "sys/mounts/transit",
    "remote_address": "127.0.0.1",
    "remote_port": 56733
  },
  "time": "2025-06-04T19:53:19.8045Z",
  "type": "response"
}

    The audit log entry describes the time, the client_token, and the request.

  3. Display the request of the last logged object.

    $ cat vault-audit.log | jq -s ".[-1].request"

    Example output:

    {
  "client_id": "0DHqvq2D77kL2/JTPSZkTMJbkFVmUu0TzMi0jiXcFy8=",
  "client_token": "hmac-sha256:f8c4bb61690d36aacdf170062cf820f3a28a588eb5fa45ab4f23af10affe9782",
  "client_token_accessor": "hmac-sha256:b08da96919570945ebb32537d04e879305fe84713d37213830b5e386eeb3f044",
  "data": {
    "config": {
      "default_lease_ttl": "hmac-sha256:553eb6c907b5b3c84542696eb3e2ea925545eee554be01bd9ba97a87fb658d77",
      "force_no_cache": false,
      "max_lease_ttl": "hmac-sha256:553eb6c907b5b3c84542696eb3e2ea925545eee554be01bd9ba97a87fb658d77",
      "options": null
    },
    "description": "hmac-sha256:344a2b0e7a1c76d5c070c8f44d447be0de238190376d7707423257631e5d877f",
    "external_entropy_access": false,
    "local": false,
    "options": null,
    "seal_wrap": false,
    "type": "hmac-sha256:3b5d775a808975472dcd878310635d292e5ee3a14d33f24034bc19d1f2854a93"
  },
  "id": "6a37d5f2-9844-dec4-be97-23ab6d821d77",
  "mount_accessor": "system_53409ed3",
  "mount_class": "secret",
  "mount_point": "sys/",
  "mount_running_version": "v1.18.4+builtin.vault",
  "mount_type": "system",
  "namespace": {
    "id": "root"
  },
  "operation": "update",
  "path": "sys/mounts/transit",
  "remote_address": "127.0.0.1",
  "remote_port": 56733
}

    The request object describes the operation performed on the path.

  4. Display the request's path and the request's operation.

    $ cat vault-audit.log | jq -s ".[-1].request.path,.[-1].request.operation"
"sys/mounts/transit"
"update"

    The output displays that this operation performed an update on the path sys/mounts/transit.

    You have collected the information you need to define a Vault policy based on the actions you performed.

    Note

    To learn more about querying the audit log, read the Querying audit device logs tutorial.

ACL policy workflow

Use the information you collected from the audit device log to define a policy that grants the capability to update at the path sys/mounts/transit.

Example policy:

path "sys/mounts/transit" {
  capabilities = ["update"]
}

A good practice is to define your policy as files and place them under version control as part of a Vault configuration codification strategy.

  1. Write the policy example to the file transit-updates.hcl using the path, and the operation as the capabilities from the audit log.

    $ cat > transit-updates.hcl << EOF
path "sys/mounts/transit" {
  capabilities = ["update"]
}
EOF

  2. Write the policy to Vault.

    $ vault policy write transit-updates transit-updates.hcl
Success! Uploaded policy: transit-updates

    Note

    You can also manage and create ACL policies from files in the Vault web UI. Refer to the Vault policies tutorial for detailed examples,

    You performed operations against your Vault cluster, and reviewed the logs from the audit device to discover the necessary information to define a Vault policy.

    As part of your policy workflow, you can update existing ACL policies by changing the file content, and writing the policy again with the same name. The existing policy will be overwritten, but existing tokens with the policy attached will use the previous policy until a new token is requested.

Additional ACL policy challenges

Use the techniques you've learned here to complete these optional challenges.

Create an encryption key

  1. Create a encryption key named user-data.

    $ vault write -f transit/keys/user-data
Success! Data written to: transit/keys/user-data

  2. Display the request path and the request operation of the last logged object.

    $ cat vault-audit.log | jq -s ".[-1].request.path,.[-1].request.operation"
"transit/keys/user-data"
"sys/mounts/transit"
"update"

Challenge

Define a policy that grants the capability to update at path transit/keys/user-data.

Encrypt plaintext with the key

The user-data key is capable of encrypting base64 encoded plaintext and returns the ciphertext.

  1. Encrypt user sensitive data with the user-data key.

    $ vault write transit/encrypt/user-data plaintext=$(base64 <<< "sensitive user data")
Key            Value
---            -----
ciphertext     vault:v1:k1nJTMMmYM2nsYitgHRml6ExwWySGtkqgaOz3JI6kV+IT9995O26qOY5Uec/dn+l
key_version    1

  2. Display the request path and the request operation of the last logged object.

    $ cat vault-audit.log | jq -s ".[-1].request.path,.[-1].request.operation"
"transit/encrypt/user-data"
"transit/encrypt/user-data"
"create"

Challenge

Define a policy that grants the capability to update at path transit/encrypt/user-data.

path "transit/encrypt/user-data" {
  capabilities = ["update"]
}

Decrypt ciphertext with the key

The same key is used to decrypt the ciphertext and return the base64 encoded plaintext.

  1. Create a variable named USER_SECRET_DATA that stores the cipher text generated.

    $ USER_SECRET_DATA=$(vault write -format=json \
   transit/encrypt/user-data plaintext=$(base64 <<< "sensitive user data") \
   | jq -r ".data.ciphertext")

  2. Decrypt the user sensitive data with the user-data key.

    $ vault write transit/decrypt/user-data ciphertext=$USER_SECRET_DATA
Key          Value
---          -----
plaintext    c2Vuc2l0aXZlIHVzZXIgZGF0YQo=

  3. Display the request path and the request operation of the last logged object.

    $ cat vault-audit.log | jq -s ".[-1].request.path,.[-1].request.operation"
"transit/decrypt/user-data"
"update"

Challenge

Define a policy that grants the capability to update at path transit/decrypt/user-data.

path "transit/decrypt/user-data" {
  capabilities = ["update"]
}

Enable a key/value secrets engine

For the next challenges, you will work with the Key/Value secrets engine.

  1. Enable an instance of the K/V secrets engine version 2 at the path kv-v2.

    $ vault secrets enable -version=2 -path=kv-v2 kv

  2. Display the request path and the request operation of the last logged object.

    $ cat vault-audit.log | jq -s ".[-1].request.path,.[-1].request.operation"
"sys/mounts/kv-v2"
"update"

  3. Write a secret in the newly enabled secrets engine.

    $ vault kv put kv-v2/my-secret vault=awesome
==== Secret Path ====
kv-v2/data/my-secret

======= Metadata =======
Key                Value
---                -----
created_time       2022-09-01T14:43:41.273398Z
custom_metadata    <nil>
deletion_time      n/a
destroyed          false
version            1

  4. Display the request path and the request operation of the last logged object.

    $ cat vault-audit.log | jq -s ".[-1].request.path,.[-1].request.operation"
"kv-v2/data/my-secret"
"create"

Challenge

Write a policy to Vault that grants the capability to list and read at the paths kv-v2/data/my-secret and kv-v2/metadata/my-secret.

vault policy write my-secret-read-list - <<EOF
path "kv-v2/+/my-secret" {
  capabilities = ["read", "list"]
}
EOF

Note

The policy uses the + character for path matching so that it can match the sub-paths kv-v2/data and kv-v2/metadata required by the Key/Value Secrets Engine version 2. You can learn more in the KV Secrets Engine - Version 2 ACL rules documentation.

Validation

  1. Create a token with the policy attached and set its value to the environment variable MY_SECRET_TOKEN.

    $ export MY_SECRET_TOKEN="$(vault token create -field=token -policy=my-secret-read-list)"

  2. Read the secret using the new token with attached my-secret-read-list policy.

    $ VAULT_TOKEN=$MY_SECRET_TOKEN vault kv get kv-v2/my-secret
==== Secret Path ====
kv-v2/data/my-secret

======= Metadata =======
Key                Value
---                -----
created_time       2022-09-01T14:43:41.273398Z
custom_metadata    <nil>
deletion_time      n/a
destroyed          false
version            1

==== Data ====
Key      Value
---      -----
vault    awesome

Clean up

Follow these steps to clean up the scenario content from your local environment.

  1. Stop the Vault server process.

    $ pkill vault

  2. Remove the example policy file.

    $ rm -f transit-updates.hcl

  3. Remove the audit device log file.

    $ rm -f vault-audit.log

  4. Unset environment variables.

    $ unset VAULT_ADDR VAULT_TOKEN MY_SECRET_TOKEN

Next steps

In this tutorial, you performed operations and then queried the audit device log to discover the path and action involved in the operations. Learn more about querying the audit log in the Querying audit device logs tutorial.

Defining policies requires understanding the paths and capabilities of each authentication method and secrets engine. Learn other approaches in the Write a policy from API documentation tutorial.

Help and reference

Previous

Write a policy using API documentation

 Next

Simplify management of policies