Tokenize Data with Transform Secrets Engine

29min
Enterprise
Vault

Note

Transform secrets engine requires Vault Enterprise Advanced Data Protection (ADP) license.

Challenge

When encrypting sensitive data, preservation of the original data format or length may be required to meet certain industry standards such as HIPAA or PCI. To fulfill this requirement, the transform secrets engine performs format preserving encryption (FPE).

However, there are organizations that care more about the irreversibility of the tokenized data and not so much about preserving the original data format. Therefore, the transform secrets engine's FPE transformation may not meet the governance, risk and compliance (GRC) strategy they are looking for due to the use of reversible cryptography to perform FPE.

Solution

Transform secrets engine has a data transformation method to tokenize sensitive data stored outside of Vault. Tokenization replaces sensitive data with unique values (tokens) that are unrelated to the original value in any algorithmic sense. Therefore, those tokens cannot risk exposing the plaintext satisfying the PCI-DSS guidance.

Characteristics of the tokenization transformation:

Non-reversible identification: Protect data pursuant to requirements for data irreversibility (PCI-DSS, GDPR, etc.)
Integrated Metadata: Supports metadata for identifying data type and purpose
Extreme scale and performance: Support for performantly managing billions of tokens across clouds as well as on-premise

Prerequisites

To perform the tasks described in this tutorial, you need:

Vault Enterprise 1.6.0 or later with Advanced Data Protection module
jq installed to process the JSON output for readability

Note

To explore Vault Enterprise features, you can sign up for a free 30-day trial.

Policy requirements

Note

For the purpose of this tutorial, you can use root token to work with Vault. However, it is recommended that root tokens are only used for just enough initial setup or in emergencies. As a best practice, use tokens with appropriate set of policies based on your role in the organization.

To perform all tasks demonstrated in this tutorial, your policy must include the following permissions:

# Work with transform secrets engine
path "transform/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Enable secrets engine
path "sys/mounts/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# List enabled secrets engine
path "sys/mounts" {
  capabilities = [ "read", "list" ]
}

If you are not familiar with policies, refer to the policies tutorial.

Lab setup

Open a terminal and start a Vault dev server with root as the root token.

$ vault server -dev -dev-root-token-id root

The Vault dev server defaults to running at 127.0.0.1:8200. The server is also initialized and unsealed.

Insecure operation

Do not run a Vault dev server in production. This approach is only used here to simplify the unsealing process for this demonstration.

Export an environment variable for the vault CLI to address the Vault server.

$ export VAULT_ADDR=http://127.0.0.1:8200

Export an environment variable for the vault CLI to authenticate with the Vault server.

$ export VAULT_TOKEN=root

The Vault server is ready.

Setup the Transform secrets engine

Create a role named, mobile-pay which is attached to credit-card transformation. The tokenized value has a fixed maximum time-to-live (TTL) of 24 hours.

Enable the transform secrets engine at transform/.
```
$ vault secrets enable transform
```
Create a role named mobile-pay with a transformation named credit-card.
```
$ vault write transform/role/mobile-pay transformations=credit-card
Success! Data written to: transform/role/mobile-pay
```
The mobile-pay role is created.
The role is created but the credit-card transformation does not exist, yet.
Create a transformation named credit-card which sets the generated token's time-to-live (TTL) to 24 hours.
```
$ vault write transform/transformations/tokenization/credit-card \
  allowed_roles=mobile-pay \
  max_ttl=24h
```
Output:
```
Success! Data written to: transform/transformations/tokenization/credit-card
```
The max_ttl is an optional parameter which allows you to control how long the token should stay valid.
NOTE: Set the allowed_roles parameter to a wildcard (*) to allow all roles or with globs at the end for pattern matching (e.g. mobile-*).

Display details about the credit-card transformation.

$ vault read transform/transformations/tokenization/credit-card

Key              Value
---              -----
allowed_roles    [mobile-pay]
mapping_mode     default
max_ttl          24h
templates        <nil>
type             tokenization

Notice that the type is set to tokenization.

Enable transform secrets engine using /sys/mounts endpoint.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
      --request POST \
      --data '{"type":"transform"}' \
      $VAULT_ADDR/v1/sys/mounts/transform

Create a role named mobile-pay with a transformation named credit-card.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
      --request POST \
      --data '{"transformations": ["credit-card"]}' \
      $VAULT_ADDR/v1/transform/role/mobile-pay

The mobile-pay role is created.

Create credit-card tokenization transformation.
First, create an API request payload.
```
$ tee payload.json <<EOF
{
  "max_ttl": "24h",
  "allowed_roles": [ "mobile-pay" ]
}
EOF
```
The max_ttl is an optional parameter which allows you to control how long the token should stay valid.
NOTE: Set the allowed_roles parameter to a wildcard (*) to allow all roles or with globs at the end for pattern matching (e.g. mobile-*).
Create a transformation named credit-card.
```
$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data @payload.json \
    $VAULT_ADDR/v1/transform/transformations/tokenization/credit-card
```

List existing tokenization transformations to verify that credit-card was created successfully.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --request LIST \
    $VAULT_ADDR/v1/transform/transformations/tokenization \
    | jq ".data"
{
  "keys": [
    "credit-card",
  ]
}

Read back the credit-card transformation.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    $VAULT_ADDR/v1/transform/transformations/tokenization/credit-card \
    | jq ".data"
{
  "allowed_roles": [
    "mobile-pay"
  ],
  "mapping_mode": "default",
  "max_ttl": 86400,
  "templates": null,
  "type": "tokenization"
}

Notice that the type is set to tokenization.

Tokenize secrets

The Vault client applications must have the following in their policy to perform tokenization transformation using the Transform secrets engine enabled at transform/.

Required Client Policy

# To request data encoding using any of the roles
# Specify the role name in the path to narrow down the scope
path "transform/encode/mobile-pay" {
   capabilities = [ "update" ]
}

# To request data decoding using any of the roles
# Specify the role name in the path to narrow down the scope
path "transform/decode/mobile-pay" {
   capabilities = [ "update" ]
}

# To validate the token
path "transform/validate/mobile-pay" {
   capabilities = [ "update" ]
}

# To retrieve the metadata belong to the token
path "transform/metadata/mobile-pay" {
   capabilities = [ "update" ]
}

# To check and see if the secret is tokenized
path "transform/tokenized/mobile-pay" {
   capabilities = [ "update" ]
}

Encode a value with the mobile-pay role with some metadata.

$ vault write transform/encode/mobile-pay value=1111-2222-3333-4444 \
     transformation=credit-card \
     ttl=8h \
     metadata="Organization=HashiCorp" \
     metadata="Purpose=Travel" \
     metadata="Type=AMEX"

The ttl value is an optional parameter. Remember that the max_ttl was set to 24 hours when you created the credit-card transformation. You can overwrite that value to make the token's TTL to be shorter.

In addition, you can set optional metadata about the data.

The output displays the encoded value.

Key              Value
---              -----
encoded_value    Q4tYgFXHxUbU5XZoQsxusAzxhNyVWmCFWxAT8SGkHoYB3VkrmQGXVR

Set the generated token value in a MY_TOKEN environment variable for testing.
Example:
```
$ export MY_TOKEN="Q4tYgFXHxUbU5XZoQsxusAzxhNyVWmCFWxAT8SGkHoYB3VkrmQGXVR"
```

Retrieve the metadata of the token.

$ vault write transform/metadata/mobile-pay value=$MY_TOKEN transformation=credit-card

Key                Value
---                -----
expiration_time    2021-03-13 06:49:58.041608 +0000 UTC
metadata           map[Organization:HashiCorp Purpose:Travel Type:AMEX]

Notice that expiration_time is displayed. Since you have overwritten the max_ttl, the ttl is set to 8 hours.

Validate the token value.

$ vault write transform/validate/mobile-pay value=$MY_TOKEN transformation=credit-card

Key      Value
---      -----
valid    true

Validate that the credit card number has been tokenized already.

$ vault write transform/tokenized/mobile-pay value=1111-2222-3333-4444 transformation=credit-card

Key          Value
---          -----
tokenized    true

Retrieve the original plaintext credit card value.

$ vault write transform/decode/mobile-pay transformation=credit-card value=$MY_TOKEN

Key              Value
---              -----
decoded_value    1111-2222-3333-4444

Encode a value with the mobile-pay role with some metadata.

First, create an HTTP request payload.

$ tee payload.json <<EOF
{
  "value": "1111-2222-3333-4444",
  "transformation": "credit-card",
  "ttl": "8h",
  "metadata": {
    "Organization": "HashiCorp",
    "metadata": "Purpose=Travel",
    "metadata": "Type=AMEX"
  }
}
EOF

In addition, you can set optional metadata about the data.

Encode a value with the mobile-pay role.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data @payload.json \
    $VAULT_ADDR/v1/transform/encode/mobile-pay | jq ".data"

Example output:

{
  "encoded_value": "9UrVvkyFti3hHaT1cWqs5Hch14hKL2B8bMtgRbqYjyepgCgK6Hy5zXtrHLzqJs93C41DRGnf"
}

Retrieve the metadata of the token. Be sure to replace the <generated_token> value with actual value (encoded_value) that was returned.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data '{"value": <generated_token>, "transformation": "credit-card"}' \
    $VAULT_ADDR/v1/transform/metadata/mobile-pay | jq ".data"

Example output:

{
  "expiration_time": "2021-03-13 06:49:58.041608 +0000 UTC",
  "metadata": {
    "Organization": "HashiCorp",
    "Purpose": "Travel",
    "Type": "AMEX"
  }
}

Notice that expiration_time is displayed. Since you have overwritten the max_ttl, the ttl is set to 8 hours.

Validate the token value. Be sure to replace the <generated_token> value with actual value (encoded_value) that was returned.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data '{"value": <generated_token>, "transformation": "credit-card"}' \
    $VAULT_ADDR/v1/transform/validate/mobile-pay | jq ".data"

Output:

{
  "valid": true
}

Validate that the credit card number has been tokenized already.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data '{"value": "1111-2222-3333-4444", "transformation": "credit-card"}' \
    $VAULT_ADDR/v1/transform/tokenized/mobile-pay | jq ".data"

Output:

{
  "tokenized": true
}

Retrieve the original plaintext credit card value. Be sure to replace the <generated_token> value with actual value (encoded_value) that was returned.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data '{"value": <generated_token>, "transformation": "credit-card"}' \
    $VAULT_ADDR/v1/transform/decode/mobile-pay | jq ".data"

Output:

{
  "decoded_value": "1111-2222-3333-4444"
}

Convergent tokenization

Requirement

This feature requires Vault 1.11.0 or later.

If you run the command multiple times, you would notice that it returns a different encoded value every time.

Example:

$ vault write transform/encode/mobile-pay value=5555-6666-7777-8888 transformation=credit-card
Key              Value
---              -----
encoded_value    Q4tYgFXHxURKc5zaVDRhi3QT35htvADKtnBpBazpoBZPG373FP1mXs     

$ vault write transform/encode/mobile-pay value=5555-6666-7777-8888 transformation=credit-card
Key              Value
---              -----
encoded_value    Q4tYgFXHxUVpMnf4SSVQZWm1YhAG2eSGvS3dBeDNmm7fkPLnw3JV54

In some use cases, you may want to have the same encoded value for a given input so that you can query your database to count the number of entries for a given secret.

Key derivation is supported to allow the same key to be used for multiple purposes by deriving a new key based on a user-supplied context value. In this mode, convergent encryption can optionally be supported, which allows the same input values to produce the same ciphertext.

Create a transformation named credit-card-convergent which sets the enables the convergent encryption. When you define a transformation, set convergent=true.

$ vault write transform/transformations/tokenization/credit-card-convergent \
     allowed_roles="*" \
     convergent=true

Output:

Success! Data written to: transform/transformations/tokenization/credit-card-convergent

Add the credit-card-convergent to the mobile-pay role.

$ vault write transform/role/mobile-pay transformations="credit-card, credit-card-convergent"
Success! Data written to: transform/role/mobile-pay

Encode a value using the credit-card-convergent transformation.

$ vault write transform/encode/mobile-pay value=5555-6666-7777-8888 \
     transformation=credit-card-convergent

Example output:

Key              Value
---              -----
encoded_value    DaCJhefr1oRUoY1vCtgJ8HixeazFhWHzFsNoNGeDWnyGGe76Xp8Er4UyW76xiPyQ2Eh2jGztnb3x

Run the command again.

$ vault write transform/encode/mobile-pay value=5555-6666-7777-8888 \
     transformation=credit-card-convergent

Example output:

The same encrypted value is returned.

Key              Value
---              -----
encoded_value    DaCJhefr1oRUoY1vCtgJ8HixeazFhWHzFsNoNGeDWnyGGe76Xp8Er4UyW76xiPyQ2Eh2jGztnb3x

Create credit-card-convergent tokenization transformation.

First, create an API request payload.

$ tee payload-convergent.json <<EOF
{
   "convergent": true,
   "allowed_roles": [ "*" ]
}
EOF

NOTE: Set the convergent parameter to "true" to enable convergent encryption.

Create a transformation named credit-card-convergent.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
     --request POST \
     --data @payload-convergent.json \
     $VAULT_ADDR/v1/transform/transformations/tokenization/credit-card-convergent

Add the credit-card-convergent to the mobile-pay role.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
     --request POST \
     --data '{"transformations": ["credit-card","credit-card-convergent"]}' \
     $VAULT_ADDR/v1/transform/role/mobile-pay

Encode a value using the credit-card-convergent transformation.

First, create an HTTP request payload.

$ tee payload-input-2.json <<EOF
{
  "value": "5555-6666-7777-8888",
  "transformation": "credit-card-convergent"
}
EOF

Encode a value with the mobile-pay role.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data @payload-input-2.json \
    $VAULT_ADDR/v1/transform/encode/mobile-pay | jq ".data"

Example output:

{
  "encoded_value": "DaCJhefr1oRUoY1vCtgJ8HixeazFhWHzFsNoNGeDWnyGGe76Xp8Er4UyW76xiPyQ2Eh2jGztnb3x"
}

Run the command again.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data @payload-input-2.json \
    $VAULT_ADDR/v1/transform/encode/mobile-pay | jq ".data"

Example output:

The same encrypted value is returned.

{
  "encoded_value": "DaCJhefr1oRUoY1vCtgJ8HixeazFhWHzFsNoNGeDWnyGGe76Xp8Er4UyW76xiPyQ2Eh2jGztnb3x"
}

Lookup token

When the transformation is configured with convergent encryption, you can look up the tokenized value (token).

Encode the value using the credit-card-convergent transformation with time-to-live (TTL) of 8 hours.

$ vault write transform/encode/mobile-pay value=5555-6666-7777-8888 \
     transformation=credit-card-convergent ttl=8h

Example output:

Key              Value
---              -----
encoded_value    MQEgMY8jwXYJNb9p177dYCttwRUcctAeo89yVmMPiLotzfe3sx2btpBGNLijo2KA9cy4hAnTfeV78D5rJwyBdC5j6tMEZgH

Notice that the encoded value (token) is longer than the one without TTL.

Look up the token for a card number, "5555-6666-7777-8888".

$ vault write transform/tokens/mobile-pay value=5555-6666-7777-8888 \
    transformation=credit-card-convergent

Output:

Key       Value
---       -----
tokens    [DaCJhefr1oRUoY1vCtgJ8HixeazFhWHzFsNoNGeDWnyGGe76Xp8Er4UyW76xiPyQ2Eh2jGztnb3x]

Now, look up with expiration of "any".

$ vault write transform/tokens/mobile-pay value=5555-6666-7777-8888 \
    transformation=credit-card-convergent expiration="any"

Output:

Key       Value
---       -----
tokens    [DaCJhefr1oRUoY1vCtgJ8HixeazFhWHzFsNoNGeDWnyGGe76Xp8Er4UyW76xiPyQ2Eh2jGztnb3x MQEgMY8jwXYJNb9p177dYCttwRUcctAeo89yVmMPiLotzfe3sx2btpBGNLijo2KA9cy4hAnTfeV78D5rJwyBdC5j6tMEZgH]

This returns two tokens. In absence of the "expiration" paramter, the command returns token with no expiration. When the expiration is set to "any", it returns tokens with any expiration.

Look up token that are expiration between a given range using min_expiration and min_expiration which are RFC3339 formatted time and date.

Example:

$ vault write transform/tokens/mobile-pay value=5555-6666-7777-8888 \
    transformation=credit-card-convergent \
    min_expiration="2022-06-20T06:30:50+00:00" \
    max_expiration="2022-06-20T14:30:50+00:00"

Key       Value
---       -----
tokens    [DaCJhefr1oRUoY1vCtgJ8HixeazFhWHzFsNoNGeDWnyGGe76Xp8Er4UyW76xiPyQ2Eh2jGztnb3x MQEgMY8jwXYJNb9p177dYCttwRUcctAeo89yVmMPiLotzfe3sx2btpBGNLijo2KA9cy4hAnTfeV78D5rJwyBdC5j6tMEZgH]

Encode the value using the credit-card-convergent transformation with time-to-live (TTL) of 8 hours.

First, create an HTTP request payload.

$ tee payload-input-3.json <<EOF
{
  "value": "5555-6666-7777-8888",
  "transformation": "credit-card-convergent",
  "ttl": "8h"
}
EOF

Encode a value with the mobile-pay role.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data @payload-input-3.json \
    $VAULT_ADDR/v1/transform/encode/mobile-pay | jq ".data"

Example output:

{
  "encoded_value": "MQEgMY8jwXMzbK94WCG1SsWvrAP6JBUNBLyLM45Kb4h42La4zHj15N6ZG3Q3aaQqPvDhf6wXCm6B3L4L4zx19yaPiskLbNm"
}

Notice that the encoded value (token) is longer than the one without TTL.

Look up the token for a card number, "5555-6666-7777-8888".

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data '{"value": "5555-6666-7777-8888", "transformation": "credit-card-convergent"}' \
    $VAULT_ADDR/v1/transform/tokens/mobile-pay | jq ".data"

Example output:

{
  "tokens": [
    "DaCJhefr1oRUoY1vCtgJ8HixeazFhWHzFsNoNGeDWnyGGe76Xp8Er4UyW76xiPyQ2Eh2jGztnb3x"
  ]
}

Now, look up with expiration of "any".

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data '{"expiration": "any", "value": "5555-6666-7777-8888", "transformation": "credit-card-convergent"}' \
    $VAULT_ADDR/v1/transform/tokens/mobile-pay | jq ".data"

Example output:

{
  "tokens": [
    "DaCJhefr1oRUoY1vCtgJ8HixeazFhWHzFsNoNGeDWnyGGe76Xp8Er4UyW76xiPyQ2Eh2jGztnb3x",
    "MQEgMY8jwXMzbK94WCG1SsWvrAP6JBUNBLyLM45Kb4h42La4zHj15N6ZG3Q3aaQqPvDhf6wXCm6B3L4L4zx19yaPiskLbNm"
  ]
}

Look up token that are expiration between a given range using min_expiration and min_expiration which are RFC3339 formatted time and date.

Example:

$ tee payload-range.json <<EOF
{
  "value": "5555-6666-7777-8888",
  "transformation": "credit-card-convergent",
  "min_expiratio": "2022-06-20T06:30:50+00:00",
  "max_expiration": "2022-06-20T14:30:50+00:00"
}
EOF

$ curl -s --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data @payload-range.json \
    $VAULT_ADDR/v1/transform/tokens/mobile-pay | jq ".data"
{
  "tokens": [
    "DaCJhefr1oRUoY1vCtgJ8HixeazFhWHzFsNoNGeDWnyGGe76Xp8Er4UyW76xiPyQ2Eh2jGztnb3x",
    "MQEgMY8jwXMzbK94WCG1SsWvrAP6JBUNBLyLM45Kb4h42La4zHj15N6ZG3Q3aaQqPvDhf6wXCm6B3L4L4zx19yaPiskLbNm"
  ]
}

Key rotation

Note

The automatic key rotation requires Vault Enterprise 1.12.0 or later.

Rotating keys regularly limits the amount of information produced by a key if that key ever becomes compromised. In this section, you are going to enable automatic key rotation for your tokenization keys.

Read the key information for credit-card transformation.

$ vault read transform/tokenization/keys/credit-card

Key                       Value
---                       -----
auto_rotate_period        0s
latest_version            1
min_available_version     0
min_decryption_version    1
name                      credit-card
type                      aes256-gcm96

Notice that the latest_version is 1.

Rotate the key for credit-card transformation.

$ vault write -force transform/tokenization/keys/credit-card/rotate
Success! Data written to: transform/tokenization/keys/credit-card/rotate

Read the key information again.

$ vault read transform/tokenization/keys/credit-card

Key                       Value
---                       -----
auto_rotate_period        0s
latest_version            2
min_available_version     0
min_decryption_version    1
name                      credit-card
type                      aes256-gcm96

The latest_version is now 2.

Now, instead of manually rotating the key, configure the key to be automatically rotated every 90 days to reduce operational overhead.
```
$ vault write transform/tokenization/keys/credit-card/config \
   auto_rotate_period=90d
```
Note
The minimum permitted value for the auto_rotate_period is 1 hour.

Verify the configuration.

$ vault read transform/tokenization/keys/credit-card

Output:

Key                       Value
---                       -----
auto_rotate_period        2160h
latest_version            1
min_available_version     0
min_decryption_version    1
name                      credit-card
type                      aes256-gcm96

If the key gets compromised, you can rotate the key using the transform/tokenization/keys/<transformation_name>/rotate, and then set the min_decryption_version to the latest key version so that the older (possibly compromised) key will not be able to decrypt the data.

Test the auto-rotation

Because the minimum rotation period you can set is 1 hour, you will need to come back later to see that the key is rotated.

Set the rotation period to 1 hour.

$ vault write transform/tokenization/keys/credit-card/config \
   auto_rotate_period=1h

Encrypt another value for testing.

$ export MY_TOKEN_2=$(vault write -format=json transform/encode/mobile-pay \
   value=1234-5678-9012-3456 transformation=credit-card ttl=8h \
   | jq -r ".data.encoded_value")

The value, 1234-5678-9012-3456 is encoded with credit-card key version of 2, and the returned encoded value is stored in the MY_TOKEN_2 environment variable.

You can make sure that the environment variable holds the encoded value.

$ echo $MY_TOKEN_2
eRwUjS2L9e7fjTv7gFa3Kaf1LakuWc387XdECkbWBBLu6AiMCMWyG8mr1cPBkrSj6keC4DV69

Wait for at least 1 hour to see the key has been rotated.

$ vault read transform/tokenization/keys/credit-card

Key                       Value
---                       -----
auto_rotate_period        2160h
latest_version            3
min_available_version     0
min_decryption_version    1
name                      credit-card
type                      aes256-gcm96

You can test that the data encoded by version 1 of the key can be still decoded because min_decryption_version is 1.
```
$ vault write transform/decode/mobile-pay transformation=credit-card value=$MY_TOKEN
```
Also, you should be able to decode MY_TOKEN_2.
```
$ vault write transform/decode/mobile-pay transformation=credit-card value=$MY_TOKEN_2
```
If you change the min_decryption_version to 2, you will be able to decode MY_TOKEN_2 but not MY_TOKEN.

Read the key information for credit-card transformation.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    $VAULT_ADDR/v1/transform/tokenization/keys/credit-card | jq ".data"

Output:

{
   "auto_rotate_period": 0,
   "latest_version": 1,
   "min_available_version": 0,
   "min_decryption_version": 1,
   "name": "credit-card",
   "type": "aes256-gcm96"
}

Notice that the latest_version is 1.

Rotate the key for credit-card transformation.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    $VAULT_ADDR/v1/transform/tokenization/keys/credit-card/rotate

Read the key information again.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    $VAULT_ADDR/v1/transform/tokenization/keys/credit-card | jq ".data"

Output:

{
   "auto_rotate_period": 0,
   "latest_version": 2,
   "min_available_version": 0,
   "min_decryption_version": 1,
   "name": "credit-card",
   "type": "aes256-gcm96"
}

The latest_version is now 2.

Now, instead of manually rotating the key, configure the key to be automatically rotated every 90 days to reduce operational overhead.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data '{ "auto_rotate_period": "90d" }' \
    $VAULT_ADDR/v1/transform/tokenization/keys/credit-card/config

Note

The minimum permitted value for the auto_rotate_period is 1 hour.

Verify the configuration.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    $VAULT_ADDR/v1/transform/tokenization/keys/credit-card | jq ".data"

Output:

{
   "auto_rotate_period": 7776000,
   "latest_version": 2,
   "min_available_version": 0,
   "min_decryption_version": 1,
   "name": "credit-card",
   "type": "aes256-gcm96"
}

Test the auto-rotation

Because the minimum rotation period you can set is 1 hour, you will need to come back later to see that the key is rotated.

Set the rotation period to 1 hour.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data '{ "auto_rotate_period": "1h" }' \
    $VAULT_ADDR/v1/transform/tokenization/keys/credit-card/config

Encode another value for testing.

First, create an HTTP request payload.

$ tee payload.json <<EOF
{
  "value": "1234-5678-9012-3456",
  "transformation": "credit-card",
  "ttl": "8h"
}
EOF

Now, encode the value with the mobile-pay role.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data @payload.json \
    $VAULT_ADDR/v1/transform/encode/mobile-pay | jq ".data"

Example output:

{
   "encoded_value": "eRwUjS2L9dis8xUQwgDpvGYdPb5ht1poFyFCeeYL9oN2afLGDbwhmbzvc4Lc6QJnQ2ZmJ28ZA"
}

The value, 1234-5678-9012-3456 is encoded with credit-card key version of 2. Store the returned encoded value in the MY_TOKEN_2 environment variable.

$ export MY_TOKEN_2="eRwUjS2L9dis8xUQwgDpvGYdPb5ht1poFyFCeeYL9oN2afLGDbwhmbzvc4Lc6QJnQ2ZmJ28ZA"

Wait for at least 1 hour to see the key has been rotated.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    $VAULT_ADDR/v1/transform/tokenization/keys/credit-card | jq ".data"

Output:

{
   "auto_rotate_period": 3600,
   "latest_version": 3,
   "min_available_version": 0,
   "min_decryption_version": 1,
   "name": "credit-card",
   "type": "aes256-gcm96"
}

You can test that the data encoded by version 1 of the key can be still decoded because min_decryption_version is 1.

Create a request payload for MY_TOKEN.

$ tee payload-my-token.json <<EOF
{
   "value": "$MY_TOKEN", 
   "transformation": "credit-card"
}
EOF

Decode the value.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   --request POST \
   --data @payload-my-token.json \
   $VAULT_ADDR/v1/transform/decode/mobile-pay | jq ".data"

Also, you should be able to decode MY_TOKEN_2.

$ tee payload-my-token-2.json <<EOF
{
   "value": "$MY_TOKEN_2", 
   "transformation": "credit-card"
}
EOF

Decode the value.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   --request POST \
   --data @payload-my-token-2.json \
   $VAULT_ADDR/v1/transform/decode/mobile-pay | jq ".data"

If you change the min_decryption_version to 2, you will be able to decode MY_TOKEN_2 but not MY_TOKEN.

Setup external token storage

Unlike format preserving encryption (FPE) transformation, tokenization is a stateful procedure to facilitate mapping between tokens and various cryptographic values (one way HMAC of the token, encrypted metadata, etc.) including the encrypted plaintext itself which must be persisted.

At scale, this could put a lot of additional load on the Vault's storage backend. To avoid this, you have an option to use external storage to persist data for tokenization transformation.

Note

Currently, PostgreSQL, MySQL, and MSSQL are supported as external storage for tokenization.

To demonstrate, run a PostgreSQL database in a Docker container. Create a new transformation named, "passport" which uses this PostgreSQL as its storage rather than using the Vault's storage backend.

Run PostgreSQL Docker image in a container.

Start a postgres instance which listens to port 5432, and the superuser (root) password is set to rootpassword.

$ docker run --name postgres -e POSTGRES_USER=root \
     -e POSTGRES_PASSWORD=rootpassword \
     -d -p 5432:5432 postgres

You can verify that the postgres container is running.

$ docker ps

CONTAINER ID        IMAGE            ...         PORTS                    NAMES
befcf913da91        postgres         ...         0.0.0.0:5432->5432/tcp   postgres

Create a new role, "global-id".

$ vault write transform/role/global-id transformations=passport
Success! Data written to: transform/role/global-id

Create a store which points to the postgres.

$ vault write transform/stores/postgres \
   type=sql \
   driver=postgres \
   supported_transformations=tokenization \
   connection_string='postgresql://{{username}}:{{password}}@localhost/root?sslmode=disable' \
   username=root \
   password=rootpassword

Create a schema in postgres to store tokenization artifacts.

$ vault write transform/stores/postgres/schema transformation_type=tokenization \
    username=root password=rootpassword

Create a new transformation named, "passport" which points to the postgres store.

$ vault write transform/transformations/tokenization/passport \
    allowed_roles=global-id stores=postgres

Open another terminal and connect to the postgres container.

$ docker exec -it postgres bash

Start psql.

$ psql -U root

Check to verify that there is no entry.

$ select * from tokens;

storage_token | key_version | ciphertext | encrypted_metadata | fingerprint | expiration_time
---------------+-------------+------------+--------------------+-------------+-----------------
(0 rows)

Return to the terminal you were running Vault CLI, and encode some test data.

$ vault write transform/encode/global-id \
    transformation=passport \
    value="123456789"

Example output:

Key              Value
---              -----
encoded_value    Q4tYgFXHxUS3PnQLiUnyH2JfGeEZQDFXMMaFXLU6MZfiix1tjqwgNX

Return to the postgres container, and check the data entry.

$ select * from tokens;

Example output:

  storage_token        | key_version |       ciphertext          | encrypted_metadata | ...
--------------------------+-------------+---------------------------+--------------------+-...
\x128aa3c24699...snip... |           1 | \x1ee7cc3505e31...snip... |                    | ...
(1 row)

As you encode more data, the table entry grows.

Enter \q to quit the psql session.
Enter exit to exit out of the Docker container.

Create a new role, "global-id".

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data '{"transformations": ["passport"]}' \
    $VAULT_ADDR/v1/transform/role/global-id

Create a store which points to the postgres.

First, create a request payload.

$ tee payload-store.json <<EOF
{
   "type": "sql",
   "supported_transformations": "tokenization",
   "connection_string": "postgresql://{{username}}:{{password}}@localhost/root?sslmode=disable",
   "driver": "postgres",
   "username": "root",
   "password": "rootpassword"
}
EOF

Create a store using the transform/stores/postgres endpoint.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data @payload-store.json \
    $VAULT_ADDR/v1/transform/stores/postgres

Create a schema in postgres to store tokenization artifacts.

First, create a request payload.

$ tee payload-scm.json <<EOF
{
   "transformation_type": "tokenization",
   "username": "root",
   "password": "rootpassword"
}
EOF

Create a schema using the transform/stores/postgres/schema endpoint.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data @payload-scm.json \
    $VAULT_ADDR/v1/transform/stores/postgres/schema

Create a new transformation named, "passport" which points to the postgres store.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
     --request POST \
     --data '{ "allowed_roles": [ "global-id" ], "stores": "postgres" }' \
     $VAULT_ADDR/v1/transform/transformations/tokenization/passport

Open another terminal and connect to the postgres container.

$ docker exec -it postgres bash

Start psql.

$ psql -U root

Check to verify that there is no entry.

$ select * from tokens;

storage_token | key_version | ciphertext | encrypted_metadata | fingerprint | expiration_time
---------------+-------------+------------+--------------------+-------------+-----------------
(0 rows)

Return to the terminal you were executing Vault API using cURL, and encode some test data.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
     --request POST \
     --data '{ "value": "123456789", "transformation": "passport"}' \
     $VAULT_ADDR/v1/transform/encode/global-id | jq ".data"

Example output:

{
  "encoded_value": "Q4tYgFXHxUTaU7ekcJhzZ9CUtzWmgMh4MmAXLiaSsVe4Y9TAsB3iAs"
}

Return to the postgres container, and check the data entry.

$ select * from tokens;

The table contains a row of data associated with the encode request.

  storage_token        | key_version |       ciphertext          | encrypted_metadata | ...
-------------------------+-------------+---------------------------+--------------------+-...
\x128aa3c24699...snip... |           1 | \x1ee7cc3505e31...snip... |                    | ...
(1 row)

As you encode more data, the table entry grows.

Enter \q to quit the psql session.
Enter exit to exit out of the Docker container.

Bring your own key (BYOK)

When your use case requires an external key, users of Vault version 1.12.0 or greater can use BYOK functionality to import an existing encryption key that was generated outside Vault.

The target key for import can originate from an HSM or other external source, and must be prepared according to its origin before you can import it.

Note

Tokenization transformations with imported keys do not currently support convergent tokenization.

The example shown here will use a 256-bit AES key, referred to as the target key. To successfully import the target key, you must perform the following operations to prepare it.

Generate an ephemeral 256-bit AES key.
Wrap the target key using the ephemeral AES key with AES-KWP.
Wrap the AES key under the Vault wrapping key using RSAES-OAEP with MGF1 and either SHA-1, SHA-224, SHA-256, SHA-384, or SHA-512.
Delete the ephemeral AES key.
Append the wrapped target key to the wrapped AES key.
Base64 encode the result.

A specific code example for preparing and wrapping the key for import is beyond the scope of this tutorial. For more details about wrapping the key for import including instructions for wrapping key from an HSM, refer to the key wrapping guide.

Before you can wrap the key for import, you must read the wrapping key from Vault so that it can be used to prepare your key.

$ vault read -field=public_key transform/wrapping_key
-----BEGIN PUBLIC KEY-----
MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAttRvwFiW9kPtTutUeC3s
Q3PXXrCOuHrUlAqJxH/nNj7HpuERSJg+HW70b16YCU7mo+lVhpD+bMW3WVzov3v3
jXu6LSZYPZjlO8cEUtQQfydpWVC98cxjn8WCKrIeI+J211/xQQIsOT8sF9MYe11G
Pz/VGdEj+BKel7HWWD1nJ4i0Uo10iN76r0E5GioN8E5wK8L/11k/puEB3EcWhyPp
6aQ8PUbFqU8cmChd2Jg+Cj0k81sp72vOPkDJ7oAekYeR/iIYUjUINlz2Yob/qt25
AuF1t+TSLcI+RHzTetpL6/oHnbKdyM95MNq4XlELun4x2xOa4Q2EVkczwnb+X1kO
0d3FL2dAmMVxCXQnMdd/e2rJx8HxQZX02R4IfojmvfD5x/yICYrqm6bmv9RzTCxE
myQMVUYZsEfXgCKffW0vmEZOFIKvLkI5iaaHqMPk3SCGpMnniybEFt7dmXAnswpE
mtxf+5/gm+aTBOUmEWy0WYb+hHK0RXnUC9ScRWBd+eOMONEm+N7nUUS4e57+EnjH
OrjshTEU/B2BO8DvfYyuYyFzBiWzCJywj2eOaQxYdg8Y05YgDrE1GVEOceErDxT8
eoP/MQgUvVOAsD1KIAT5WmmNOCZ0HHAHjOKDdZ1FWtasNBjGJI7g5a1oCHuwl1p/
CZ2i8Rd7XOEnWZ+ld7l+qS0CAwEAAQ==
-----END PUBLIC KEY-----

The output is the (4096-bit RSA) wrapping key.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" --request GET --silent \
    $VAULT_ADDR/v1/transform/wrapping_key | jq

Example output:

{
  "request_id": "aa998396-098f-391b-213e-572fd0882608",
  "lease_id": "",
  "renewable": false,
  "lease_duration": 0,
  "data": {
    "public_key": "-----BEGIN PUBLIC KEY-----\nMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAttRvwFiW9kPtTutUeC3s\nQ3PXXrCOuHrUlAqJxH/nNj7HpuERSJg+HW70b16YCU7mo+lVhpD+bMW3WVzov3v3\njXu6LSZYPZjlO8cEUtQQfydpWVC98cxjn8WCKrIeI+J211/xQQIsOT8sF9MYe11G\nPz/VGdEj+BKel7HWWD1nJ4i0Uo10iN76r0E5GioN8E5wK8L/11k/puEB3EcWhyPp\n6aQ8PUbFqU8cmChd2Jg+Cj0k81sp72vOPkDJ7oAekYeR/iIYUjUINlz2Yob/qt25\nAuF1t+TSLcI+RHzTetpL6/oHnbKdyM95MNq4XlELun4x2xOa4Q2EVkczwnb+X1kO\n0d3FL2dAmMVxCXQnMdd/e2rJx8HxQZX02R4IfojmvfD5x/yICYrqm6bmv9RzTCxE\nmyQMVUYZsEfXgCKffW0vmEZOFIKvLkI5iaaHqMPk3SCGpMnniybEFt7dmXAnswpE\nmtxf+5/gm+aTBOUmEWy0WYb+hHK0RXnUC9ScRWBd+eOMONEm+N7nUUS4e57+EnjH\nOrjshTEU/B2BO8DvfYyuYyFzBiWzCJywj2eOaQxYdg8Y05YgDrE1GVEOceErDxT8\neoP/MQgUvVOAsD1KIAT5WmmNOCZ0HHAHjOKDdZ1FWtasNBjGJI7g5a1oCHuwl1p/\nCZ2i8Rd7XOEnWZ+ld7l+qS0CAwEAAQ==\n-----END PUBLIC KEY-----\n"
  },
  "wrap_info": null,
  "warnings": null,
  "auth": null
}

The output is a JSON object with a data.public_key field containing the (4096-bit RSA) wrapping key.

Use the wrapping key value at step 3 in the previously detailed preparation steps. Once you have prepared and base64 encoded the ciphertext, export the value to the environment variable IMPORT_CIPHERTEXT.

Example:

$ export IMPORT_CIPHERTEXT=ceXhQrVMuf70i2qL3DvQu/0AFhkPXAV6JyzbPdTs9A/Twjd8PGs/2XV3VhBgvhb4Fr1xWnVmIUKwxgP+emBlIqwpmoJsnkVNJSMpXP0YG+MkvheB9ATlfGXTlf6RLt7OaOtSSBxeVZQBtuWuVnbatTQiXhhC91J49V4+n1JiDjs0tRpz8hUxuzyedkjXWv8Mn0gD4nHV1GgrxLvNGPrrk2Y3xcZO4MoO3Lp447BjPTXwSwmR2rSOeW9+MsuYPfjx2dKC/uJRr6GM2wyyzaRu+5N+goNDaPzTAcNhutb/isf9wAOC3v8SGYXkF9919/ioxZKejWDscFLKBU/6O3+n7Zgf+NLJP/5qyjFOR6F/OnyXpEaOcD+5zpW0yppo/SqThS14E3jDIDmvwxiDBY0SHMBebLYx/t6Y29P+kAP3dPBXeHejHTZ0kJlQuKNcu2Ge1EkTFvR4vScvFlG4OcZIDzxwVxbS6IR83m/xeFyQhuMc/PntDvIJBnUb+AREtZCtNOcwh2Qei8iHfSttCNzROLRZ4ETaBUEKBBWKJztscDfKFOAGKiJIZeze+nVYK7sf94PMJF1r3ET75IyvwCXUEQoPtacg8nyUx6UVpp4BP6xzeOcEPKr1GPEj1kjcj43+rhozzzrs5AgTknQAcorpUwHFUezQtKWyEh/wastp6elX7V0IBL8YmKmWmbIz2hS+uH9TspQM9BnJL/ex18qgmpwIhfR40hzo

Create a new transformation role named legacy-system to use for the transformation that you will import the key into.

$ vault write transform/role/legacy-system transformations=application-form
Success! Data written to: transform/role/legacy-system

$ curl \
    --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data '{"transformations": ["application-form"]}' \
    $VAULT_ADDR/v1/transform/role/legacy-system

Note

This command is expected to produce no output.

Import the key into the application-form transformation. Imported keys do not support rotation by default, so include the allow_rotation parameter and set its value to true so that you can also try rotating the imported key. Add the allowed_roles parameter and specify the legacy-system role.

$ vault write transform/transformations/tokenization/application-form/import \
    ciphertext=$IMPORT_CIPHERTEXT allowed_roles=legacy-system allow_rotation=true
Success! Data written to: transform/transformations/tokenization/application-form/import

Create a JSON payload containing ciphertext value and any additional parameters.

$ cat > payload.json << EOF
{
  "ciphertext": "ceXhQrVMuf70i2qL3DvQu/0AFhkPXAV6JyzbPdTs9A/Twjd8PGs/2XV3VhBgvhb4Fr1xWnVmIUKwxgP+emBlIqwpmoJsnkVNJSMpXP0YG+MkvheB9ATlfGXTlf6RLt7OaOtSSBxeVZQBtuWuVnbatTQiXhhC91J49V4+n1JiDjs0tRpz8hUxuzyedkjXWv8Mn0gD4nHV1GgrxLvNGPrrk2Y3xcZO4MoO3Lp447BjPTXwSwmR2rSOeW9+MsuYPfjx2dKC/uJRr6GM2wyyzaRu+5N+goNDaPzTAcNhutb/isf9wAOC3v8SGYXkF9919/ioxZKejWDscFLKBU/6O3+n7Zgf+NLJP/5qyjFOR6F/OnyXpEaOcD+5zpW0yppo/SqThS14E3jDIDmvwxiDBY0SHMBebLYx/t6Y29P+kAP3dPBXeHejHTZ0kJlQuKNcu2Ge1EkTFvR4vScvFlG4OcZIDzxwVxbS6IR83m/xeFyQhuMc/PntDvIJBnUb+AREtZCtNOcwh2Qei8iHfSttCNzROLRZ4ETaBUEKBBWKJztscDfKFOAGKiJIZeze+nVYK7sf94PMJF1r3ET75IyvwCXUEQoPtacg8nyUx6UVpp4BP6xzeOcEPKr1GPEj1kjcj43+rhozzzrs5AgTknQAcorpUwHFUezQtKWyEh/wastp6elX7V0IBL8YmKmWmbIz2hS+uH9TspQM9BnJL/ex18qgmpwIhfR40hzo", "allowed_roles": "legacy-system", "allow_rotation": true
}
EOF

Import the key.

$ curl \
    --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data @payload.json \
    $VAULT_ADDR/v1/transform/transformations/tokenization/application-form/import

Note

This command is expected to produce no output.

Try using the newly imported key to encode a value and some metadata.

$ vault write transform/encode/legacy-system value=887345 \
     transformation=application-form \
     ttl=8h \
     metadata="Organization=ColoTown" \
     metadata="Purpose=Server-Cage" \
     metadata="Type=PIN"
Key              Value
---              -----
encoded_value    eRwUjS2L9e5mNKLD9Rqt6c49QfsWiqbCEWjtaM334NStmZRG4K7parh6UXJqhYpMVLT9Fe81q

Create a JSON payload.

$ cat > payload.json << EOF
{
   "value": "887345",
   "metadata": "organization=ColoTown,purpose=Server-Cage,type=PIN",
   "transformation": "application-form"
}
EOF

Encode the values.

$ curl \
    --silent \
    --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data @payload.json \
    $VAULT_ADDR/v1/transform/encode/legacy-system | jq

Example output:

{
  "request_id": "08754cc7-90a3-e68a-2824-0a35db768ccd",
  "lease_id": "",
  "renewable": false,
  "lease_duration": 0,
  "data": {
    "encoded_value": "Q4tYgFXHxUXeL61wuEeFMMAeho1gXj8yw5VtNKNUY4NRdBCkvY9FRB"
  },
  "wrap_info": null,
  "warnings": null,
  "auth": null
}

Note

The response payload may return a tweak along with the encoded value if the tweak_source for the specified transformation is set to generated. The resource owner should properly store this tweak, which must be supplied back when decrypting the encoded value.

The imported key is working, and the encoded value returned by the application-form transformation is using the imported key.

Note

TO import subsequent versions of the key, you must use the import_version API endpoint.

Let's take a look a the key information.

$ vault read transform/tokenization/keys/application-form
Key                       Value
---                       -----
auto_rotate_period        0s
latest_version            1
min_available_version     0
min_decryption_version    1
name                      application-form
type                      aes256-gcm96

$ curl \
    --silent \
    --header "X-Vault-Token: $VAULT_TOKEN" \
    --request GET \
    $VAULT_ADDR/v1/transform/tokenization/keys/application-form | jq

Example output:

{
  "request_id": "93d63de8-c83d-3190-3e6a-ec33ecdf8478",
  "lease_id": "",
  "renewable": false,
  "lease_duration": 0,
  "data": {
    "auto_rotate_period": 0,
    "latest_version": 1,
    "min_available_version": 0,
    "min_decryption_version": 1,
    "name": "application-form",
    "type": "aes256-gcm96"
  },
  "wrap_info": null,
  "warnings": null,
  "auth": null
}

The key's latest_version is currently 1..

Rotate the key.

Note

Once an imported key is rotated within Vault, it will no longer support importing key material with the import_version endpoint.

$ vault write -force transform/tokenization/keys/application-form/rotate
Success! Data written to: transform/tokenization/keys/application-form/rotate

$ curl \
    --silent \
    --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    $VAULT_ADDR/v1/transform/tokenization/keys/application-form/rotate

Note

This command is expected to produce no output.

Check the key information once more.

$ vault read transform/tokenization/keys/application-form
Key                       Value
---                       -----
auto_rotate_period        0s
latest_version            2
min_available_version     0
min_decryption_version    1
name                      application-form
type                      aes256-gcm96

$ curl \
    --silent \
    --header "X-Vault-Token: $VAULT_TOKEN" \
    --request GET \
    $VAULT_ADDR/v1/transform/tokenization/keys/application-form | jq

Example output:

{
  "request_id": "a01d18ea-8219-0187-12ce-d0012dfbeb18",
  "lease_id": "",
  "renewable": false,
  "lease_duration": 0,
  "data": {
    "auto_rotate_period": 0,
    "latest_version": 2,
    "min_available_version": 0,
    "min_decryption_version": 1,
    "name": "application-form",
    "type": "aes256-gcm96"
  },
  "wrap_info": null,
  "warnings": null,
  "auth": null
}

The key's latest_version is currently 2., and you can no longer import external versions of the key as it is now internally maintained by Vault.

Clean up

Unset the VAULT_TOKEN environment variable.
```
$ unset VAULT_TOKEN
```
Unset the VAULT_ADDR environment variable.
```
$ unset VAULT_ADDR
```
Unset the IMPORT_CIPHERTEXT environment variable.
```
$ unset IMPORT_CIPHERTEXT
```
Stop and remove the Postgres container.
```
$ docker rm postgres --force
```
You can stop the Vault dev server by pressing Ctrl+C where the server is running. Or, execute the following command.
```
$ pgrep -f vault | xargs kill
```

Summary

Transformation secrets engine introduced tokenization transformation feature which replaces sensitive data with unique value (token) that are unrelated to the original value in any algorithmic sense. This can help organizations to meet certain industry standards.

If retaining the original data format is important, refer to the Transform Secrets Engine to learn about the format preserving encryption (FPE) transformation.

Help and Reference

24 tutorials

Vault Enterprise
Learn features that are only available to Vault Enterprise.
- Vault

Challenge

Solution

Characteristics of the tokenization transformation:

Prerequisites

Policy requirements

Lab setup

Setup the Transform secrets engine

Tokenize secrets

Convergent tokenization

Lookup token

Key rotation

Setup external token storage

Bring your own key (BYOK)

Clean up

Summary

Help and Reference

This tutorial also appears in: