HSM Integration - Seal Wrap

12min
Enterprise
Vault
Video

Enterprise Only

Vault's Hardware Security Module (HSM) auto-unseal and Seal Wrap features require a Vault Enterprise Plus license.

Vault Enterprise integrates with Hardware Security Module (HSM) platforms to opt-in automatic unsealing. HSM integration provides three pieces of special functionality:

Root Key Wrapping: Vault protects its root key (previously known as master key) by transiting it through the HSM for encryption rather than splitting into key shares
Automatic Unsealing: Vault stores its encrypted root key in storage, allowing for automatic unsealing
Seal Wrapping to provide FIPS KeyStorage-conforming functionality for Critical Security Parameters

In some large organizations, there is a fair amount of complexity in designating key officers, who might be available to unseal Vault installations as the most common pattern is to deploy Vault immutably. As such automating unseal using an HSM provides a simplified yet secure way of unsealing Vault nodes as they get deployed.

Vault pulls its encrypted root key from storage and transit it through the HSM for decryption via PKCS #11 API. Once the root key is decrypted, Vault uses the root key to decrypt the encryption key to resume with Vault operations.

Challenge

The Federal Information Processing Standard (FIPS) 140-2 is a U.S. Government computer security standard used to accredit cryptography modules. If your product or service does not follow FIPS' security requirements, it may complicate your ability to operate with U.S. Government data.

Aside from doing business with U.S. government, your organization may care about FIPS which approves various cryptographic ciphers for hashing, signature, key exchange, and encryption for security.

Solution

Integrate Vault with FIPS 140-2 certified HSM and enable the Seal Wrap feature to protect your data.

Vault encrypts secrets using 256-bit AES in GCM mode with a randomly generated nonce prior to writing them to its persistent storage. By enabling seal wrap, Vault wraps your secrets with an extra layer of encryption leveraging the HSM encryption and decryption.

Benefits of the Seal Wrap:

Conformance with FIPS 140-2 directives on Key Storage and Key Transport as certified by Leidos
Supports FIPS level of security equal to HSM
- For example, if you use Level 3 hardware encryption on an HSM, Vault will be using FIPS 140-2 Level 3 cryptography
Allows Vault to be deployed in high security GRC environments (e.g. PCI-DSS, HIPAA) where FIPS guidelines important for external audits
Pathway for Vault's use in managing Department of Defense's (DOD) or North Atlantic Treaty Organization (NATO) military secrets

Prerequisites

This intermediate operations tutorial assumes that you have:

A supported HSM cluster to integrate with Vault
Vault Enterprise version 0.9.0 or later

Note

To explore Vault Enterprise features, you can sign up for a free 30-day trial from here.

Step 1: Configure HSM Auto-unseal

When a Vault server is started, it normally starts in a sealed state where a quorum of existing unseal keys is required to unseal it. By integrating Vault with HSM, your Vault server can be automatically unsealed by the trusted HSM key provider.

To integrate your Vault Enterprise server with an HSM cluster, the configuration file must define the PKCS11 seal stanza providing necessary connection information.

Example: config-hsm.hcl

# Provide your AWS CloudHSM cluster connection information
seal "pkcs11" {
  lib = "/opt/cloudhsm/lib/libcloudhsm_pkcs11.so"
  slot = "1"
  pin = "vault:Password1"
  key_label = "hsm_demo"
  hmac_key_label = "hsm_hmac_demo"
  generate_key = "true"
}

# Configure the storage backend for Vault
storage "file" {
  path = "/tmp/vault"
}

# Addresses and ports on which Vault will respond to requests
listener "tcp" {
  address          = "0.0.0.0:8200"
  tls_disable      = "true"
}

ui = true
disable_mlock = true

For the purpose of this tutorial, the storage backend is set to the local file system (/tmp/vault) to make the verification step easy.

Warning

Although the listener stanza disables TLS (tls_disable = "true") for this tutorial, Vault should always be used with TLS in production to provide secure communication between clients and the Vault server. It requires a certificate file and key file on each Vault host.

The example configuration defines the following in its seal stanza:

Parameter	Description
`lib`	Path to the PKCS #11 library on the virtual machine where Vault Enterprise is installed.
`slot`	The slot number to use.
`pin`	PKCS #11 PIN for login.
`key_label`	Defines the label of the key you want to use.
`hmac_key_label`	Defines the label of the key you want to use for HMACing.
`generate_key`	If no existing key with the label specified by `key_label` can be found at Vault initialization time, Vault generates a key.

IMPORTANT: Having Vault generate its own key is the easiest way to get up and running, but for security, Vault marks the key as non-exportable. If your HSM key backup strategy requires the key to be exportable, you should generate the key yourself. Refer to the key generation attributes.

For the full list of configuration parameters, refer to the documentation.

If you are using systemd, the ExecStart parameter should point to the correct location of your configuration file. For example, if your configuration file is located at /etc/vault.d/config-hsm.hcl, the vault.service file may look as follow.

Example: /etc/systemd/system/vault.service

[Unit]
Description="HashiCorp Vault - A tool for managing secrets"
Documentation=https://www.vaultproject.io/docs/
Requires=network-online.target
After=network-online.target
ConditionFileNotEmpty=/etc/vault.d/vault.hcl
StartLimitIntervalSec=60
StartLimitBurst=3

[Service]
User=vault
Group=vault
ProtectSystem=full
ProtectHome=read-only
PrivateTmp=yes
PrivateDevices=yes
SecureBits=keep-caps
AmbientCapabilities=CAP_IPC_LOCK
Capabilities=CAP_IPC_LOCK+ep
CapabilityBoundingSet=CAP_SYSLOG CAP_IPC_LOCK
NoNewPrivileges=yes
ExecStart=/usr/local/bin/vault server -config=/etc/vault.d/config-hsm.hcl
ExecReload=/bin/kill --signal HUP $MAINPID
KillMode=process
KillSignal=SIGINT
Restart=on-failure
RestartSec=5
TimeoutStopSec=30
StartLimitInterval=60
StartLimitIntervalSec=60
StartLimitBurst=3
LimitNOFILE=65536
LimitMEMLOCK=infinity

[Install]
WantedBy=multi-user.target

Start the Vault server.
Example:
If starting from the command:
```
$ vault server -config=/etc/vault.d/config-hsm.hcl
```
If starting as a service using systemd, first enable the vault service.
```
$ sudo systemctl enable vault
```
Start the Vault server as a service.
```
$ sudo systemctl start vault
```

Check the Vault status.

$ vault status

Key                      Value
---                      -----
Recovery Seal Type       pkcs11
Initialized              false
Sealed                   true
...

Initialize Vault.

First, set the VAULT_ADDR environment variable.

$ export VAULT_ADDR="http://127.0.0.1:8200"

Initialize Vault.

$ vault operator init

Example output:

Recovery Key 1: iz1XWxe4CM+wrOGqRCx8ex8kB2XvGJEdfjhXFC+MA6Rc
Recovery Key 2: rKZETr6IAy686IxfO3ZBKXPDAOkkwkpSepIME+bjeUT7
Recovery Key 3: 4XA/KJqFOm+jzbBkKQuRVePEYPrQe3H3TmFVmdlUjRFv
Recovery Key 4: lfnaYoZufP0uhooO3mHDAKGNZB5HLP9HYYb+LAfKkUmd
Recovery Key 5: L169hHj3DMpphGsOnS8TEz3Febvdx3vsG3Xr8kGWdUtW

Initial Root Token: s.AWnDagUkKNNbvkENiL72wysn

Success! Vault is initialized

Recovery key initialized with 5 key shares and a key threshold of 3. Please
securely distribute the key shares printed above.

When Vault is initialized while using an HSM, rather than unseal keys being returned to the operator, recovery keys are returned. These are generated from an internal recovery key that is split via Shamir's Secret Sharing, similar to Vault's treatment of unseal keys when running without an HSM. Some Vault operations such as generation of a root token require these recovery keys.

Check the Vault status again.

$ vault status

Key                      Value
---                      -----
Recovery Seal Type       shamir
Initialized              true
Sealed                   false
Total Recovery Shares    1
Threshold                1
Version                  1.2.3+ent.hsm
Cluster Name             vault-cluster-dfcace02
Cluster ID               0fe5ab51-ff92-fdb0-9c37-f739679554f8
HA Enabled               false

Vault is now unsealed (Sealed value is now false).

To verify auto-unseal, stop and restart the Vault server and check its status.

If starting the Vault server as a service using systemd:

$ sudo systemctl restart vault

Verify the Vault server status.

$ vault status
Key                      Value
---                      -----
Recovery Seal Type       shamir
Initialized              true
Sealed                   false
...

Step 2: Enable Seal Wrap

Note

For FIPS 140-2 compliance, seal wrap requires FIPS 140-2 Certified HSM which is supported by Vault Enterprise.

For some values, seal wrapping is always enabled including the recovery key, any stored key shares, the root key, the keyring, and more. When working with the key/value secrets engine, you can enable seal wrap to wrap all data.

To compare seal wrapped data against unwrapped data, enable key/value v1 secrets engine at two different paths: kv-unwrapped and kv-seal-wrapped.
Enable k/v v1 without seal wrap at kv-unwrapped.
```
$ vault secrets enable -path=kv-unwrapped kv
```
Enable k/v v1 with seal wrap. To do so, use the '-seal-wrap' flag when you enable the KV workflow.
```
$ vault secrets enable -path=kv-seal-wrapped -seal-wrap kv
```
To enable seal wrap, pass the -seal-wrap flag when you enable a secrets engine.

List the enabled secrets engines with details.

$ vault secrets list -detailed

Path                Plugin       Accessor              ...    Seal Wrap    ...
----                ------       --------                     -----------  ...
cubbyhole/          cubbyhole    cubbyhole_b36dd7e1    ...    false        ...
identity/           identity     identity_b5650a96     ...    false        ...
kv-seal-wrapped/    kv           kv_fe02767b           ...    true         ...
kv-unwrapped/       kv           kv_36d321c6           ...    false        ...
...

Notice that the Seal Wrap parameter value is true for kv-seal-wrapped.

To compare seal wrapped data against unwrapped data, enable key/value v1 secrets engine at two different paths: kv-unwrapped and kv-seal-wrapped.

Enable k/v v1 without seal wrap at kv-unwrapped.

$ curl --header "X-Vault-Token: <TOKEN>" \
      --request POST \
      --data '{ "type": "kv" }' \
      http://127.0.0.1:8200/v1/sys/mounts/kv-unwrapped

Enable k/v v1 with seal wrap. To enable seal wrap, set the seal_wrap parameter to true in the request payload when you enable a secrets engine.

$ curl --header "X-Vault-Token: <TOKEN>" \
      --request POST \
      --data '{ "type": "kv", "seal_wrap": true }' \
      http://127.0.0.1:8200/v1/sys/mounts/kv-seal-wrapped

List the enabled secrets engines.

$ curl --header "X-Vault-Token: <TOKEN>" \
      http://127.0.0.1:8200/v1/sys/mounts | jq
{
  # ...snip...
    "kv-seal-wrapped/": {
      "accessor": "kv_fe02767b",
      "config": {
        "default_lease_ttl": 0,
        "force_no_cache": false,
        "max_lease_ttl": 0
      },
      "description": "",
      "local": false,
      "options": null,
      "seal_wrap": true,
      "type": "kv",
      "uuid": "58ab2df7-6de3-7cb6-da85-ca562fefeb67"
    },
    "kv-unwrapped/": {
      "accessor": "kv_36d321c6",
      "config": {
        "default_lease_ttl": 0,
        "force_no_cache": false,
        "max_lease_ttl": 0
      },
      "description": "",
      "local": false,
      "options": null,
      "seal_wrap": false,
      "type": "kv",
      "uuid": "460a5be2-4150-87b1-592c-8319a95cc852"
    },
  # ...snip...
}

Notice that the seal_wrap parameter value is true at kv-seal-wrapped/.

Open a web browser and launch the Vault UI (e.g. http://127.0.0.1:8200/ui) and then login.
Select Enable new engine.
Select KV from the list, and then click Next.
Enter kv-unwrapped in the path field and select Version 1 for KV version.
Return to the Secrets Engines page and click Enable Engine.
Select KV from the list, and then click Next.
Enter kv-seal-wrapped in the path field. Select Version 1 for KV version.
Click Method Options to expand, and select the check box for Seal Wrap.
Click Enable Engine.

Step 3: Test the Seal Wrap Feature

In this step, you are going to:

Write some test data
View the encrypted secrets

Write a secret at kv-unwrapped/unwrapped for testing.

$ vault kv put kv-unwrapped/unwrapped password="my-long-password"

Read the path to verify.

$ vault kv get kv-unwrapped/unwrapped
====== Data ======
Key         Value
---         -----
password    my-long-password

Write the same secret at kv-seal-wrapped/wrapped for testing.

$ vault kv put kv-seal-wrapped/wrapped password="my-long-password"

Read the path to verify.

$ vault kv get kv-seal-wrapped/wrapped
====== Data ======
Key         Value
---         -----
password    my-long-password

Using a valid token, you can write and read secrets the same way regardless of the seal wrap.

Write a secret at kv-unwrapped/unwrapped.

First, create an API request payload containing test data to write.

$ tee payload.json <<EOF
{
  "data": {
    "password": "my-long-password"
  }
}
EOF

Write secret at kv-unwrapped/unwrapped.

$ curl --header "X-Vault-Token: <TOKEN>" \
      --request POST \
      --data @payload.json \
      http://127.0.0.1:8200/v1/kv-unwrapped/unwrapped

Read the path to verify.

$ curl --header "X-Vault-Token: <TOKEN>" \
      http://127.0.0.1:8200/v1/kv-unwrapped/unwrapped | jq
{
  # ...snip...
  "data": {
    "data": {
      "password": "my-long-password"
    }
  },
  # ...snip...
}

Write the same secret at kv-seal-wrapped/wrapped.

$ curl --header "X-Vault-Token: <TOKEN>" \
      --request POST \
      --data @payload.json \
      http://127.0.0.1:8200/v1/kv-seal-wrapped/wrapped

Read the path to verify.

$ curl --header "X-Vault-Token: <TOKEN>" \
      http://127.0.0.1:8200/v1/kv-seal-wrapped/wrapped | jq
{
  ...
  "data": {
    "data": {
      "password": "my-long-password"
    }
  },
  ...
}

Using a valid token, you can write and read secrets the same way regardless of the seal wrap.

View the encrypted secrets

Remember that the Vault server was configured to use the local file system (/tmp/vault) as its storage backend in this example.

# Configure the storage backend for Vault
storage "file" {
  path = "/tmp/vault"
}

SSH into the machine where the Vault server is running, and check the stored values in the /tmp/vault directory.

$ cd /tmp/vault/logical

Under the /tmp/vault/logical directory, there are two sub-directories. One maps to kv-unwrapped/ and another maps to kv-seal-wrapped/ although you cannot tell by the folder names.

View the secret at rest. One of the directory maps to kv-unwrapped/unwrapped.

Example:

$ cd 2da357cd-55f2-7eed-c46e-c477b70bed18

View its content - password value is encrypted.

$ cat _unwrapped
{"Value":"AAAAAQICk547prhuhMiBXLq2lx8ZkMpSB3p+GKHAwuMhKrZGSeqsFevMS6YoqTVlbvpU9B4zWPZ2HA
SeNZ3YMw=="}

Another directory maps to kv-seal-wrapped/wrapped.

$ cd ../5bcea44d-28a3-87af-393b-c6d398fe41d8

View its content. The password value is encrypted.

$ cat _wrapped
{"Value":"ClBAg9oN7zBBaDBZcsilDAyGkL7soPe7vBA5+ADADuyzo8GuHZHb9UFN2nF1h0OpKEgCIkG3JNHcXt
tZqCi6szcuNBgF3pwhWGwB4FREM3b5CRIQYK7239Q92gRGrcBBeZD6ghogEtSBDmZJBahk7n4lIYF3X4iBqmwZgH
Vo4lzWur7rzncgASofCIIhENEEGghoc21fZGVtbyINaHNtX2htYWNfZGVtb3M="}

Secrets are encrypted regardless; however, the seal-wrapped value is significantly longer despite the fact that both values are the same, my-long-password.

When Vault's Seal Wrap feature is used with a FIPS 140-2 certified HSM, Vault will store Critical Security Parameters (CSPs) in a manner that is compliant with KeyStorage and KeyTransit requirements.

Help and Reference

12 tutorials

Deploy Cluster with Integrated Storage
If you are responsible for setting up and maintaining a Vault cluster using integrated storage as a persistence layer, get started here.
- Vault
5 tutorials

Auto Unseal
Automatically unsealing Vault reduces the operational complexity of keeping the Vault unseal keys secure.
- Vault

Challenge

Solution

Benefits of the Seal Wrap:

Prerequisites

Step 1: Configure HSM Auto-unseal

Step 2: Enable Seal Wrap

Step 3: Test the Seal Wrap Feature

View the encrypted secrets

Help and Reference

This tutorial also appears in: