Transit secrets engine

The transit secrets engine handles cryptographic functions on data in-transit. Vault doesn't store the data sent to the secrets engine. It can also be viewed as "cryptography as a service" or "encryption as a service". The transit secrets engine can also sign and verify data; generate hashes and HMACs of data; and act as a source of random bytes.

The primary use case for transit is to encrypt data from applications while still storing that encrypted data in some primary data store. This relieves the burden of proper encryption/decryption from application developers and pushes the burden onto the operators of Vault.

Key derivation is supported, which allows the same key to be used for multiple purposes by deriving a new key based on a user-supplied context value. In this mode, convergent encryption can optionally be supported, which allows the same input values to produce the same ciphertext.

Datakey generation allows processes to request a high-entropy key of a given bit length be returned to them, encrypted with the named key. Normally this will also return the key in plaintext to allow for immediate use, but this can be disabled to accommodate auditing requirements.

Working set management

The Transit engine supports versioning of keys. Key versions that are earlier than a key's specified min_decryption_version gets archived, and the rest of the key versions belong to the working set. This is a performance consideration to keep key loading fast, as well as a security consideration: by disallowing decryption of old versions of keys, found ciphertext corresponding to obsolete (but sensitive) data can not be decrypted by most users, but in an emergency the min_decryption_version can be moved back to allow for legitimate decryption.

Currently this archive is stored in a single storage entry. With some storage backends, notably those using Raft or Paxos for HA capabilities, frequent rotation may lead to a storage entry size for the archive that is larger than the storage backend can handle. For frequent rotation needs, using named keys that correspond to time bounds (e.g. five-minute periods floored to the closest multiple of five) may provide a good alternative, allowing for several keys to be live at once and a deterministic way to decide which key to use at any given time.

NIST rotation guidance

Periodic rotation of the encryption keys is recommended, even in the absence of compromise. For AES-GCM keys, rotation should occur before approximately 2³² encryptions have been performed by a key version, following the guidelines of NIST publication 800-38D. It is recommended that operators estimate the encryption rate of a key and use that to determine a frequency of rotation that prevents the guidance limits from being reached. For example, if one determines that the estimated rate is 40 million operations per day, then rotating a key every three months is sufficient.

Key types

As of now, the transit secrets engine supports the following key types (all key types also generate separate HMAC keys):

aes128-gcm96: AES-GCM with a 128-bit AES key and a 96-bit nonce; supports encryption, decryption, key derivation, and convergent encryption
aes256-gcm96: AES-GCM with a 256-bit AES key and a 96-bit nonce; supports encryption, decryption, key derivation, and convergent encryption (default)
chacha20-poly1305: ChaCha20-Poly1305 with a 256-bit key; supports encryption, decryption, key derivation, and convergent encryption
ed25519: Ed25519; supports signing, signature verification, and key derivation
ecdsa-p256: ECDSA using curve P-256; supports signing and signature verification
ecdsa-p384: ECDSA using curve P-384; supports signing and signature verification
ecdsa-p521: ECDSA using curve P-521; supports signing and signature verification
rsa-2048: 2048-bit RSA key; supports encryption, decryption, signing, and signature verification
rsa-3072: 3072-bit RSA key; supports encryption, decryption, signing, and signature verification
rsa-4096: 4096-bit RSA key; supports encryption, decryption, signing, and signature verification
hmac: HMAC; supporting HMAC generation and verification.
managed_key: Managed key; supports a variety of operations depending on the backing key management solution. See Managed Keys for more information.

Note: In FIPS 140-2 mode, the following algorithms are not certified and thus should not be used: chacha20-poly1305 and ed25519.

Note: All key types support HMAC operations through the use of a second randomly generated key created key creation time or rotation. The HMAC key type only supports HMAC, and behaves identically to other algorithms with respect to the HMAC operations but supports key import. By default, the HMAC key type uses a 256-bit key.

RSA operations use one of the following methods:

OAEP (encrypt, decrypt), with SHA-256 hash function and MGF,
PSS (sign, verify), with configurable hash function also used for MGF, and
PKCS#1v1.5: (sign, verify), with configurable hash function.

Convergent encryption

Convergent encryption is a mode where the same set of plaintext+context always result in the same ciphertext. It does this by deriving a key using a key derivation function but also by deterministically deriving a nonce. Because these properties differ for any combination of plaintext and ciphertext over a keyspace the size of 2^256, the risk of nonce reuse is near zero.

This has many practical uses. One common usage mode is to allow values to be stored encrypted in a database, but with limited lookup/query support, so that rows with the same value for a specific field can be returned from a query.

To accommodate for any needed upgrades to the algorithm, different versions of convergent encryption have historically been supported:

Version 1 required the client to provide their own nonce, which is highly flexible but if done incorrectly can be dangerous. This was only in Vault 0.6.1, and keys using this version cannot be upgraded.
Version 2 used an algorithmic approach to deriving the parameters. However, the algorithm used was susceptible to offline plaintext-confirmation attacks, which could allow attackers to brute force decryption if the plaintext size was small. Keys using version 2 can be upgraded by simply performing a rotate operation to a new key version; existing values can then be rewrapped against the new key version and will use the version 3 algorithm.
Version 3 uses a different algorithm designed to be resistant to offline plaintext-confirmation attacks. It is similar to AES-SIV in that it uses a PRF to generate the nonce from the plaintext.

Setup

Most secrets engines must be configured in advance before they can perform their functions. These steps are usually completed by an operator or configuration management tool.

Enable the Transit secrets engine:
```
$ vault secrets enable transit
Success! Enabled the transit secrets engine at: transit/
```
By default, the secrets engine will mount at the name of the engine. To enable the secrets engine at a different path, use the -path argument.

Create a named encryption key:

$ vault write -f transit/keys/my-key
Success! Data written to: transit/keys/my-key

Usually each application has its own encryption key.

Usage

After the secrets engine is configured and a user/machine has a Vault token with the proper permission, it can use this secrets engine.

Encrypt some plaintext data using the /encrypt endpoint with a named key:
NOTE: All plaintext data must be base64-encoded. The reason for this requirement is that Vault does not require that the plaintext is "text". It could be a binary file such as a PDF or image. The easiest safe transport mechanism for this data as part of a JSON payload is to base64-encode it.
```
$ vault write transit/encrypt/my-key plaintext=$(echo "my secret data" | base64)

Key           Value
---           -----
ciphertext    vault:v1:8SDd3WHDOjf7mq69CyCqYjBXAiQQAVZRkFM13ok481zoCmHnSeDX9vyf7w==
```
The returned ciphertext starts with vault:v1:. The first prefix (vault) identifies that it has been wrapped by Vault. The v1 indicates the key version 1 was used to encrypt the plaintext; therefore, when you rotate keys, Vault knows which version to use for decryption. The rest is a base64 concatenation of the initialization vector (IV) and ciphertext.
Note that Vault does not store any of this data. The caller is responsible for storing the encrypted ciphertext. When the caller wants the plaintext, it must provide the ciphertext back to Vault to decrypt the value.
Vault HTTP API imposes a maximum request size of 32MB to prevent a denial of service attack. This can be tuned per listener block in the Vault server configuration.
Decrypt a piece of data using the /decrypt endpoint with a named key:
```
$ vault write transit/decrypt/my-key ciphertext=vault:v1:8SDd3WHDOjf7mq69CyCqYjBXAiQQAVZRkFM13ok481zoCmHnSeDX9vyf7w==

Key          Value
---          -----
plaintext    bXkgc2VjcmV0IGRhdGEK
```
The resulting data is base64-encoded (see the note above for details on why). Decode it to get the raw plaintext:
```
$ base64 --decode <<< "bXkgc2VjcmV0IGRhdGEK"
my secret data
```
It is also possible to script this decryption using some clever shell scripting in one command:
```
$ vault write -field=plaintext transit/decrypt/my-key ciphertext=... | base64 --decode
my secret data
```
Using ACLs, it is possible to restrict using the transit secrets engine such that trusted operators can manage the named keys, and applications can only encrypt or decrypt using the named keys they need access to.
Rotate the underlying encryption key. This will generate a new encryption key and add it to the keyring for the named key:
```
$ vault write -f transit/keys/my-key/rotate
Success! Data written to: transit/keys/my-key/rotate
```
Future encryptions will use this new key. Old data can still be decrypted due to the use of a key ring.
Upgrade already-encrypted data to a new key. Vault will decrypt the value using the appropriate key in the keyring and then encrypt the resulting plaintext with the newest key in the keyring.
```
$ vault write transit/rewrap/my-key ciphertext=vault:v1:8SDd3WHDOjf7mq69CyCqYjBXAiQQAVZRkFM13ok481zoCmHnSeDX9vyf7w==

Key           Value
---           -----
ciphertext    vault:v2:0VHTTBb2EyyNYHsa3XiXsvXOQSLKulH+NqS4eRZdtc2TwQCxqJ7PUipvqQ==
```
This process does not reveal the plaintext data. As such, a Vault policy could grant almost an untrusted process the ability to "rewrap" encrypted data, since the process would not be able to get access to the plaintext data.

Bring your own key (BYOK)

Note: Key import functionality supports cases in which there is a need to bring in an existing key from an HSM or other outside system. It is more secure to have Transit generate and manage a key within Vault.

First, the wrapping key needs to be read from transit:

$ vault read transit/wrapping_key

The wrapping key will be a 4096-bit RSA public key.

Then the wrapping key is used to create the ciphertext input for the import endpoint, as described below. In the below, the target key refers to the key being imported.

HSM

If the key is being imported from an HSM that supports PKCS#11, there are two possible scenarios:

If the HSM supports the CKM_RSA_AES_KEY_WRAP mechanism, that can be used to wrap the target key using the wrapping key.
Otherwise, two mechanisms can be combined to wrap the target key. First, a 256-bit AES key should be generated and then used to wrap the target key using the CKM_AES_KEY_WRAP_KWP mechanism. Then the AES key should be wrapped under the wrapping key using the CKM_RSA_PKCS_OAEP mechanism using MGF1 and either SHA-1, SHA-224, SHA-256, SHA-384, or SHA-512.

The ciphertext is constructed by appending the wrapped target key to the wrapped AES key.

The ciphertext bytes should be base64-encoded.

Manual process

If the target key is not stored in an HSM or KMS, the following steps can be used to construct the ciphertext for the input of the import endpoint:

Generate an ephemeral 256-bit AES key.
Wrap the target key using the ephemeral AES key with AES-KWP.

Note: When wrapping a symmetric key (such as an AES or ChaCha20 key), wrap the raw bytes of the key. For instance, with an AES 128-bit key, this'll be a byte array 16 characters in length that will directly be wrapped without base64 or other encodings.

When wrapping an asymmetric key (such as a RSA or ECDSA key), wrap the PKCS8 encoded format of this key, in raw DER/binary form. Do not apply PEM encoding to this blob prior to encryption and do not base64 encode it.

Wrap the AES key under the Vault wrapping key using RSAES-OAEP with MGF1 and either SHA-1, SHA-224, SHA-256, SHA-384, or SHA-512.
Delete the ephemeral AES key.
Append the wrapped target key to the wrapped AES key.
Base64 encode the result.

For more details about wrapping the key for import into transit, see the key wrapping guide.

Tutorial

Refer to the Encryption as a Service: Transit Secrets Engine tutorial to learn how to use the transit secrets engine to handle cryptographic functions on data in-transit.

API

The Transit secrets engine has a full HTTP API. Please see the Transit secrets engine API for more details.