Restore a Vault snapshot

Manually restore data for a Vault cluster leader node with a saved snapshot.

Before you start

You must have a working knowledge of how Vault saves data.
You must have a valid Vault cluster configuration using integrated storage.
You must know, and be able to contact your unseal/recovery key holders. In addition to the new root token generated during reinitialization, you need the original cluster unseal keys to unseal Vault after restoring the snapshot.
You must have permission to access encrypted data in backed storage.

Step 1: Isolate your test instance

Upgrade or restore tests should always be performed in a fully isolated network environment. Isolation is critical to prevent both unwanted cluster-to-cluster communication (which maintains data consistency) and to stop the test instance from attempting to revoke 3rd-party credentials (secrets, etc.). If the test instance revokes live credentials, they may expire, which might result in irrevocable leases for the production Vault cluster from which the snapshot was taken.

Here are concrete steps that you can take to effectively isolate the Vault server while testing snapshot restoration and server upgrades.

Remove the test server from production load balancers. Deregister the instance from any external or internal load balancer target groups, and confirm no client traffic can still reach the node through service discovery or DNS.
Block inbound and outbound network access. Apply restrictive security groups, firewall rules, or ACL rules so the server only allows access from an administrative workstation or bastion host. Deny outbound access to production dependencies unless explicitly required for testing.
Place the test server on a separate subnet or VLAN. Move the server into a non-routable test subnet or an isolated network segment, and ensure there is no routing path back to production application networks.
Disable cluster participation. Prevent the server from joining or rejoining the production Vault cluster. Verify the server cannot communicate with production storage backends, retry_join targets, or cluster peers.
Use separate DNS or no DNS registration. Do not register the instance in production DNS or service discovery. If you require name resolution, use a temporary hostname that clients will not query.
Stop or disable automation that could reconnect the test server. Pause auto-scaling, orchestration, configuration management, or monitoring actions that might reattach the host to production. Disable scripts that automatically restart Vault with production settings.
Replace production-integrated configuration values. Remove or override production listener addresses, storage settings, seal configuration, telemetry sinks, and audit destinations if they would connect outward. Point the restored instance only to test-safe resources.
Restrict operator access paths. Limit administration to a bastion host, VPN, or console session. Log who can access the isolated server during testing.
Validate isolation before restore. Test that production clients cannot connect to the instance. Confirm the instance cannot reach production peers, storage, KMS, or service endpoints.

You can confidently restore the snapshot once you:

Confirm the safety of you network, configuration, and access controls.
Conduct functional testing with non-production users, tokens, and workflows.

Step 2: Bring your Vault cluster back online

Your Vault cluster must be online to restore a snapshot.

Resolve the circumstances that required you to restore from backup, reinitialize your Vault cluster with new storage, and authenticate with the new initial root token generated during re-initialization. The new root token is temporary as you will overwrite the cluster state with the snapshot data.

Step 3: Copy the snapshot file to the cluster

You must save your snapshot file as a local file on the cluster to restore the data.

To restore a snapshot to a disaster recovery replication cluster, you must copy your Vault snapshot files for the primary and DR replica clusters onto restored members of the respective clusters.

To restore a snapshot to a performance replication cluster, you must copy your Vault snapshot files for the primary and secondary performance replica clusters onto restored members of the respective clusters.

Step 4: Force a snapshot restore

You must use force the snapshot restore since the auto-unseal or Shamir keys are not consistent with the snapshot data, which came from a different cluster.

Run vault operator raft snapshot restore with the local snapshot file path and the -force flag:

$ vault operator raft snapshot restore -force <local_file_path>

For example:

$ vault operator raft snapshot restore -force /tmp/snapshots/backup.snap

Call the /v1/sys/storage/raft/snapshot-force endpoint with the local snapshot file path:

$ curl                                       \
    --request POST                           \
    --header "X-Vault-Token: ${VAULT_TOKEN}" \
    --data-binary @<local_file_path>         \
    ${VAULT_ADDR}/v1/sys/storage/raft/snapshot-force

For example:

$ curl                                       \
    --request POST                           \
    --header "X-Vault-Token: ${VAULT_TOKEN}" \
    --data-binary /tmp/snapshots/backup.snap \
    ${VAULT_ADDR}/v1/sys/storage/raft/snapshot-force

Step 5: Unseal Vault

Have each person with an unseal key share run vault operator unseal with their original unseal key until reaching the number of key shares required by your original quorum configuration to unseal your Vault cluster:

$ vault operator unseal

Enter the unseal key when prompted:

Unseal Key (will be hidden):

Have each person with an unseal key share run v1/sys/unseal with their original unseal key until reaching the number of key shares required by your original quorum configuration to unseal your Vault cluster:

$ curl                                       \
   --request POST                            \
   --header "X-Vault-Token: ${VAULT_TOKEN}"  \
   --data '{"key": "<unseal_key>"}'          \
   ${VAULT_ADDR}/v1/sys/unseal