Backup and Restore HCS on Azure State

10min
Consul

One of the main advantages of deploying Consul as a managed service is that all the routine maintenance operations are taken care of by HashiCorp SREs.

This includes automated backups that will save your Consul state and will permit you to restore it at any given time.

This removes the need to have a backup plan for your datacenter but, you might still want to be able to take a backup manually, for example before a configuration change so that you can quickly restore it in case something goes wrong.

In this tutorial, you will learn about the different types of snapshots available in your HCS on Azure datacenter, and perform snapshot and restore operations for your Consul datacenter.

Prerequisites

To complete this tutorial, you will need the following.

HashiCorp Consul Service on Azure
Basic command line access
The Azure CLI v2.18.0+
The Azure CLI Extension for HCS on Azure v0.4.9+

Install the HashiCorp extension

While HCS on Azure can be provisioned from the Azure portal, the post provisioning operator workflow, including backup and restore operations, requires you to use the official Azure CLI extension for HCS on Azure. You can install the extension directly from your shell using the following az command. Enter y when prompted to confirm.

$ az extension add --source https://releases.hashicorp.com/hcs/0.4.9/hcs-0.4.9-py2.py3-none-any.whl

Are you sure you want to install this extension? (y/n): y
The installed extension 'hcs' is in preview.

Available snapshots

There are three types of snapshots: Manual, Scheduled, and Automatic.

In this section, you'll review the different types of snapshots. In the next sections, you'll learn the commands available to interact with snapshots in HCS on Azure.

Manual snapshots

Manual snapshots are snapshots manually initiated by either you as an end-user or by an HashiCorp internal operator if needed for maintenance or troubleshooting reasons.

Internal operator snapshots are called HashiCorp Operator Internal Snapshots. They are only created when an HashiCorp internal operator invokes an internal endpoint to trigger one. End-users can’t create these snapshots, and they cannot be renamed or deleted.

They are considered manual snapshots (but they do not count against the quota).

The HashiCorp Operator Internal Snapshots are not functionally different from the other types of snapshots, other than the restriction on creation and deletion. Additionally, they aren’t created unless there is known need for it from the operator’s side (i.e. incident troubleshooting).

Storage limitations

HCS on Azure permits you to keep up to thirty (30) manual snapshots.

Unless you manually delete them, snapshots are persisted up to thirty (30) days after creation. An automated job will run every twenty-four hours to take a new snapshot and remove snapshots older than 30 days.

Scheduled snapshots

Your HCS on Azure cluster state is periodically saved, once every twenty-four hours. These snapshots, called Scheduled, are taken to ensure that you can always restore your Consul datacenter to a previous state.

Note

The scheduled snapshots constitute a valid backup strategy. However, we still suggest you to take a manual snapshot before every major configuration change in case you have multiple major configuration changes in less than 24h.

Automatic snapshots

To add resiliency to your HCS on Azure cluster, additional snapshots are taken automatically before HCS on Azure cluster updates. This will keep your data and state safe and permit HashiCorp to rollback immediately in case of issues.

Available snapshot fields

All the snapshot types are functionally identical. The following information is exposed to help you identify the different snapshots, their type, and the state the snapshot is in.

Name	What is it?	Format example
`finishedAt`	The time that the snapshot completed execution	2020-11-02T18:19:28Z
`id`	The id of the snapshot	snapshot-20201102T181312Z
`name`	The name of the snapshot	snapshot-name
`requestedAt`	The time that the snapshot was requested	2020-11-02T18:19:28Z
`restoredAt`	The time that the snapshot was restored. This field is only populated if the snapshot has been restored	2020-11-02T18:19:28Z
`size`	Size of the snapshot in bytes	4023
`state`	The state of the snapshot	PENDING CREATING STORED FAILED DELETING DELETING_FAILED RESTORING
`type`	The type of the snapshot	Manual Automatic Scheduled

Create a manual snapshot

Manual snapshots can be triggered using the create command. It is necessary to specify a name for the snapshot.

The snapshot will not be created locally, but will be stored on HCS among the automatic and scheduled ones.

$ az hcs snapshot create \
    --resource-group <resource-group> \
    --name <managed-app-name> \
    --snapshot-name <snapshot-name>

{
  "snapshot": {
    "finishedAt": "2021-02-03T09:18:29Z",
    "id": "snapshot-20210203T091717Z",
    "name": "snapshot-hcs-test",
    "requestedAt": "2021-02-03T09:17:18Z",
    "size": "4637",
    "state": "STORED",
    "type": "Manual"
  }
}

List all snapshots

Once the snapshot is created it will be shown in the list of available snapshots.

$ az hcs snapshot list \
    --resource-group <resource-group> \
    --name <managed-app-name>

{
  "snapshots": [
    {
      "finishedAt": "2021-02-03T09:18:29Z",
      "id": "snapshot-20210203T091717Z",
      "name": "snapshot-hcs-test",
      "requestedAt": "2021-02-03T09:17:18Z",
      "size": "4637",
      "state": "STORED",
      "type": "Manual"
    },
    {
      "finishedAt": "2021-02-03T07:56:08Z",
      "id": "snapshot-20210203T075500Z",
      "name": "Scheduled Snapshot",
      "requestedAt": "2021-02-03T07:55:01Z",
      "size": "4640",
      "state": "STORED",
      "type": "Scheduled"
    }
  ]
}

Rename manual snapshots

If needed, it is possible to rename an existing snapshot using the rename option. You will have to use the ID of the snapshot you want to rename as an argument and provide a new name for it.

$ az hcs snapshot rename \
    --resource-group <resource-group> \
    --name <managed-app-name> \
    --snapshot <snapshot-id> \
    --snapshot-name <snapshot-name>

The command will output the details of the snapshot and will show the new name.

{
  "snapshot": {
    "finishedAt": "2021-02-03T09:18:29Z",
    "id": "snapshot-20210203T091717Z",
    "name": "snapshot-manual-20210202",
    "requestedAt": "2021-02-03T09:17:18Z",
    "size": "4637",
    "state": "STORED",
    "type": "Manual"
  }
}

Scheduled or automatic snapshots cannot be renamed. If you try to rename one of them you'll receive the following error message:

HashiCorp Operator Internal Snapshots cannot be renamed

Restore a snapshot

If needed, you can restore an existing snapshot using the restore command.

$ az hcs snapshot restore \
    --resource-group <resource-group> \
    --name <managed-app-name> \
    --snapshot <snapshot-id>

{
  "snapshot": {
    "finishedAt": "2021-02-04T13:27:08Z",
    "id": "snapshot-20210203T091717Z",
    "name": "snapshot-manual-20210202",
    "requestedAt": "2021-02-03T09:17:18Z",
    "restoredAt": "2021-02-04T13:27:08Z",
    "size": "4637",
    "state": "STORED",
    "type": "Manual"
  }
}

You can notice that, after the restore, the snapshot has the restoredAt field populated with the restore timestamp.

Take a snapshot before a restore

Restoring a snapshot will replace your current status with the one contained in the snapshot you are restoring.

To avoid unexpected data loss, you should always take a new snapshot before a restore in case you realize that the snapshot you are restoring does not contain some new configuration.

To simplify the process for you, the restore command allows you the use of a flag, --take-snapshot, to automate the process for you and automatically take a snapshot before restoring the one you want to restore.

$ az hcs snapshot restore \
    --resource-group <resource-group> \
    --name <managed-app-name> \
    --snapshot <snapshot-id> \
    --take-snapshot

Once the command completes, you will be able to find the newly created snapshot using the list command.

$ az hcs snapshot list \
    --resource-group <resource-group> \
    --name <managed-app-name>

{
  "snapshots": [
    {
      "finishedAt": "2021-02-04T18:17:54Z",
      "id": "snapshot-20210204T181753Z",
      "name": "auto",
      "requestedAt": "2021-02-04T18:17:53Z",
      "size": "4628",
      "state": "STORED",
      "type": "Automatic"
    }
  ]
}

As you can confirm, the new snapshot created will be marked as Automatic.

Delete manual snapshots

Finally, you can delete a snapshot using the delete option for the CLI command.

Warning

Deleting a snapshot is a permanent operation. It is not possible to recover a deleted snapshot once deleted.

$ az hcs snapshot delete \
    --resource-group <resource-group> \
    --name <managed-app-name> \
    --snapshot <snapshot-id>

"Snapshot deleted: snapshot-20210203T091717Z"

Scheduled or automatic snapshots cannot be deleted. If you try to delete one of them you'll receive the following error message:

HashiCorp Operator Internal Snapshots cannot be deleted

Next Steps

In this tutorial, you learned about the different types of snapshots available on the HashiCorp Consul Service (HCS) on Azure platform.

You learned how to interact with the snapshot option using the az hcs command.

Specifically, you learned how to:

Create a new snapshot
List available snapshots
Rename an existing snapshot
Restore an existing snapshot
Delete an existing snapshot

To learn more about the HashiCorp Consul Service (HCS) on Azure, checkout the rest of the tutorials in our HCS on Azure collection, or experiment by deploying one of your own services.

To learn more about HashiCorp's fully managed cloud platform, check out our HashiCorp Cloud Platform collection.

If you need guidance on how to deploy your services to Consul service mesh, refer to our tutorials for getting started with service mesh on Kubernetes or VMs.

To learn even more about operating, observing, and monitoring your Consul service mesh, check out the following tutorials and collections.

Browse Tutorials