Session recording recovery

If a worker connection fails during a session recording, the recording may not complete successfully. The failure could happen due to a network connectivity issue or because a worker is scaled down, for example.

When a worker starts, it automatically analyzes any session recordings on the local disk and attempts to recover the SSH recordings that were not completed. The recovery process runs in the background to finalize the uncompleted SSH recordings. You can view any recovered session recordings normally.

If Boundary cannot recover a recording, the worker collects any verification error details and reports them. You cannot replay session recordings if they have verification errors.

Boundary does not perform ongoing validation on the files in remote storage after the recovery. The recording status only reflects that the recording was valid and available for playback when it was initially added to the storage bucket. If files are modified in the remote bucket later, the recording may appear as available, but playback could still fail with validation errors.

Session recording recovery requires both the Boundary worker and controllers to be version 1.0.0 or later. It is not supported for RDP session recordings at this time.

Note

If a worker cannot access one or more files or directories associated with a session recording, the recovery process fails. The process cannot continue until an administrator restores the required permissions on the worker's local disk.

Once the permissions are restored, you must restart the worker to begin the recovery process for the recording.

Verification errors

Boundary reports any verification error details for BSR session, connection, and channel files. The verification error details answer the following questions:

Which container had a problem?
Which file or files were affected?
What kind of verification problem was found for each file?

You may receive the following verification error types.

`MissingFile`

Boundary could not find a required file.

The MissingFile error usually means that a signature file, checksum file, meta file, summary file, or other expected BSR file is missing. Verification could not continue for that specific file because it is not available.

The MissingFile error is mutually exclusive with all other error types for the same file.

`ChecksumMismatch`

The file was present, but the computed checksum did not match the expected checksum.

The ChecksumMismatch error usually means that the file contents changed after the checksum file was created. The file could be corrupted, stale, or may not belong with the current checksum set.

`InvalidSignature`

Cryptographic signature verification failed and the signed content cannot be trusted.

The InvalidSignature error usually means the signature file exists, but Boundary could not validate the signed content with the expected public key material. The signed artifact may have been altered, or the signature or key material does not match.

`MalformedFileContent`

The file exists, but its contents could not be parsed or decoded as a valid BSR artifact.

The MalformedFileContent error usually means that the file is truncated, corrupted, or not in the expected format. Boundary was able to open the file, but could not successfully read it as the expected content type.

`Unknown`

Boundary detected an unspecified type of verification error.

The Unknown error is a fallback classification error. You should treat it as a generic verification failure and inspect the associated error message for more detail.

Multiple file errors

Boundary records errors per file, not per recording. Therefore, different files in the same recording can have different error types.

A single file can have more than one error type, except for when the error is MissingFile. The ChecksumMismatch and MalformedFileContent errors can appear together for the same file. For example, if a summary file has ChecksumMismatch and MalformedFileContent, the file failed integrity verification and could not be cleanly decoded.

Recovery checksum and signature files

By default, Boundary creates the following recording verification files to verify the original BSR files for a recording:

Boundary may also create the following checksum and signature pair for recovery purposes:

The recovery checksum and signature files are used as a trusted way to verify the artifacts that Boundary creates during the recovery process. They allow Boundary to:

Detect whether recovery artifacts are present
Detect whether recovery artifacts were modified or corrupted after they were written
Verify recovery material before treating it as trustworthy during inspection or recovery

Without the recovery verification files, Boundary would be able to see the recovery files in storage, but it would not be able to check them for integrity and authenticity. Boundary only creates the recovery verification files when a recording has one or more verification errors. If a recording completes normally, you will not see these files.

More information

For general information about recording sessions, refer to the Session recording overview.
To view recorded sessions, refer to Find and view SSH sessions or Export and view RDP sessions.
For a better understanding of the BSR format, refer to BSR file data structure.