debug command starts a process that monitors a Vault server, probing
information about it for a certain duration.
Gathering information about the state of the Vault cluster often requires the
operator to access all necessary information via various API calls and terminal
debug command aims to provide a simple workflow that produces a
consistent output to help operators retrieve and share information about the
server in question.
debug command honors the same variables that the base command
accepts, such as the token stored via a previous login or the environment
VAULT_ADDR. The token used determines the
permissions and, in turn, the information that
debug may be able to collect.
The address specified determines the target server that will be probed against.
If the command is interrupted, the information collected up until that point gets persisted to an output directory.
Regardless of whether a particular target is provided, the ability for
to fetch data for the target depends on the token provided. Some targets, such
server-status, queries unauthenticated endpoints which means that it can be
queried all the time. Other targets require the token to have ACL permissions to
query the matching endpoint in order to get a proper response. Any errors
encountered during capture due to permissions or otherwise will be logged in the
-target flag can be specified multiple times to capture specific
information when debug is running. By default, it captures all information.
|Sanitized version of the configuration state.|
|Information about the instance running the server, such as CPU, memory, and disk.|
|Runtime profiling data, including heap, CPU, goroutine, and trace profiling.|
|Health and seal status.|
Note that the
pprof targets are only queried
on active and performance standby nodes due to the the fact that the information
pertains to the local node and the request should not be forwarded.
Additionally, host information is not available on the OpenBSD platform due to
library limitations in fetching the data without enabling
[Enterprise] Telemetry can be gathered from a DR Secondary active node via the
metrics target if unauthenticated_metrics_access is enabled.
The output of the bundled information, once decompressed, is contained within a single directory. Each target, with the exception of profiling data, is captured in a single file. For each of those targets collection is represented as a JSON array object, with each entry captured at each interval as a JSON object.
$ tree vault-debug-2019-10-15T21-44-49Z/ vault-debug-2019-10-15T21-44-49Z/ ├── 2019-10-15T21-44-49Z │ ├── goroutine.prof │ ├── heap.prof │ ├── profile.prof │ └── trace.out ├── 2019-10-15T21-45-19Z │ ├── goroutine.prof │ ├── heap.prof │ ├── profile.prof │ └── trace.out ├── 2019-10-15T21-45-49Z │ ├── goroutine.prof │ ├── heap.prof │ ├── profile.prof │ └── trace.out ├── 2019-10-15T21-46-19Z │ ├── goroutine.prof │ ├── heap.prof │ ├── profile.prof │ └── trace.out ├── 2019-10-15T21-46-49Z │ ├── goroutine.prof │ └── heap.prof ├── config.json ├── host_info.json ├── index.json ├── metrics.json ├── replication_status.json └── server_status.json
Start debug using reasonable defaults:
$ vault debug
Start debug with different duration, intervals, and metrics interval values, and skip compression:
$ vault debug -duration=1m -interval=10s -metrics-interval=5s -compress=false
Start debug with specific targets:
$ vault debug -target=host -target=metrics
The following flags are available in addition to the standard set of flags included on all commands.
(bool: true)- Toggles whether to compress output package The default is true.
(int or time string: "2m")- Duration to run the command. The default is 2m0s.
(int or time string: "30s")- The polling interval at which to collect profiling data and server state. The default is 30s.
(string: "standard")- Log format to be captured if "log" target specified. Supported values are "standard" and "json". The default is "standard".
(int or time string: "10s")- The polling interval at which to collect metrics data. The default is 10s.
(string)- Specifies the output path for the debug package. Defaults to an time-based generated file name.
(string: all targets)- Target to capture, defaulting to all if none specified. This can be specified multiple times to capture multiple targets. Available targets are: config, host, metrics, pprof, replication-status, server-status.