Metrics from HashiCorp-managed clusters
This page describes how to scrape Consul server metrics from HashiCorp-managed clusters.
HashiCorp ensures the availability of HashiCorp-managed clusters and provides observability features through HCP Consul Central. You can set up a custom metrics collector to track cluster usage.
For opinionated, preconfigured graphs of Consul server and Envoy proxy metrics, refer to the HCP Consul observability overview.
To access server metrics, you need a telemetry agent capable of both resolving a DNS record to IP addresses and scraping Prometheus-format metrics. This page provides example configurations for Prometheus and OpenTelemetry Collector using Docker.
Accessing server metrics requires the cluster's address and an ACL token with a minimum of
agent:read permission. To get the cluster's address and create an ACL token using HCP Consul, refer to Access HashiCorp-managed clusters.
Configure your telemetry collector to scrape metrics from the cluster's
/v1/agent/metrics endpoint and then start the collector. The following examples demonstrate configurations for Prometheus and OpenTelemetry Collector.
To use a Prometheus metrics collector, configure it to do the following:
- Resolve the cluster's DNS address to IP addresses.
- Scrape metrics from each server's
- Call the agent endpoint with a Bearer token authorized to read metrics.
In the following example configuration, these fields are specified so that the collector scrapes agent metrics every 60 minutes by running a job named
--- global: scrape_interval: "60s" scrape_configs: - job_name: "hcp-consul-cluster" # Resolve the IP addresses of servers. scheme: "https" dns_sd_configs: - names: - "<consul-cluster-name.consul.alphanumeric-id.aws.hashicorp.cloud>" type: "A" port: 443 # Set the Consul token as the Bearer token. authorization: credentials: "<token>" # Query the agent metrics endpoint. metrics_path: "/v1/agent/metrics" # Disable TLS verification. tls_config: insecure_skip_verify: true
To start the metrics collector, run the following command:
$ prometheus --config.file=scrape.yaml
You can use the Prometheus Receiver in the OpenTelemetry Collector to scrape metrics from a HashiCorp-managed cluster.
The following example configures a metrics pipeline with a Processor and Exporter. The OTLP Exporter requires an OTLP gRPC endpoint. Observability platforms such as New Relic and Honeycomb have public endpoints for metrics ingestion. Others platforms such as Datadog have their own Exporter.
--- receivers: prometheus: config: global: scrape_interval: "60s" scrape_configs: - job_name: "hcp-consul-cluster" scheme: "https" dns_sd_configs: - names: - "<consul-cluster-name.consul.alphanumeric-id.aws.hashicorp.cloud>" type: "A" port: 443 authorization: credentials: "<token>" metrics_path: "/v1/agent/metrics" tls_config: insecure_skip_verify: true processors: batch: send_batch_max_size: 1000 send_batch_size: 100 timeout: "60s" exporters: otlp: endpoint: "<otlp-compatible-endpoint>" service: pipelines: metrics: receivers: [prometheus] processors: [batch] exporters: [otlp]
To start the Collector in a Docker container, run the following command:
$ docker run -v $(pwd)/config.yaml:/etc/otelcol-contrib/config.yaml otel/opentelemetry-collector-contrib:0.86.0