Set up audit logs for HCP Vault Radar

7min
|
HCP
Vault

In this tutorial, you will follow the HashiCups operations and SRE teams as they configure audit log streaming. Audit log streaming allows HashiCups to monitor service access during the proof-of-concept (POC) implementation of HCP Vault Radar.

Scenario introduction

HashiCups finished setting up their ticketing system to create new tickets when HCP Vault Radar discovers secrets in one of their GitHub repositories.

Oliver (operations) and Steve (SRE) now need the ability to monitor access to the HCP Vault Radar service by streaming audit logs to their log aggregation service. This will help the team track access and changes to the HCP Vault Radar service.

Vault Radar leverages the HashiCorp Cloud Platform (HCP) audit logging capabilities to provide audit logs for actions taken by users and service principals in your HCP organization and by the Vault Radar service itself.

Prerequisites

Completed the Scan a repository for secrets with HCP Vault Radar tutorial.
Access to the HCP Portal with a user assigned the admin role.
Access to a Datadog account where you can create an API token and store logs (free trial will support this tutorial).

Note

Configuring audit log streaming may result in additional charges in your Datadog account. If you do not intend to use this for production, delete the streaming configuration after completing this tutorial.

Configure Datadog settings

(Persona: Operations)

Oliver will make use of Datadog during the POC. The Datadog integration requires an API key. HCP audit log streaming also supports AWS Cloudwatch and Splunk.

Log in to your Datadog account.
Hover over your username in the bottom left corner and click API Keys.
Click New Key.
Enter hcp-poc-org in the Name field and click Create Key.
Copy the API key value.
Save the API key, you will use this value when configuring audit log streaming in HCP.

Set up HCP audit log streaming

(Persona: Operations)

Now that Oliver has an API key, they can configure audit log streaming in HCP.

Open a new tab and log in to your HCP organization.
Navigate to the organization page where you enabled Vault Radar.
Click Audit log streaming.
Click Create streaming destination.
Select Datadog and enter the following values:
- Destination name: hcp-poc-org
- API key: API key you created in the previous section.
- Datadog site region: The region for your Datadog account (for example, US1).
Click Test connection, you should see the message Test event sent.
Click Save.
Wait until the Status column shows Streaming.

View audit logs in Datadog

(Persona: SRE)

Steve will now verify that audit logs are streaming to Datadog.

Switch back to your Datadog account.
Click Logs in the left navigation menu and then click Explorer.
You should see two log entries from HCP: The test message, and another log entry for enabling audit log streaming.

Click the message with the latest time stamp.

The audit log shows details about the action taken in HCP, including the user who performed the action, the HCP organization ID, and other metadata.

{
  "stream": {
    "organization_id": "2570acc0-8d87-4443-a8a8-fcf8cacf750d",
    "topic": "hashicorp.platform.audit"
  },
  "resource": {
    "description": "hcp-org",
    "location": {
      "organization_id": "2570acc0-8d87-4443-a8a8-fcf8cacf750d"
    },
    "id": "8b43c543-bf71-43bc-b4e6-a8dc181ea45e",
    "type": "hashicorp.streaming.destination"
  },
  "service": "HCP",
  "control_plane_event": {
    "authentication_info": {
      "principal": {
        "id": "d4b1306d-c16b-47ed-a42b-afd5e8fd5bc8",
        "type": "PRINCIPAL_TYPE_USER",
        "user": {
          "full_name": "Oliver Preston",
          "subject": "d4b1306d-c16b-47ed-a42b-afd5e8fd5bc8",
          "id": "d4b1306d-c16b-47ed-a42b-afd5e8fd5bc8",
          "identity_type": "SOCIAL_GITHUB",
          "email": "oliver.preston@hashicorp.com"
        }
      }
    },
    "status_code": "OK",
    "request_info": {
      "http_path": "/logs/2021-03-30/organizations/2570acc0-8d87-4443-a8a8-fcf8cacf750d/destinations",
      "http_client_ip": "4.3.2.1",
      "http_verb": "POST",
      "http_user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/143.0.0.0 Safari/537.36"
    },
    "description": "Created streaming destination",
    "action": "CREATE",
    "authorization_info": [
      {
        "permissions": [
          "log.streaming.create"
        ],
        "organization_id": "2570acc0-8d87-4443-a8a8-fcf8cacf750d"
      }
    ]
  },
  "id": "019bb797-00d3-7038-80c5-7ecdd90a8caf",
  "timestamp": "2026-01-13T13:41:25.331354486Z"
}

Each audit log entry contains information related the action taken in HCP, or Vault Radar. Some key fields to note in this example:

Line 14: The service that generated the audit log entry. In this example, it is HCP.
Line 19: The type of user that performed the action. In this example, it is a PRINCIPAL_TYPE_USER. This user type indicates a human user, compared to a service prinicpal user, typically used by automation tools.
Line 21-23: Details about the user that performed the action, including their name and unique HCP user ID.
Line 36-37: The description of the action take, and HTTP verb and used to perform the action.
Line 43: The organization ID where you performed the action.

Trigger a Vault Radar audit event

(Persona: Operations)

Oliver will now test that Vault Radar audit logs are also streaming to Datadog by editing the global ignore rules in Vault Radar.

Switch back to the HCP Portal.
Navigate to the project page where you enabled Vault Radar.
Click Vault Radar.
Click Settings.
Click Global Ignore Rules.
Add the comment ## Ignore docs directory above -paths: in the text box.
Click Save.
Return to the Datadog Logs Explorer.

Click the message with the latest time stamp.

The audit log example is truncated for brevity.

...snipped...
    "metadata": {
      "service_name": "Vault Radar",
      "correlation_id": "2865712c-5272-4cdd-a902-fa14b4f8ee5d",
      "action_success": true
    },
    "status_code": "OK",
    "request_info": {
      "http_path": "/2023-05-01/vault-radar/projects/70d220a8-c83e-47bf-99a9-7a2ce0f98156/api/settings/blubracket-ignores",
      "http_client_ip": "1.2.3.4",
      "http_verb": "POST",
      "http_user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/143.0.0.0 Safari/537.36"
    },
    "description": "Radar - Update Global Ignore Rules",
    "action": "UPDATE",
    "authorization_info": [
      {
        "project_id": "70d220a8-c83e-47bf-99a9-7a2ce0f98156",
        "permissions": [
          "vault-radar.admin-settings.create"
        ],
        "organization_id": "2570acc0-8d87-4443-a8a8-fcf8cacf750d",
        "resource_id": "70d220a8-c83e-47bf-99a9-7a2ce0f98156"
      }
    ]
  }

In this example, you can see on:

Line 3: The service_name is Vault Radar, indicating this audit log entry is from the Vault Radar service compared to HCP for general HCP audit logs.
Line 14: The description field describes the action taken in Vault Radar when you edited the ignore rule.
Line 18: The project_id field shows the project where the action occurred.

You can now view, monitor, and alert on HCP platform and Vault Radar service related events.

Summary

In this tutorial, you learned how to ship audit logs from the HashiCorp Cloud Platform to the Datadog log aggregation service. You can also ship audit logs to AWS Cloudwatch and Splunk Cloud. HCP Vault Radar integrates directly with the HCP audit logging capabilities so engineering teams can track access and changes to the HCP platform and Vault Radar.

Clean up

If you do not intend to continue using audit log streaming, delete the audit log streaming destination to avoid incurring unexpected charges to your Datadog account.

Navigate to the organization page.
Click Audit log streaming.
Click the ellipsis (...) under Actions and click Delete streaming destination.
Type DELETE in the text box and click Delete.

Next steps

You have completed the proof-of-concept implementation of HCP Vault Radar at HashiCups. You can now explore production configuration in the HCP Vault Radar Operations collection to learn how to install the Vault Radar agent, correlate Vault Radar findings with secrets stored in HashiCorp Vault, and how to copy secrets to Vault to help you manage the remediation of leaked secrets.

Additional resources

Set up ticketing

Explore tutorial library