Scan AWS S3 for secrets

Beta feature

This feature is currently available as a beta release. Beta functionality is stable but possibly incomplete and subject to change. We strongly discourage using beta features in production.

HCP Vault Radar allows users to scan AWS S3 buckets for application data, configuration files, backups, and other sensitive information.

Vault Radar supports versioned buckets, allowing you to scan the previous versions of objects.

If you are new to HCP Vault Radar, checkout the HCP Vault Radar quickstart series.

Prerequisites

You need one of the following AWS authentication methods:

HCP Vault Radar agent version 0.38.0 or later
IAM role authentication with an EC2 instance or configured IAM role
Environment variables authentication with AWS Access Key ID and Secret Access Key

Warning

HCP Vault Radar does not support short lived AWS credentials through environment variables (AWS_SESSION_TOKEN).

Both authentication methods support an optional assume role ARN for cross-account access or elevated permissions. For more information about assuming roles, refer to the AWS STS AssumeRole documentation.

Required permissions

The IAM user, role, or assumed role must have the following permissions:

Service	Permission	Documentation
STS	`sts:GetCallerIdentity`	GetCallerIdentity API
S3	`s3:ListAllMyBuckets`	ListBuckets API
S3	`s3:ListBucket`	ListObjectsV2 API
S3	`s3:GetObject`	GetObject API
Account (optional)	`account:GetAccountInformation`	GetAccountInformation API
S3	`s3:GetObjectVersion`	GetObject API
S3	`s3:ListBucketVersions`	ListObjectVersions API
S3 (optional)	`s3:GetBucketPolicyStatus`	GetBucketPolicyStatus API
S3 (optional)	`s3:GetBucketPublicAccessBlock`	GetPublicAccessBlock API
S3 (optional)	`s3:GetBucketAcl`	GetBucketAcl API
S3 (optional)	`s3:GetObjectAcl`	GetObjectAcl API

Optional permission limitations

account:GetAccountInformation: Scans without this permission will not include the AWS account name on the UI for data source metadata.
s3:GetBucketPolicyStatus, s3:GetBucketAcl, s3:GetBucketPublicAccessBlock and s3:GetObjectAcl: Scans without these optional permissions do not include visibility values of the bucket or the detected secrets.

Example IAM policies:

AWS lets you define IAM policies at a high level as well as very granular levels.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "sts:GetCallerIdentity"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets",
                "s3:ListBucket",
                "s3:ListBucketVersions",
                "s3:GetBucketPolicyStatus",
                "s3:GetBucketAcl",
                "s3:GetBucketPublicAccessBlock"
            ],
            "Resource": "arn:aws:s3:::*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:GetObjectVersion",
                "s3:GetObjectAcl"
            ],
            "Resource": "arn:aws:s3:::*/*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "account:GetAccountInformation"
            ],
            "Resource": "*"
        }
    ]
}

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "sts:GetCallerIdentity",
                "account:GetAccountInformation"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "s3:ListAllMyBuckets",
            "Resource": [
                "arn:aws:s3:::*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:ListBucketVersions",
                "s3:GetBucketPolicyStatus",
                "s3:GetBucketAcl",
                "s3:GetBucketPublicAccessBlock"
            ],
            "Resource": [
                "arn:aws:s3:::my-app-logs",
                "arn:aws:s3:::my-app-data",
                "arn:aws:s3:::my-app-backups"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:GetObjectVersion",
                "s3:GetObjectAcl"
            ],
            "Resource": [
                "arn:aws:s3:::my-app-logs/*",
                "arn:aws:s3:::my-app-data/*",
                "arn:aws:s3:::my-app-backups/*"
            ]
        }
    ]
}

Add an AWS S3 data source

Set up and manage AWS S3 agent scan from the Vault Radar module in the HCP Portal.

Click Settings.
Click Data Sources.
Select Vault Radar Agent Scan.
Select AWS S3 in the Object Storage section.
Select an AWS authentication method from the Authentication method pulldown menu.
Enter the details for the selected method.
- Select IAM Role if you want to use instance profile or role-based authentication.
- (Optional) Enter an assume role ARN in the Assume Role ARN text field if you need to assume a different role for access.
- Select AWS Credentials from environment variables if you want to use access keys.
- Enter your AWS Access Key ID location in the AWS Access Key ID Env variable text field (default: env://AWS_ACCESS_KEY_ID).
- Enter your AWS Secret Access Key location in the AWS Secret Access Key Env variable text field (default: env://AWS_SECRET_ACCESS_KEY).
- (Optional) Enter an assume role ARN in the Assume Role ARN text field if you need to assume a different role for access.
Click Next to validate and establish the connection.
Choose the buckets to monitor. You can either choose all buckets, or individual buckets.
Note
Vault Radar skips buckets that do not have the `s3:GetObject` policy granted, even if you've chosen it from the list here.
Click Finish.
Vault Radar scans all the objects in the selected buckets and reports the results.

Assign a group to a resource

Once you add a data source, an HCP user with the admin role must assign a group to each of the monitored resources within each data source. You can assign each resource to only one group.

Note

Users with the HCP IAM admin role do not need to be a member of other groups. Accounts with the admin role have full access to Vault Radar.

If you do not already have a group, refer to the Identity and Access Management groups documentation to create a group.

Navigate to the Project dashboard.
Click Access control (IAM).
Click Add new assignment.
Search the name of the group in the Search for an assignee search field.
Click the group name in the search results.
Click the Select service pulldown menu and select Vault Radar.
Click the Select role pulldown menu and select the Vault Radar Developer role.
Click Save.
Click Back to Dashboard.
Click Vault Radar.
Click Resources.
Select the resource you want to assign to a group and click Assign groups.
Click the Assign resoruce to group pulldown menu.
Select the group that requires access to the resource.
Select either the Viewer or Contributor role.
Click OK.

Update data source host

Navigate to Settings, and then Data Source Hosts, and click the three dots to the right.
To update the monitored data sources, click Edit data sources
To update the token, click Edit data source host details

Tutorials

If you are new to HCP Vault Radar, checkout the HCP Vault Radar quickstart series.