Scan AWS S3 with Radar Agent

Beta feature

This feature is currently available as a beta release. Beta functionality is stable but possibly incomplete and subject to change. We strongly discourage using beta features in production.

HCP Vault Radar allows users to scan AWS S3 buckets for application data, configuration files, backups, and other sensitive information.

Vault Radar supports versioned buckets, allowing you to scan the previous versions of objects.

Connect AWS S3

Scanning AWS S3 buckets requires the HCP Vault Radar agent. Once you deploy the agent, you can add AWS S3 as a data source.

Prerequisites

You need one of the following AWS authentication methods:

HCP Vault Radar agent version 0.38.0 or later
IAM role authentication with an EC2 instance or configured IAM role
Environment variables authentication with AWS Access Key ID and Secret Access Key

Warning

HCP Vault Radar does not support short lived AWS credentials through environment variables (AWS_SESSION_TOKEN).

Both authentication methods support an optional assume role ARN for cross-account access or elevated permissions. For more information about assuming roles, refer to the AWS STS AssumeRole documentation.

Required permissions

The IAM user, role, or assumed role must have the following permissions:

Service	Permission	Documentation
STS	`sts:GetCallerIdentity`	GetCallerIdentity API
S3	`s3:ListAllMyBuckets`	ListBuckets API
S3	`s3:ListBucket`	ListObjectsV2 API
S3	`s3:GetObject`	GetObject API
Account (optional)	`account:GetAccountInformation`	GetAccountInformation API

The account:GetAccountInformation permission is optional. Scans without this permission will not include the AWS account name on the UI for data source metadata.

Example IAM policies:

AWS lets you define IAM policies at a high level as well as very granular levels.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "sts:GetCallerIdentity"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets",
                "s3:ListBucket"
            ],
            "Resource": "arn:aws:s3:::*"
        },
        {
            "Effect": "Allow",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::*/*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "account:GetAccountInformation"
            ],
            "Resource": "*"
        }
    ]
}

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "sts:GetCallerIdentity",
                "account:GetAccountInformation"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "s3:ListAllMyBuckets",
            "Resource": [
                "arn:aws:s3:::*",
            ]
        },
        {
            "Effect": "Allow",
            "Action": "s3:ListBucket",
            "Resource": [
                "arn:aws:s3:::my-app-logs",
                "arn:aws:s3:::my-app-data",
                "arn:aws:s3:::my-app-backups"
            ]
        },
        {
            "Effect": "Allow",
            "Action": "s3:GetObject",
            "Resource": [
                "arn:aws:s3:::my-app-logs/*",
                "arn:aws:s3:::my-app-data/*",
                "arn:aws:s3:::my-app-backups/*"
            ]
        }
    ]
}

Configure the agent for AWS S3 scanning

Set up and manage AWS S3 agent scan from the Vault Radar module in the HCP Portal.

Click Settings.
Click Data Sources.
Select Vault Radar Agent Scan.
Select AWS S3 in the Object Storage section.
Select an AWS authentication method from the Authentication method pulldown menu.
Enter the details for the selected method.
- Select IAM Role if you want to use instance profile or role-based authentication.
- (Optional) Enter an assume role ARN in the Assume Role ARN text field if you need to assume a different role for access.
- Select AWS Credentials from environment variables if you want to use access keys.
- Enter your AWS Access Key ID location in the AWS Access Key ID Env variable text field (default: env://AWS_ACCESS_KEY_ID).
- Enter your AWS Secret Access Key location in the AWS Secret Access Key Env variable text field (default: env://AWS_SECRET_ACCESS_KEY).
- (Optional) Enter an assume role ARN in the Assume Role ARN text field if you need to assume a different role for access.
Click Next to validate and establish the connection.
Choose the buckets to monitor. You can either choose all buckets, or individual buckets.
Note
Vault Radar skips buckets that do not have the `s3:GetObject` policy granted, even if you've chosen it from the list here.
Click Finish.
Vault Radar scans all the objects in the selected buckets and reports the results.