Scan GitHub for secrets

Connect GitHub as a data source to HCP Vault Radar to scan your GitHub repositories for sensitive data and secrets. Vault Radar supports scanning GitHub Cloud and GitHub Enterprise Server. You can connect to GitHub either as a data source host to auto discover GitHub organizations or as a data source tied to a single GitHub organization.

If you are new to HCP Vault Radar, checkout the HCP Vault Radar quickstart tutorial series.

Note

Vault Radar has a limit of 5000 repositories per connection. If you have more than 5000 repositories, Vault Radar will select the 5000 repositories with the most recent activity.

Security and operations considerations

You can add GitHub as a data source in HCP Vault Radar using either a GitHub App or a personal access token (PAT).

The GitHub App is the recommended authentication method because it uses short-lived installation access tokens with scoped permissions, providing more granular access control. Installation access tokens are well-suited for use cases where long-lived user credentials are undesirable.

PAT authentication is easier to set up, but it uses long-lived tokens tied to a specific GitHub user, which can pose security risks if the token is compromised. PAT authentication also provides broader permissions, and has access to any repositories the user has access to.

Data source vs data source host

You can use GitHub App authentication with HCP Vault Radar data source. A data source supports a single GitHub organization per connection. If you have multiple GitHub organizations, you can add multiple data sources to Vault Radar, one for each organization.

You can use PAT authentication as either a data source, or a data source host. A data source host supports multiple GitHub organizations in a single connection.

This is an important security and operations trade-off to consider when deciding which authentication method to use.

Requirement	GitHub App authentication	Personal access token (PAT) authentication
Single connection for multiple organizations	❌	✅
Connection per organization	✅	✅
Short-lived tokens	✅	❌
Limited repository access	✅	❌
Organization-level governance	✅	❌
Independent identity	✅	❌
Tied to a specific GitHub user	❌	✅

Prerequisites

HCP IAM user with the HCP owner, or admin role
Vault Radar agent deployed and running (agent scan only)
Python 3.10.20 or later installed (GitHub App authentication only)
For GitHub App authentication with GitHub Cloud or GitHub Enterprise Server, create a GitHub App.
For GitHub personal access token (PAT) authentication with GitHub Cloud or GitHub Enterprise Server, create a PAT with the following permission scopes:
```
repo > all
admin:org > read:org
admin:repo_hook > all
admin:org_hook > all
user > read:user
```

Add a GitHub data source with app authentication

You can add and scan GitHub Cloud or GitHub Enterprise Server using the HCP Vault Radar cloud scan or HCP Vault Radar agent scan.

Using app authentication with the HCP Vault Radar cloud scan does not require Python 3.10.20 or later to complete the setup.

Set up HCP Vault Radar

Log into the HCP Portal with an HCP IAM user that has the HCP owner or admin role.
Click Vault Radar.
Click Settings.
Click Data sources.
Click Add data source.
Select HCP Vault Radar Scan.
Select GitHub Cloud.
Under Select authentication method, click the dropdown and select GitHub OAuth to authenticate with the GitHub App.
Click Connect to GitHub.
On redirect to GitHub, select which GitHub Org to Onboard and the repositories to install on.
On successful redirect to Vault Radar, click Next.
Select either All active repositories or Select repositories to monitor.
Click Finish.

Set up GitHub app authentication

You must create and register a GitHub App before setting up HCP Vault Radar.

Download githubchecks-app.gen.py.

import logging
import sys
from secrets import token_urlsafe
from urllib.parse import urlencode, urlparse

def init_logger():
    logger = logging.getLogger(__name__)
    logger.setLevel(logging.INFO)
    h = logging.StreamHandler(sys.stdout)
    logger.addHandler(h)

    return logger

logger = init_logger()

if __name__ == '__main__':
    print("Please select an option GitHub Cloud or GitHub Enterprise Server:")
    print("1. GitHub Cloud")
    print("2. GitHub Enterprise Server")
    choice = input("Enter 1 or 2: ")

    name = 'HashiCorp Vault Radar Checks App'
    domain = 'github.com'
    webhook_url =  'https://api.hashicorp.cloud/2023-05-01/vault-radar/api/github-apps/events'
    if choice == '1':
        name = input('Enter a unique GitHub app name: ')
    elif choice == '2':
        domain = input('Enter your GitHub Enterprise Server domain name (i.e. github.companyname.com): ')
        domain = urlparse(domain if '//' in domain else f'//{domain}').netloc
        webhook_url = webhook_url + f"?domain={domain}"
    else:
        print("Invalid option")
        sys.exit()

    query_params = {
        'name': name,
        'webhook_secret': token_urlsafe(nbytes=32),
        'webhook_url': webhook_url,
        'public': True,
        'webhook_active': True,
        'url': 'https://www.hashicorp.com/',
        'checks': 'write',
        'events[]': ['check_run', 'check_suite', 'pull_request'],
        'pull_requests': 'read',
    }
    query_param_string = urlencode(query_params, doseq=True, safe=':/?=[]')
    
    url = f'https://{domain}/settings/apps/new?{query_param_string}'
    logger.info(url)

You can view the source code for this script in the HashiCorp guides GitHub repository.

Run the script and follow the prompt.
```
$ python3 github-checks-app-gen.py
```
When prompted, select GitHub Cloud by entering 1.
Provide a name for your Vault Radar GitHub App.
Copy/paste the URL into a new browser window/tab.

Copy the webhook_secret=webhook_secret in the URL.

Example output:

Enter your GitHub Enterprise Server domain name (i.e. github.companyname.com): acme.github.com
https://acme.github.com...webhook_secret=Z1KPOYifctpzOjfphKj_hqRlZbrDOBG9AU7hgj7iPrk...

Add the webhook_secret into the form.
Remove ?domain=github.com from the Webhook URL.
Scroll down and click Create GitHub App.
Take note of the values from your app settings page:
- Webhook secret from the URL above
- App name
- App ID
- Private key
Save these values in a secure location, such as IBM Vault.
Configure an environment variable on the agent with the value of the private key.
Example configuration:
If you run the agent on Kubernetes, add the environment variable to the agent deployment manifest.
```
env:
  - name: GH_RADAR_PRIVATE_KEY
    value: `string-of-your-private-key`
```
Tip
Use the same variable name on all agents.
You are ready to add GitHub as a data source in HCP Vault Radar.

Set up HCP Vault Radar

Log into the HCP Portal with an HCP IAM user that has the HCP owner or admin role.
Click Vault Radar.
Click Settings.
Click Data sources.
Click Add data source.
Select HCP Vault Radar Agent Scan.
Select GitHub Cloud.
Enter the path to the environment variable with the personal access token in the Enter environment path to your GitHub personal access token field.
Example:
```
env://GH_RADAR_PRIVATE_KEY
```
Under Select authentication method, click the dropdown and select GitHub OAuth to authenticate with the GitHub App.
Click Connect to GitHub.
On redirect to GitHub, select which GitHub Org to onboard and the repositories to install on.
On successful redirect to Vault Radar, click Next.
Select either All active repositories or Select repositories to monitor.
Click Finish.

Set up GitHub app authentication

You must create and register a GitHub App before setting up HCP Vault Radar.

Download githubchecks-app.gen.py.

import logging
import sys
from secrets import token_urlsafe
from urllib.parse import urlencode, urlparse

def init_logger():
    logger = logging.getLogger(__name__)
    logger.setLevel(logging.INFO)
    h = logging.StreamHandler(sys.stdout)
    logger.addHandler(h)

    return logger

logger = init_logger()

if __name__ == '__main__':
    print("Please select an option GitHub Cloud or GitHub Enterprise Server:")
    print("1. GitHub Cloud")
    print("2. GitHub Enterprise Server")
    choice = input("Enter 1 or 2: ")

    name = 'HashiCorp Vault Radar Checks App'
    domain = 'github.com'
    webhook_url =  'https://api.hashicorp.cloud/2023-05-01/vault-radar/api/github-apps/events'
    if choice == '1':
        name = input('Enter a unique GitHub app name: ')
    elif choice == '2':
        domain = input('Enter your GitHub Enterprise Server domain name (i.e. github.companyname.com): ')
        domain = urlparse(domain if '//' in domain else f'//{domain}').netloc
        webhook_url = webhook_url + f"?domain={domain}"
    else:
        print("Invalid option")
        sys.exit()

    query_params = {
        'name': name,
        'webhook_secret': token_urlsafe(nbytes=32),
        'webhook_url': webhook_url,
        'public': True,
        'webhook_active': True,
        'url': 'https://www.hashicorp.com/',
        'checks': 'write',
        'events[]': ['check_run', 'check_suite', 'pull_request'],
        'pull_requests': 'read',
    }
    query_param_string = urlencode(query_params, doseq=True, safe=':/?=[]')
    
    url = f'https://{domain}/settings/apps/new?{query_param_string}'
    logger.info(url)

You can view the source code for this script in the HashiCorp guides GitHub repository.

Run the script and follow the prompt.
```
$ python3 github-checks-app-gen.py
```
When prompted, select GitHub Enterprise Server by entering 2.
Enter your GitHub Enterprise Server domain name.
Example:
```
github.example.com
```
Copy/paste the URL into a new browser window/tab.

Copy the webhook_secret=webhook_secret in the URL.

Example output:

Enter your GitHub Enterprise Server domain name (i.e. github.example.com): github.example.com
https://github.example.com...webhook_secret=Z1KPOYifctpzOjfphKj_hqRlZbrDOBG9AU7hgj7iPrk...

Add the webhook_secret into the form.
Scroll down and click Create GitHub App.
Take note of the values from your app settings page:
- Webhook secret from the URL above
- App name
- App ID
- Client secret
- Private key
Save these values in a secure location, such as IBM Vault.
You are ready to add GitHub as a data source in HCP Vault Radar.

Set up HCP Vault Radar

Log into the HCP Portal with an HCP IAM user that has the HCP owner or admin role.
Click Vault Radar.
Click Settings.
Click Data sources.
Click Add data source.
Select HCP Vault Radar Scan.
Select GitHub Enterprise Server.
Under Select authentication method, click the dropdown and select GitHub OAuth to authenticate with the GitHub App.
Select which GitHub App to authenticate with from the GitHub app dropdown.
Click Connect to GitHub.
On redirect to GitHub, select which GitHub Org to onboard and the repositories to install on.
On successful redirect to Vault Radar, click Next.
Select either All active repositories or Select repositories to monitor.
Click Finish.

Set up GitHub app authentication

You must create and register a GitHub App before setting up HCP Vault Radar.

Download githubchecks-app.gen.py.

import logging
import sys
from secrets import token_urlsafe
from urllib.parse import urlencode, urlparse

def init_logger():
    logger = logging.getLogger(__name__)
    logger.setLevel(logging.INFO)
    h = logging.StreamHandler(sys.stdout)
    logger.addHandler(h)

    return logger

logger = init_logger()

if __name__ == '__main__':
    print("Please select an option GitHub Cloud or GitHub Enterprise Server:")
    print("1. GitHub Cloud")
    print("2. GitHub Enterprise Server")
    choice = input("Enter 1 or 2: ")

    name = 'HashiCorp Vault Radar Checks App'
    domain = 'github.com'
    webhook_url =  'https://api.hashicorp.cloud/2023-05-01/vault-radar/api/github-apps/events'
    if choice == '1':
        name = input('Enter a unique GitHub app name: ')
    elif choice == '2':
        domain = input('Enter your GitHub Enterprise Server domain name (i.e. github.companyname.com): ')
        domain = urlparse(domain if '//' in domain else f'//{domain}').netloc
        webhook_url = webhook_url + f"?domain={domain}"
    else:
        print("Invalid option")
        sys.exit()

    query_params = {
        'name': name,
        'webhook_secret': token_urlsafe(nbytes=32),
        'webhook_url': webhook_url,
        'public': True,
        'webhook_active': True,
        'url': 'https://www.hashicorp.com/',
        'checks': 'write',
        'events[]': ['check_run', 'check_suite', 'pull_request'],
        'pull_requests': 'read',
    }
    query_param_string = urlencode(query_params, doseq=True, safe=':/?=[]')
    
    url = f'https://{domain}/settings/apps/new?{query_param_string}'
    logger.info(url)

You can view the source code for this script in the HashiCorp guides GitHub repository.

Run the script and follow the prompt.
```
$ python3 github-checks-app-gen.py
```
When prompted, select GitHub Enterprise Server by entering 2.
Enter your GitHub Enterprise Server domain name.
Example:
```
github.example.com
```
Copy/paste the URL into a new browser window/tab.

Copy the webhook_secret=webhook_secret in the URL.

Example output:

Enter your GitHub Enterprise Server domain name (i.e. github.companyname.com): acme.github.com
https://acme.github.com...webhook_secret=Z1KPOYifctpzOjfphKj_hqRlZbrDOBG9AU7hgj7iPrk...

Add the webhook_secret into the form.
Scroll down and click Create GitHub App.
Take note of the values from your app settings page:
- Webhook secret from the URL above
- App name
- App ID
- Private key
Save these values in a secure location, such as IBM Vault.
Configure an environment variable on the agent with the value of the private key.
Example configuration:
If you run the agent on Kubernetes, add the environment variable to the agent deployment manifest.
```
env:
  - name: GH_RADAR_PRIVATE_KEY
    value: `string-of-your-private-key`
```
Tip
Use the same variable name on all agents.
You are ready to add GitHub as a data source in HCP Vault Radar.

Set up HCP Vault Radar

Log into the HCP Portal with an HCP IAM user that has the HCP owner or admin role.
Click Vault Radar.
Click Settings.
Click Data sources.
Click Add data source.
Select Vault Radar Agent Scan.
Select GitHub Enterprise Server.
Enter the path to the environment variable with the personal access token in the Enter environment path to your GitHub personal access token field.
Example:
```
env://GH_RADAR_PRIVATE_KEY
```
Under Select authentication method, click the dropdown and select GitHub OAuth to authenticate with the GitHub App.
Select which GitHub App to authenticate with from the GitHub app dropdown.
Click Connect to GitHub.
On redirect to GitHub, select which GitHub Org to onboard and the repositories to install on.
On successful redirect to Vault Radar, click Next.
Select either All active repositories or Select repositories to monitor.
Click Finish.

Add a GitHub data source host with PAT authentication

You can add and scan GitHub Cloud or GitHub Enterprise Server using the HCP Vault Radar cloud scan or HCP Vault Radar agent scan.

Add GitHub Cloud as a data source to scan your GitHub organization's repositories for secrets.

Log into the HCP Portal with an HCP IAM user that has the HCP owner or admin role.
Click Vault Radar.
Click Settings.
Click Data Source Hosts.
Click Add data source host.
Select HCP Vault Radar Scan.
Select GitHub Cloud.
Enter the personal access token in the Enter your GitHub personal access token field.
Click Next.
Select either All data sources or Select data sources.
Click Finish.

Log into the HCP Portal with an HCP IAM user that has the HCP owner or admin role.
Click Vault Radar.
Click Settings.
Click Data Source Hosts.
Click Add data source host.
Select Vault Radar Agent Scan.
Select GitHub Cloud.
Enter the path to the environment variable with the personal access token in the Enter environment path to your GitHub personal access token field.
Example:
```
env://GITHUB_TOKEN
```
Click Next.
Select either All data sources or Select data sources.
Click Finish.

Add GitHub Enterprise Server as a data source to scan your GitHub organization's repositories for secrets.

Log into the HCP Portal with an HCP IAM user that has the HCP owner or admin role.
Click Vault Radar.
Click Settings.
Click Data Source Hosts.
Click Add data source host.
Select HCP Vault Radar Scan.
Select GitHub Enterprise Server.
Enter the following information:
- Enter your GitHub domain name: Fully qualified domain name of your GitHub Enterprise Server.
- Enter your GitHub personal access token: Access token from your GitHub Enterprise Server.
Click Next.
Select either All data sources or Select data sources.
Click Finish.

Log into the HCP Portal with an HCP IAM user that has the HCP owner or admin role.
Click Vault Radar.
Click Settings.
Click Data Source Hosts.
Click Add data source host.
Select Vault Radar Agent Scan.
Select GitHub Enterprise Server.
Enter the following information:
- Enter your GitHub domain name: Fully qualified domain name of your GitHub Enterprise Server.
- Enter environment path to your GitHub personal access token: Access token from your GitHub Enterprise Server.
Example:
```
env://GITHUB_TOKEN
```
Click Next.
Select either All data sources or Select data sources.
Click Finish.

Assign a group to a resource

Once you add a data source, an HCP user with the admin role must assign a group to each of the monitored resources within each data source. You can assign each resource to only one group.

Note

Users with the HCP IAM admin role do not need to be a member of other groups. Accounts with the admin role have full access to Vault Radar.

If you do not already have a group, refer to the Identity and Access Management groups documentation to create a group.

Navigate to the Project dashboard.
Click Access control (IAM).
Click Add new assignment.
Search the name of the group in the Search for an assignee search field.
Click the group name in the search results.
Click the Select service pulldown menu and select Vault Radar.
Click the Select role pulldown menu and select the Vault Radar Developer role.
Click Save.
Click Back to Dashboard.
Click Vault Radar.
Click Resources.
Select the resource you want to assign to a group and click Assign groups.
Click the Assign resoruce to group pulldown menu.
Select the group that requires access to the resource.
Select either the Viewer or Contributor role.
Click OK.

Update data source host

Navigate to Settings.
Click Data Source Hosts
Click the vertical ellipsis to the right of the data source host.
To update the monitored data sources, click Edit data sources.
To update the token, click Edit data source host details.
To delete the data source host, click Delete data source host.

Update data source

Navigate to Settings.
Click Data sources.
Click the vertical ellipsis to the right of the data source.
To update the monitored data sources, click Edit data sources.
To update the authentication method, select the new authentication method from the Select authentication method dropdown.
To update the token, click Edit data source host details.
To delete the data source, click Delete data source.

Tutorials

Learn how to evaluate and implement HCP Vault Radar in your environment with our tutorials.

HCP Vault Radar quickstart - Follow HashiCups, a fictitious coffee company, as they onboard HCP Vault Radar and scan their data sources for secrets.
HCP Vault Radar operations - Learn how HashiCups operations team uses HCP Vault Radar advanced features to scan data sources with the Vault Radar agent, and correlates findings with HCP Vault to manage secrets.
HCP Vault Radar developer - Learn how HashiCups developers use HCP Vault Radar advanced features to understand secret exposure, identify potential risks, and prevent leaked secrets during the software development lifecycle.