Vault
Introduction to Vault for application developers
HashiCorp Vault helps application developers improve the security of their applications by providing a set of services to create, access, and securely store information. This tutorial introduces Vault to application developers and demonstrates how to integrate Vault into the software development life cycle to enhance and simplify the security of your applications.
In the Vault foundations, you followed HashiCups, a fictional company, as they learned about the basic features in Vault to secure their applications.
In this set of tutorials, you will follow Danielle, a developer at HashiCups, as they learn how to integrate Vault into their software development life cycle to improve application security.
Scenario
Danielle is the lead engineer and manager for the development team. The development team writes, builds, and ships multiple applications and services for HashiCups. They are responsible for the build pipelines that package these applications and services, and deliver them to an artifact repository.
These pipelines require credentials to access third party services such as databases and cloud providers.
In addition to the pipelines, the applications and services themselves require credentials to access these same services.
HashiCups has set several goals for the company that the development team must meet.
- Allow teams to securely store API keys, passwords, certificates, etc.
- Restrict access to cloud environments to those people or services that require access to operate the business.
- Time box access to the cloud environments.
- Encrypt all customer data in transit and at rest.
Use cases for Vault
The HashiCups development team has identified several use cases that Vault helps to solve.
HashiCups CI/CD pipelines require access to several systems during the build process. Vault can help to securely store and manage the credentials required to access these systems. Some of these systems require long-lived credentials, such as usernames and passwords, or API keys. Others systems support creating credentials on-demand, which are short-lived, and allow HashiCups to time box access to these systems.
HashiCups requires their applications to encrypt all data both in transit and at rest. Vault allows you to store, or generate TLS certificates. When the CI/CD pipeline deploys the application, it can apply TLS certificates from Vault to ensure encryption of all traffic and data in transit.
Vault also allows you to encrypt data at rest. This ensures HashiCups meets their requirements for encrypting customer data at rest.
Danielle and the development team work with HashiCorp to identify the plugins that help them to meet their goals.
Compare secrets engine plugins
The development team needs to securely store credentials such as usernames, passwords, and API keys.
The Vault key/value secrets engine plugin allows teams to store any type of secret. The development team can use the K/V secrets engine to store credentials for third party services, such as API keys, TLS certificates, and passwords.
In addition to storing generated TLS certificates, Vault provides the PKI secrets engine. The PKI secrets engine plugin allows the development team to generate dynamic certificates using an external certificate authority (CA), or with Vault acting as the CA.
For other systems, such as database servers or cloud providers, the development team can use dynamic credentials from Vault. The database secrets engine plugin supports several relational database management system (RDBMS) such as PostgreSQL, MySQL, and Microsoft SQL Server. Other secrets engine plugins support public cloud providers such as AWS, Azure, and GCP.
When using a plugin that supports dynamic credentials, the development team ensures access to these systems is time boxed. These credentials, created on-demand, are automatically revoked after a specified time to live (TTL). By automatically revoking credentials, HashiCups also limits the possibility of exposing secrets, since they no longer function after their defined TTL.
To ensure the HashiCups applications encrypt data at rest, the development team has the option to use the Transit secrets engine plugin. The Transit secrets engine encrypts data in transit to and from their applications, allowing the application to store the data in some external storage system.
Compare authentication plugins
Before Danielle, or any of the HashiCups services can access Vault, they must authenticate with Vault.
Authentication plugins, known as auth methods, allow users and services to authenticate with Vault, and receive a token with a policy attached to access Vault.
The type of auth method you select depends on whether the entity accessing Vault is a human or a machine. For humans, there are basic auth methods like userpass, or more complex auth methods like OIDC or LDAP. For machines, you can use AppRole, or platform specific auth methods like Kubernetes, AWS, Azure, or GCP.
When Vault authorizes an authentication request, it returns a token to the entity. The token has one or more policies attached that define which capabilities the entity can perform in Vault.
The most common type of policy is an access control list, written in the HashiCorp Configuration Language (HCL). The policy defines the paths the entity can access, and the operations they can perform on those paths.
Vault also works with Sentinel for advanced policy workflows. Sentinel allows you add validation and approvals to workflows in Vault. Sentinel policies are beyond the scope of this series of tutorials.
Understand access to Vault
All users and workloads must authenticate with Vault. This diagram shows a simplified flow of the authentication process for machine based authentication, though the process is similar for human based authentication.
- The workload initiates an authentication request to Vault.
- Vault connects to the configured auth method to validate the identity of the workload.
- The trusted platform validates the identity of the workload.
- The trusted platform confirms the identity of the workload to Vault.
- Vault authorizes the authentication request and returns a token, with a policy, to the workload.
Once the authentication process is complete, the workload accesses secrets stored in Vault, permitted by the policy attached to the token.
Integrate Vault ecosystem tools
There are several ways to integrate Vault with your existing services.
Vault API
Most commonly, you can use the Vault API, or the Vault CLI to integrate services with Vault. When integrating Vault into a CI/CD pipeline, you can configure the pipeline to use the Vault API, or when used with containers, you can install the Vault CLI.
Vault Proxy
Generally, all entities connect directly to Vault. In large environments, this may introduce extra latency when there is a high volume of requests. Vault Proxy acts as an API proxy for workloads connecting to Vault, for example using the transit secrets engine to encrypt data, or pull static secrets from the KV secrets engine. Vault Proxy authenticates with Vault, and manages the authentication lifecycle. Vault Proxy runs as a client-side service, or as a sidecar for containerized workloads. Vault Proxy also caches secrets, reducing the number of requests to Vault.
Vault Agent
Vault Agent aims to simplify integrating Vault with your applications. Rather than updating applications to use the Vault API to get secrets from Vault, you can configure the Vault Agent to handle authentication and secret retrieval. Vault agent stores the secret in a format that your application can use, such as environment variables or in a file. For example, Vault Agent retrieves secrets from the KV secrets engine, credentials from the database secrets engine, or TLS certificates from the PKI secrets engine. This allows you to add Vault to your existing applications without modifying the application code.
Vault Secrets Operator
Kubernetes workloads often access native Kubernetes Secrets. These secrets are static, and adds another system to manage secrets, leading to secret sprawl. The Vault Secrets Operator is a Kubernetes operator that manages the lifecycle of Kubernetes Secrets based on a secret in Vault. The operator creates, updates, and deletes secrets in Kubernetes, and ensures the secrets are up to date with Vault.
Vault Go SDK
HashiCorp publishes a Go SDK for Vault. The Go SDK allows you to integrate Vault with your Go applications. The SDK provides a set of libraries to authenticate with Vault, and to read and write secrets.
In addition to the Go SDK, there are several community SDKs for languages such as C#, Java, Node.js, PowerShell, and Python.
Knowledge checks
A quiz to test your knowledge.
What are the main use cases for Vault in the software development life cycle?
🔘 Securely store API keys, passwords, and certificates
🔘 Restrict and time box access to cloud environments
🔘 Encrypt data in transit and at rest
🔘 All of the above
❌ Securely store API keys, passwords, and certificates
❌ Restrict and time box access to cloud environments
❌ Encrypt data in transit and at rest
✅ All of the above
Vault helps teams securely store credentials, time box access using dynamic credentials, and encrypt data both in transit with TLS certificates and at rest using the Transit secrets engine.
How do dynamic credentials from Vault improve security compared to static secrets?
Dynamic credentials are created on-demand and have a short time to live (TTL), after which they are automatically revoked. This time boxes access to systems and reduces the risk of secrets being exposed or misused, unlike static secrets that remain valid indefinitely unless manually rotated.
What role does Vault Agent play in integrating Vault with applications?
Vault Agent handles authentication with Vault and secret retrieval on behalf of applications. It stores secrets in formats that applications can use, such as environment variables or files, without requiring applications to modify their code to interact directly with the Vault API.
Next steps
In this tutorial, you learned about the basic features of Vault, how clients authenticate with Vault, and about the tools you can use to integrate Vault with your existing services.
In the next tutorial, you will learn how to retrieve secrets from Vault and use them in the software development life cycle.