PKI external CA secrets engine

Overview

The PKI External CA secrets engine is a HashiCorp Enterprise plugin that enables Vault to acquire signed leaf certificates from external Certificate Authorities (CAs) through the ACME (Automatic Certificate Management Environment) protocol. You can use the plugin to leverage public CAs like Let's Encrypt, or other ACME-compliant CAs while maintaining the security of your Vault server.

You can use the PKI external CA plugin to:

1. Manage ACME accounts with external CAs.
2. Automate certificate acquisition through the ACME protocol.
3. Cache and distribute certificates to applications.

The PKI External CA secrets engine is verified to work with

• GlobalSign
• Sectigo
• DigiCert

Setup

1. Enable the plugin:

   $ vault secrets enable pki-external-ca
   

2. Configure an ACME account:

   Let's Encrypt (production)Let's Encrypt (staging)ACME CAs with EAB

   $ vault write pki-external-ca/config/acme-account/letsencrypt-prod  \
       directory_url="https://acme-v02.api.letsencrypt.org/directory"  \
       email_contacts="admin@example.com,security@example.com"         \
       key_type="ec-256"
   

   $ vault write pki-external-ca/config/acme-account/letsencrypt-staging      \
       directory_url="https://acme-staging-v02.api.letsencrypt.org/directory" \
       email_contacts="admin@example.com"                                     \
       key_type="ec-256"
   

   Some ACME servers require External Account Binding (EAB) for account registration:

   $ vault write pki-external-ca/config/acme-account/acme-eab      \
       directory_url="https://acme.example.com/v2/acme/directory"  \
       email_contacts="admin@example.com"                          \
       eab_kid="your-eab-key-id"                                   \
       eab_key="urlbase64-encoded-eab-key"                         \
       key_type="rsa-2048"
   

3. Create a role definition file (role.json) to set the domains available for request and how Vault should issue certificates. The examples below show common configurations:

   Explicit domain matchWildcard domain matchSubdomain matchIdentity template

   {
       "acme_account_name":      "letsencrypt-prod",
       "allowed_domains":        "example.com,www.example.com,api.example.com",
       "allowed_domain_options": "bare_domains"
   }
   

   {
       "acme_account_name":       "letsencrypt-prod",
       "allowed_domains":         "*.example.com",
       "allowed_domain_options":  "wildcards",
       "allowed_challenge_types": "dns-01"
   }
   

   {
       "acme_account_name":      "letsencrypt-prod",
       "allowed_domains":        "example.com",
       "allowed_domain_options": "bare_domains,subdomains"
   }
   

   {
       "acme_account_name":      "letsencrypt-prod",
       "allowed_domains":        "{{identity.entity.name}}.users.example.com",
       "allowed_domain_options": "bare_domains"
   }
   

1. Use the JSON file to create the roles in Vault:

$ vault write pki-external-ca/role/web-servers @role.json

DNS providers for automatic challenge fulfillment

The PKI External CA secrets engine integrates with certain DNS providers to enable automatic DNS-01 challenge fulfillment. You configure DNS providers globally and share them across all ACME accounts, allowing you to automate certificate issuance without manual intervention or external systems.

Challenge fulfillment modes

The plugin supports three challenge fulfillment modes:

1. Manual/External Fulfillment (default behavior): The plugin exposes challenges via API endpoints for external systems to fulfill. The external systems retrieve challenge tokens from Vault and manually create DNS records or HTTP tokens.

2. Automatic DNS Fulfillment: Vault automatically manages DNS records using configured provider credentials. Vault handles the entire ACME certificate request process when it receives an order that matches all the requested identifiers for a configured DNS provider.

3. Hybrid Fulfillment: You can configure Vault to fulfill some of the challenges automatically while others require manual fulfillment. Hybrid fulfillment is useful when you have DNS providers configured for some domains but not others.

DNS provider configuration

You manage DNS providers independently with the config/dns/<provider-type>/<name> endpoint and specify the supported identifiers using glob patterns. During ACME challenge evaluation for any order from any ACME account, Vault:

1. Selects one or more matching DNS providers based on the identifiers in the order.
2. Requires manual fulfillment for challenges with no matching provider.
3. Identifies challenges for automatic fulfillment using the DNS provider.
4. Waits for manual challenge fulfillment then performs automatic DNS updates for challenges that qualify for automatic fulfillment.

Supported DNS providers

The plugin supports multiple DNS providers, including:

• AWS Route53 - Amazon DNS service
• Azure DNS - Microsoft Azure DNS service
• Google Cloud DNS - Google Cloud Platform DNS service
• RFC2136 - Dynamic DNS updates for BIND and other RFC2136-compliant servers

Configure DNS providers

For additional details on supported arguments for DNS providers, see the API-docs page.

$ vault write pki-external-ca/config/dns/aws-route53/production \
    identifiers="*.example.com,*.prod.example.com" \
    access_key_id="AKIAIOSFODNN7EXAMPLE" \
    secret_access_key="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" \
    region="us-east-1"

$ vault write pki-external-ca/config/dns/azure-dns/production \
    identifiers="*.example.com" \
    zone_name="my-zone-id" \
    client_id="00000000-0000-0000-0000-000000000000" \
    client_secret="your-client-secret"

$ vault write pki-external-ca/config/dns/google-cloud-dns/production \
    identifiers="*.example.com" \
    credentials=@gcp-service-account.json

$ vault write pki-external-ca/config/dns/rfc2136/bind-dns \
    identifiers="*.example.com" \
    nameserver="ns1.example.com:53" \
    tsig_key_name="my-key" \
    tsig_secret="base64-encoded-secret" \
    tsig_algorithm="hmac-sha256"

Testing DNS providers

You can use the test API endpoint before using a DNS provider within an ACME workflow to confirm the DNS provider configuration works without requesting any certificates.

For example, the following command confirms that:

• The DNS provider credentials are correct.
• Vault can create and delete DNS records.
• The DNS provider can service the specified identifier.

$ vault write pki-external-ca/dns/test/workflow \
    provider_type="aws-route53" \
    provider_name="production" \
    identifier="test.example.com"

Certificate workflows

The PKI External CA secrets engine supports two distinct workflows for requesting certificates, similar to the sign and issue endpoints in the standard PKI secrets engine.

You can choose the workflow that best fits your security requirements and operational needs.

Workflow comparison

When to use each workflow

Use the CSR workflow when:

• You have existing key management processes and infrastructure.
• You need to comply with security policies that prohibit key transmission.
• You deploy single-instance applications where key sharing is not needed.

Use the identifier workflow when:

• You prioritize operational simplicity with minimal client complexity.
• You want Vault to handle key generation and management.
• You need to retrieve both the certificate and key at a later time.

How to request certificates

To create a new order and retrieve the public certificate, and optionally the private key, the following steps must be performed.

Role configuration options

Within each role, you can specify which ACME account to use, the allowed domains, and how to fulfill challenges.

Limit allowed domains

Roles support several options for controlling the domains clients can request. Each option influences how Vault matches the allowed_domains value against identifiers in the new-order request.

Bare domains

Use the bare_domains option to only allow matching domains without a subdomain prefix. For example, the following configuration allows example.com and www.example.com, but denies api.example.com and sub.example.com:

$ vault write pki-external-ca/role/example \
    acme_account_name="letsencrypt-prod" \
    allowed_domains="example.com,www.example.com" \
    allowed_domain_options="bare_domains"

Subdomains

Use the subdomains option to explicitly match to subdomains of a given domain and domains with the www prefix. For example, the following configuration allows www.example.com, api.example.com and sub.example.com, but denies example.com:

$ vault write pki-external-ca/role/example \
    acme_account_name="letsencrypt-prod" \
    allowed_domains="example.com" \
    allowed_domain_options="subdomains"

Wildcards

Use the wildcards option to match to any single subdomain of a given domain and domains with the www prefix provided it succeeds a DNS-01 challenge. For example, the following configuration allows www.example.com and any subdomain of the form <subdomain>.example.com, but denies example.com and any grandchild domains (*.*.example.com):

$ vault write pki-external-ca/role/example \
    acme_account_name="letsencrypt-prod" \
    allowed_domains="*.example.com" \
    allowed_domain_options="wildcards" \
    allowed_challenge_types="dns-01"

Globs

Use the globs option to match against glob patterns for the given domain. For example, the following configuration allows api.prod.example.com and web.staging.example.com, but denies example.com or prod.example.com:

$ vault write pki-external-ca/role/example \
    acme_account_name="letsencrypt-prod" \
    allowed_domains="*.prod.example.com,*.staging.example.com" \
    allowed_domain_options="globs"

Identity templating

Use ACL path templating in the allowed_domains field to create dynamic domain allowlists based on the authenticated entity. For example, the following configuration allows <username>.users.example.com provided the entity name of the user and the requested subdomain match:

$ vault write pki-external-ca/role/user-certs \
    acme_account_name="letsencrypt-prod" \
    allowed_domains="{{identity.entity.name}}.users.example.com" \
    allowed_domain_options="bare_domains"

When user "alice" requests a certificate:

• Allowed: alice.users.example.com
• Denied: bob.users.example.com

Available template variables:

• {{identity.entity.id}} - Entity ID
• {{identity.entity.name}} - Entity name
• {{identity.entity.metadata.<key>}} - Entity metadata
• {{identity.groups.names.<group>}} - Group membership

Domain matching behavior

The following table breaks down how the allowed_domains and allowed_domain_options for a role interact to forbid or allow identifiers when you call the new-order endpoint.

Monitoring and troubleshooting

Check order status

$ vault read pki-external-ca/role/web-servers/order/01936d8e-7c3a-7890-b123-456789abcdef/status

Order statuses:

• new - Order created, not yet submitted to ACME server
• submitted - Submitted to ACME server
• awaiting-challenge-fulfillment - Waiting for client to fulfill challenges
• vault-challenge-fulfillment - Vault is fulfilling DNS challenges
• vault-challenge-propogating - Vault waits for internal validation of the DNS challenges it made in vault_challenge_fulfillment
• notify-acme-server-challenges-completed - Ready to notify ACME server
• processing-challenge - ACME server is validating challenges
• fetching-certificate - Retrieving issued certificate, the ACME server has accepted the challenges
• completed - Certificate issued successfully
• expired - Order expired before completion
• revoked - Vault revoked the certificate
• error - An error occurred

List active orders

$ vault list pki-external-ca/role/web-servers/active-orders

Lookup certificate by serial

$ vault read pki-external-ca/lookup/cert/03:e7:1f:a2:3d

View ACME account details

$ vault read pki-external-ca/config/acme-account/letsencrypt-prod

Known limitations

Load-balanced applications with multiple instances

The current implementation works best when a single orchestrator or instance manages certificate requests. If you deploy multiple instances of an application behind a load balancer, where each instance independently requests certificates from Vault, you may encounter challenges:

• Multiple instances may create separate orders for the same identifiers simultaneously.
• Each order generates unique challenge tokens specific to that order.
• The ACME CA may validate challenges against a different instance than the one that received the challenge information and cause validation failures or certificate issuance problems.

Recommended workarounds:

1. Single orchestrator pattern: Designate one instance or orchestrator to handle certificate requests. After Vault issues the certificate, distribute it to all instances that need the certificate.

2. Identifier workflow with caching: Use the identifier workflow (not CSR workflow) so Vault generates and caches both the certificate and private key. Once cached, clients can fetch the the same certificate and key from Vault.

IP address identifiers

The plugin currently does not support requesting certificates with IP address identifiers.

ACME Renewal Information (ARI)

The plugin does not currently support the ACME Renewal Information (ARI) extension (RFC 9773), which allows ACME servers to notify clients when they should renew certificates. As a result, your clients need to decide certificate renewal timing independently.

Automatic ACME account key rotation

The plugin cannot automatically rotate ACME account keys based on key age. We recommend manually rotating account keys as needed using the /config/acme-account/:name/rotate-key endpoint.

Vault Agent integration

Use Vault Agent to automate the full certificate lifecycle for PKI external CA mounts. Refer to Automate certificates with Vault Agent and PKI external CA for configuration details.

API documentation

For detailed API documentation, see PKI External CA API.