Use HCP Packer and Terraform to manage cloud container registries

Author: Bruce Harrison

This guide will help you with creating and managing Docker containers via HCP Packer and interface them with cloud service providers (CSP) container registries. This guide will demonstrate how to push images to AWS, Azure, and GCP container registries.

This guide uses both Packer CE and the HCP Packer platform. You will use Packer CE to build containers and VM images. You will use HCP Packer to store container and VM metadata generated by Packer CE.

By using Packer CE and HCP Packer, you gain the following benefits:

• Capturing of container metadata by the HCP platform
  • Visibility into container lineage
  • Notification when a derived image is outdated due to a new version of the parent being released
  • Leverage HCP Packer webhooks for more advanced cloud providers registry management
• VM images and containers can share a common declarative file format (Packer HCL) that is used to describe and build both.

Target audience

This guide references the following roles:

• Platform operator: Someone responsible for setting up the image repositories, building the containers, and pushing them to the respective cloud provider container registries.

Validated architecture

The following diagram shows the process of pushing a container to a cloud provider's container registry using HCP Packer.

• This document focuses on running this process from a local machine, but the same concepts could easily be applied to a CI/CD mechanism such as Github Actions or Gitlab Runners.

• We recommend using Terraform to provision the underlying infrastructure.

• Due to the way Packer post-processors and docker login works, if you plan on pushing images to multiple cloud providers, you should run each build in an isolated context. Attempting to push multiple images in parallel from a single context can cause transient failures since they all compete for docker login access.

• Packer offers a docker-push post processor, but this guide does not cover this post-processor. The post-processor is inherently inflexible, and causes wide divergence in container push processes across cloud providers. Instead, you will be manually logging into registries and pushing the containers. This method is broadly applicable to CI/CD scenarios as well.

• Every cloud provider has unique constraints on how credentials are handled. This guide uses long-lived credentials for brevity, but this may not work for all organizations.

  Platform operators should consult with their internal security teams to ensure that the cloud providers authentication method aligns with company policies and practices.

Prerequisites

• An active HCP service principal.
• An existing HCP Packer Account (HCP Platform).
• Packer OSS CLI >=v1.11.0 installed.
• Have a working Packer HCL file configured for building a Docker container.
• Git CLI installed.

For your desired cloud provider, ensure you have the following:

• Necessary permissions to create resources within your chosen cloud provider.

• AWS CLI installed locally and configured with your AWS credentials

Create image repository

You can create an image repository using either Terraform or the AWS CLI.

resource "aws_ecr_repository" "demo" {
  name                 = "<repository-name>"
  image_tag_mutability = "IMMUTABLE"

  image_scanning_configuration {
    scan_on_push = true
  }
}

output "repository_url" {
  value = aws_ecr_repository.demo.repository_url
}

$ aws ecr create-repository --repository-name <repository-name>

Push container to registry

Locate your HCP service principal credentials and export them as environment variables.

$ export HCP_CLIENT_ID=
$ export HCP_CLIENT_SECRET=

In your Packer template, update the following fields:

• Replace <region> with the region you deployed your image repository.
• Replace <aws-account-number> with the account number you deployed your image repository.
• Replace <repository-name> with the name of the image repository you created.

Build the container.

$ packer build docker-debian-aws.pkr.hcl

Login to ECR via AWS CLI. Update <region> with the region you deployed your image repository. Update <aws-account-number> with the account number you deployed your image repository.

$ aws ecr get-login-password --region <region> | \
docker login --username AWS --password-stdin <aws-account-number>.dkr.ecr.<region>.amazonaws.com

Push the container to ECR. Update <repository-name> with the name of the image repository you created.

$ docker push <aws-account-number>.dkr.ecr.<region>.amazonaws.com/<repository-name>:latest

You have pushed your image to the AWS Repository. It is now available to be consumed by services with permission to pull it.

Conclusion

In this guide, you learned how to push containers to cloud provider container registries using Packer CE and HCP Packer. To learn more, check out the following resources:

• Packer installation instructions
• Packer docker-push Post Processor
• HCP Packer Registry metadata block
• Push metadata to HCP Packer