Standardize Machine Images Across Multiple Cloud Providers

As your organization grows, you may adopt a hybrid or multi-cloud strategy to enable innovation, increase resiliency, decrease costs, or integrate different systems. Packer is a cloud-agnostic tool that lets you build identical machine images for multiple platforms from a single template file. By tracking your build metadata through HCP Packer, you can query it for future downstream Packer builds or reference images in your Terraform configuration.

In this tutorial, you will build and deploy a machine image containing HashiCups, a fictional coffee-shop application, in AWS and Azure. To do so, you will use Packer to build and store the images in AWS and Azure, push the build metadata to HCP Packer, and use Terraform to deploy the images to their respective cloud providers. In the process, you will learn how to use Packer and HCP Packer to standardize machine images across multi-cloud and hybrid environments.

Prerequisites

This tutorial assumes that you are familiar with the workflows for Packer, HCP Packer, and either Terraform OSS or Terraform Cloud. If you are new to Packer, complete the Packer Get Started tutorials first. If you are new to HCP Packer, complete the Get Started with HCP Packer tutorials.

Terraform Cloud is a platform that you can use to manage and execute your Terraform projects. It includes features like remote state and execution, structured plan output, workspace resource summaries, and more. The workflow for Terraform Cloud is the same as Terraform OSS.

Select the Terraform OSS tab if you would rather complete this tutorial using Terraform OSS.

Now, install Packer 1.7.10+ locally.

You will also need an HCP account with an HCP Packer Registry.

Next, create a new HCP service principal and set the following environment variables locally.

You will also need an AWS account with credentials set as local environment variables.

If you do not have one already, create an Azure account.

In your Azure account, create an Azure Active Directory Service Principal scoped to your Subscription, with the Contributor role, and an application secret. Be sure to copy the application secret value generated by Azure. Then, set the following environment variables.

Next, create an Azure Resource Group in the US West 3 region and set the following environment variable.

Clone repository

In your terminal, clone the example repository.

$ git clone https://github.com/hashicorp/learn-packer-multicloud

Navigate to the cloned repository.

$ cd learn-packer-multicloud

Review Packer template

The packer directory contains files Packer uses to build images.

In your editor, open variables.pkr.hcl. Packer uses the environment variables you set earlier for the first four variables.

variable "arm_client_id" {
  type = string
  default = env("ARM_CLIENT_ID")
}

variable "arm_client_secret" {
  type = string
  default = env("ARM_CLIENT_SECRET")
}

variable "arm_subscription_id" {
  type = string
  default = env("ARM_SUBSCRIPTION_ID")
}

variable "azure_resource_group" {
  type = string
  default = env("TF_VAR_azure_resource_group")
}

variable "azure_region" {
  type = string
  default = "westus3"
}

variable "aws_region" {
  type = string
  default = "us-west-1"
}

Now, open packer/build.pkr.hcl.

The azure-arm.ubuntu-lts source block uses the client_id, client_secret, and subscription_id parameters to authenticate to Azure. Packer retrieves an Ubuntu 22.04 image to use as the base image and stores built images in the resource group specified by the managed_image_resource_group_name parameter.

source "azure-arm" "ubuntu-lts" {
  client_id                         = var.arm_client_id
  client_secret                     = var.arm_client_secret
  subscription_id                   = var.arm_subscription_id

  os_type         = "Linux"
  image_offer     = "0001-com-ubuntu-server-jammy"
  image_publisher = "Canonical"
  image_sku       = "22_04-lts"

  managed_image_resource_group_name = var.azure_resource_group
  ## ...
}

The amazon-ebs.ubuntu-lts source block retrieves an Ubuntu 22.04 image to use as a base image, from the region specified in the region variable. The Amazon plugin for Packer uses the AWS credential environment variables you set earlier to authenticate to AWS.

source "amazon-ebs" "ubuntu-lts" {
  source_ami_filter {
    filters = {
      virtualization-type = "hvm"
      name                = "ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*"
      root-device-type    = "ebs"
    }
    owners      = ["099720109477"]
    most_recent = true
  }
  region = var.aws_region
  ## ...
}

The build block references the image sources defined in the source blocks. Packer standardizes your machine images across clouds by following the same instructions to build both images.

build {
  source "source.amazon-ebs.ubuntu-lts" {
    name = "hashicups"
  }

  source "source.azure-arm.ubuntu-lts" {
    name               = "hashicups"
    location           = var.azure_region
    managed_image_name = "hashicups_${local.date}"
  }

  # systemd unit for HashiCups service
  provisioner "file" {
    source      = "hashicups.service"
    destination = "/tmp/hashicups.service"
  }

  # Set up HashiCups
  provisioner "shell" {
    scripts = [
      "setup-deps-hashicups.sh"
    ]
  }

 ## ...
}

First, Packer creates a virtual machine from each source image in both cloud providers. Then, it copies the HashiCups systemd unit file to each machine and runs the setup-deps-hashicups.sh script to install and configure HashiCups. When the script finishes, Packer asks each cloud provider to create a new image from each virtual machine.

Finally, Packer sends image metadata from the newly built images to the specified HCP Packer registry bucket so you can reference the images.

  ## ...

  # HCP Packer settings
  hcp_packer_registry {
    bucket_name = "learn-packer-multicloud-hashicups"
    description = <<EOT
This is an image for HashiCups.
    EOT

    bucket_labels = {
      "hashicorp-learn" = "learn-packer-multicloud-hashicups",
    }
  }
}

Build HashiCups images

Change into the packer directory.

$ cd packer

Initialize the Packer template to install the required AWS and Azure plugins.

$ packer init .
Installed plugin github.com/hashicorp/azure v1.0.6 in "…"
Installed plugin github.com/hashicorp/amazon v1.0.9 in "…"

Packer installs the plugins specified in the required plugins block in the build.pkr.hcl template. Packer plugins are standalone applications that you can use to perform tasks during builds. They extend Packer's capabilities, similarly to Terraform providers.

Now, build the HashiCups images in both AWS and Azure.

$ packer build .
amazon-ebs.ubuntu-hirsute: output will be in this color.
azure-arm.hashicups: output will be in this color.

==> azure-arm.hashicups: Publishing build details for azure-arm.hashicups to the HCP Packer registry
==> amazon-ebs.ubuntu-hirsute: Publishing build details for amazon-ebs.ubuntu-hirsute to the HCP Packer registry
## ...
Build 'amazon-ebs.ubuntu-hirsute' finished after 4 minutes 58 seconds.
## ...
Build 'azure-arm.hashicups' finished after 5 minutes 32 seconds.

==> Wait completed after 5 minutes 32 seconds

==> Builds finished. The artifacts of successful builds are:
--> azure-arm.hashicups: Azure.ResourceManagement.VMImage:

OSType: Linux
ManagedImageResourceGroupName: learn-packer-multicloud-rg
ManagedImageName: hashicups_0451
ManagedImageId: /subscriptions/c9ed8610-47a3-4107-a2b2-a322114dfb29/resourceGroups/learn-packer-multicloud-rg/providers/Microsoft.Compute/images/hashicups_0451
ManagedImageLocation: westus

--> azure-arm.hashicups: Published metadata to HCP Packer registry packer/learn-packer-multicloud-hashicups/iterations/01G13Y6310TM7SEXPYXY44B2RG
--> amazon-ebs.ubuntu-hirsute: AMIs were created:
us-west-1: ami-03cbdc7f691ad8373

--> amazon-ebs.ubuntu-hirsute: Published metadata to HCP Packer registry packer/learn-packer-multicloud-hashicups/iterations/01G13Y6310TM7SEXPYXY44B2RG

Packer builds images in parallel in each cloud provider, reducing the total build time.

Continue on to the next section while the build completes to learn how to deploy the images to multiple clouds using Terraform. To skip the deployment step, proceed to Clean up your infrastructure.

Review Terraform configuration for HashiCups

The terraform directory contains the Terraform configuration to deploy the HashiCups machine images to Azure and AWS.

Open terraform/variables.tf. This file contains variables used by the rest of the configuration. The aws_region and azure_region variables control which image metadata Terraform should request from HCP Packer for, and the regions where Terraform will deploy HashiCups images and infrastructure.

variable "aws_region" {
  description = "The AWS region Terraform should deploy your instance to"
  default     = "us-west-1"
}

variable "azure_region" {
  description = "The Azure region Terraform should deploy your instance to"
  default = "westus3"
}

variable "cidr_vpc" {
  description = "CIDR block for the VPC"
  default     = "10.1.0.0/16"
}
variable "cidr_subnet" {
  description = "CIDR block for the subnet"
  default     = "10.1.0.0/24"
}

variable "environment_tag" {
  description = "Environment tag"
  default     = "Learn"
}

variable "hcp_bucket_hashicups" {
  description = "HCP Packer bucket name for hashicups image"
  default     = "learn-packer-multicloud-hashicups"
}

variable "hcp_channel" {
  description = "HCP Packer channel name"
  default     = "production"
}

variable "azure_resource_group" {
  description = "Azure Resource Group name where Terraform will create infrastructure"
}

Now, open terraform/hcp.tf. This configuration retrieves image information from HCP Packer using data sources.

The hcp_packer_iteration.hashicups data source retrieves image iteration information from the production channel of the learn-packer-multicloud-hashicups bucket. These values are the defaults for the configuration's input variables.

data "hcp_packer_iteration" "hashicups" {
  bucket_name = var.hcp_bucket_hashicups
  channel     = var.hcp_channel
}

The hcp_packer_image data sources use the iteration ID from the hcp_packer_iteration data source to retrieve an image for each cloud provider. Notice the differences in the cloud_provider and region attributes between the two data sources.

data "hcp_packer_image" "aws_hashicups" {
  bucket_name    = data.hcp_packer_iteration.hashicups.bucket_name
  iteration_id   = data.hcp_packer_iteration.hashicups.ulid
  cloud_provider = "aws"
  region         = var.aws_region
}

data "hcp_packer_image" "azure_hashicups" {
  bucket_name    = data.hcp_packer_iteration.hashicups.bucket_name
  iteration_id   = data.hcp_packer_iteration.hashicups.ulid
  cloud_provider = "azure"
  region         = var.azure_region
}

Open terraform/aws.tf. This configuration defines a VPC and network resources, an AWS EC2 instance running HashiCups, and a security group with public access on port 80. Notice that the HashiCups instance references the hcp_packer_image.aws_hashicups data source.

resource "aws_instance" "hashicups" {
  ami                         = data.hcp_packer_image.aws_hashicups.cloud_image_id
  instance_type               = "t2.micro"
  subnet_id                   = aws_subnet.subnet_public.id
  vpc_security_group_ids      = [aws_security_group.hashicups.id]
  associate_public_ip_address = true

  tags = {
    Name = "HashiCups"
    Name = "Learn-Packer-MultiCloud"
  }
}

Open terraform/azure.tf. This configuration defines a VPC and network resources, an Azure virtual machine running HashiCups, and a security group with public access on port 80. The virtual machine references the hcp_packer_image.azure_hashicups data source.

resource "azurerm_linux_virtual_machine" "hashicups" {
  name                = "${var.prefix}-vm"
  source_image_id     = data.hcp_packer_image.azure_hashicups.cloud_image_id
  resource_group_name = data.azurerm_resource_group.main.name
  location            = data.azurerm_resource_group.main.location
  size                = "Standard_B1s"
  admin_username      = "ubuntu"
  admin_password      = "adminPass1!"

  disable_password_authentication = true
  network_interface_ids = [azurerm_network_interface.main.id]

  os_disk {
    caching              = "ReadWrite"
    storage_account_type = "Standard_LRS"
  }

  tags = {
    name  = "hashicups"
    learn = "learn-packer-multicloud"
  }
}

Initialize configuration

Change to the terraform directory.

$ cd ../terraform

$ export TF_CLOUD_ORGANIZATION=

$ terraform init

Initializing Terraform Cloud...

Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
- Reusing previous version of hashicorp/azurerm from the dependency lock file
- Reusing previous version of hashicorp/hcp from the dependency lock file
- Installing hashicorp/aws v4.30.0...
- Installed hashicorp/aws v4.30.0 (signed by HashiCorp)
- Installing hashicorp/azurerm v3.22.0...
- Installed hashicorp/azurerm v3.22.0 (signed by HashiCorp)
- Installing hashicorp/hcp v0.44.0...
- Installed hashicorp/hcp v0.44.0 (signed by HashiCorp)

Terraform Cloud has been successfully initialized!

You may now begin working with Terraform Cloud. Try running "terraform plan" to
see any changes that are required for your infrastructure.

If you ever set or change modules or Terraform Settings, run "terraform init"
again to reinitialize your working directory.

terraform {
  /*
  cloud {
    workspaces {
      name = "learn-packer-multicloud"
    }
  }
  */
  ## ...
}

$ terraform init
Initializing the backend...
##...
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Wait for Packer to finish building your images, then continue with the tutorial.

Verify machine images

When Packer finishes building your images, navigate to your learn-packer-multicloud-hashicups bucket in the HCP Packer dashboard.

Click on Iterations, then select the first iteration, labeled v1. Notice that this iteration has two builds — one for Azure, the other for AWS.

The respective AWS and Azure data sources in your Terraform configuration reference each of these artifacts.

Create HCP Packer channel

HCP Packer channels allow you to reference a specific build iteration in Packer or Terraform.

In the HCP console, click on Channels, then click on New Channel. Create a new channel named production and set it to the v1 iteration of your learn-packer-multicloud-hashicups bucket.

Terraform will query the production channel to retrieve the Azure and AWS image IDs and deploy the appropriate images.

Deploy images

In your terminal, apply your configuration to deploy HashiCups images in both Azure and AWS. Respond yes to the prompt to confirm the operation.

$ terraform apply
Running apply in Terraform Cloud. Output will stream here. Pressing Ctrl-C
will cancel the remote apply if it's still pending. If the apply started it
will stop streaming the logs, but will not stop the apply running remotely.

Preparing the remote apply...

To view this run in a browser, visit:
https://app.terraform.io/app/hashicorp-learn/learn-packer-multicloud/runs/run-000

Waiting for the plan to start...

Terraform v1.1.6
on linux_amd64
Initializing plugins and modules...

## ...

Plan: 15 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + aws_public_ip   = (known after apply)
  + azure_public_ip = (known after apply)

Do you want to perform these actions in workspace "learn-packer-multicloud"?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

## ...

Apply complete! Resources: 15 added, 0 changed, 0 destroyed.

Outputs:

aws_public_ip = "54.193.153.230"
azure_public_ip = "104.42.116.112"

Visit the addresses from the aws_public_ip and azure_public_ip outputs in your browser to view the HashiCups application.

You successfully built and deployed identical machine images across multiple clouds with Packer and Terraform.

Clean up your infrastructure

Before moving on, destroy the infrastructure you created in this tutorial.

In the terraform directory, destroy the infrastructure for the HashiCups application. Respond yes to the prompt to confirm the operation.

$ terraform destroy
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  - destroy

Terraform will perform the following actions:
##...
Plan: 0 to add, 0 to change, 15 to destroy.
##...

Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: yes
##...
Destroy complete! Resources: 15 destroyed.

Delete Azure resource group and machine images

Your Azure account still has machines images in the resource group you created for this tutorial.

In the Azure portal, visit Resource groups. Then, select the name of the resource group you created for this tutorial.

If you want to keep the Resource Group, select all images in the resource list whose name begins with hashicups_. Then select Delete to delete them.

If you no longer need the resource group, select Delete resource group and follow the on-screen instructions to delete it. Azure will delete all resources contained in the resource group, including images, before deleting the group itself.

Delete AWS machine images

Your AWS account still has machine images and their respective snapshots, which you may be charged for depending on your other usage.

In your us-west-1 AWS account, deregister the AMI by selecting it, clicking on the Actions button, then the Deregister AMI option, and finally confirm by clicking the Deregister AMI button in the confirmation dialog.

Delete the snapshots by selecting the snapshots, clicking on the Actions button, then the Delete snapshot option, and finally confirm by clicking the Delete button in the confirmation dialog.

Clean up Terraform Cloud resources

If you used the Terraform Cloud workflow, navigate to your learn-packer-multicloud workspace in Terraform Cloud and delete the workspace.

Next steps

In this tutorial, you built machine images from the same Packer template in AWS and Azure, pushed the metadata to HCP Packer, and deployed virtual machines using the built images. In the process, you learned how you can use Packer and HCP Packer to standardize your machine images as you adopt a multi-cloud strategy.

For more information on topics covered in this tutorial, check out the following resources.

• Complete the Build a Golden Image Pipeline with HCP Packer tutorial to build a sample application image with a golden image pipeline, and deploy it to AWS using Terraform.
• Complete the Set Up Terraform Cloud Run Task for HCP Packer tutorial to learn how to set up run tasks that ensure your Terraform configuration uses compliant machine images.