Upgrade Packer JSON Template to HCL2

9min
Packer

As of version 1.7.0, HCL2 is the preferred way to write Packer templates. You can use the hcl2_upgrade command to transition your existing Packer JSON template to HCL2. This enables you to preserve your existing workflows while leveraging HCL2’s advanced features like variable interpolation and configuration composability.

In this tutorial, you will upgrade a Packer JSON template that builds a Docker image to HCL2. Then, you will add provisioners and post processors to the upgraded HCL2 template.

Prerequisites

This tutorial assumes that you are familiar with the standard Packer workflow. If you are new to Packer, complete the Get Started tutorials first. These tutorials will also introduce you to HCL2 blocks.

For this tutorial, you will need:

the Packer CLI 1.7+ installed locally
Docker

Clone the sample repository

Clone the sample repository for this tutorial, which contains a Packer template to create a Docker image.

$ git clone https://github.com/hashicorp/learn-packer-upgrade-json-config.git

$ git clone https://github.com/hashicorp/learn-packer-upgrade-json-config.git

Change into the repository directory.

$ cd learn-packer-upgrade-json-config

$ cd learn-packer-upgrade-json-config

Review Packer template

Open docker-ubuntu.json.

{
  "variables": {
    "image": "ubuntu:xenial"
  },
  "builders": [
    {
      "type": "docker",
      "image": "{{user `image`}}",
      "commit": true
    }
  ],
  "provisioners": [
    {
      "type": "shell",
      "inline": [
        "echo Hello world!"
      ]
    }
  ]
}

{
  "variables": {
    "image": "ubuntu:xenial"
  },
  "builders": [
    {
      "type": "docker",
      "image": "{{user `image`}}",
      "commit": true
    }
  ],
  "provisioners": [
    {
      "type": "shell",
      "inline": [
        "echo Hello world!"
      ]
    }
  ]
}

This is a standard Packer JSON template that has a variable, a builder and a provisioner. This template runs an ubuntu:xenial Docker container, prints the "Hello world!" string, then saves the container to a Docker image.

Upgrade Packer JSON template to HCL2

Use the hcl2_upgrade command to upgrade the JSON template to HCL2. The -with-annotations flag provides context for the auto-generated template.

$ packer hcl2_upgrade -with-annotations docker-ubuntu.json
Successfully created docker-ubuntu.json.pkr.hcl

$ packer hcl2_upgrade -with-annotations docker-ubuntu.json
Successfully created docker-ubuntu.json.pkr.hcl

Review the HCL2 configuration

Open the newly created Packer template, docker-ubuntu.json.pkr.hcl.

# This file was autogenerated by the 'packer hcl2_upgrade' command. We
# recommend double checking that everything is correct before going forward. We
# also recommend treating this file as disposable. The HCL2 blocks in this
# file can be moved to other files. For example, the variable blocks could be
# moved to their own 'variables.pkr.hcl' file, etc. Those files need to be
# suffixed with '.pkr.hcl' to be visible to Packer. To use multiple files at
# once they also need to be in the same folder. 'packer inspect folder/'
# will describe to you what is in that folder.

# Avoid mixing go templating calls ( for example ```{{ upper(`string`) }}``` )
# and HCL2 calls (for example '${ var.string_value_example }' ). They won't be
# executed together and the outcome will be unknown.

# All generated input variables will be of 'string' type as this is how Packer JSON
# views them; you can change their type later on. Read the variables type
# constraints documentation
# https://www.packer.io/docs/templates/hcl_templates/variables#type-constraints for more info.
variable "image" {
  type    = string
  default = "ubuntu:xenial"
}

# source blocks are generated from your builders; a source can be referenced in
# build blocks. A build block runs provisioner and post-processors on a
# source. Read the documentation for source blocks here:
# https://www.packer.io/docs/templates/hcl_templates/blocks/source
source "docker" "autogenerated_1" {
  commit = true
  image  = "${var.image}"
}

# a build block invokes sources and runs provisioning steps on them. The
# documentation for build blocks can be found here:
# https://www.packer.io/docs/templates/hcl_templates/blocks/build
build {
  sources = ["source.docker.autogenerated_1"]

  provisioner "shell" {
    inline = ["echo Hello world!"]
  }
}

# This file was autogenerated by the 'packer hcl2_upgrade' command. We
# recommend double checking that everything is correct before going forward. We
# also recommend treating this file as disposable. The HCL2 blocks in this
# file can be moved to other files. For example, the variable blocks could be
# moved to their own 'variables.pkr.hcl' file, etc. Those files need to be
# suffixed with '.pkr.hcl' to be visible to Packer. To use multiple files at
# once they also need to be in the same folder. 'packer inspect folder/'
# will describe to you what is in that folder.

# Avoid mixing go templating calls ( for example ```{{ upper(`string`) }}``` )
# and HCL2 calls (for example '${ var.string_value_example }' ). They won't be
# executed together and the outcome will be unknown.

# All generated input variables will be of 'string' type as this is how Packer JSON
# views them; you can change their type later on. Read the variables type
# constraints documentation
# https://www.packer.io/docs/templates/hcl_templates/variables#type-constraints for more info.
variable "image" {
  type    = string
  default = "ubuntu:xenial"
}

# source blocks are generated from your builders; a source can be referenced in
# build blocks. A build block runs provisioner and post-processors on a
# source. Read the documentation for source blocks here:
# https://www.packer.io/docs/templates/hcl_templates/blocks/source
source "docker" "autogenerated_1" {
  commit = true
  image  = "${var.image}"
}

# a build block invokes sources and runs provisioning steps on them. The
# documentation for build blocks can be found here:
# https://www.packer.io/docs/templates/hcl_templates/blocks/build
build {
  sources = ["source.docker.autogenerated_1"]

  provisioner "shell" {
    inline = ["echo Hello world!"]
  }
}

The generated HCL2 template file contains three blocks:

A variable block that defines the image variable,
A source block that defines the builder Packer will use to build the image, and
A build block, a composite block, that defines what Packer will execute when running packer build docker-ubuntu.json.pkr.hcl.

The build block is similar to the full JSON build template file, with the "variables" and "source" block defining values and builders Packer will use for your images. You can reference "variables" and "source" blocks across multiple HCL2 template files or builds — see build-level source blocks for an example on source block reuse.

Update generated template

The hcl2_upgrade command automatically maps and generates the relevant HCL2 block.

You should rename these blocks to accurately represent and describe the resource. Rename docker.autogenerated_1 to docker.ubuntu.

- source "docker" "autogenerated_1" {
+ source "docker" "ubuntu" {
    commit = true
    image  = "${var.image}"
  }

  build {
-   sources = ["source.docker.autogenerated_1"]
+   sources = ["source.docker.ubuntu"]
   ## …
  }

- source "docker" "autogenerated_1" {
+ source "docker" "ubuntu" {
    commit = true
    image  = "${var.image}"
  }

  build {
-   sources = ["source.docker.autogenerated_1"]
+   sources = ["source.docker.ubuntu"]
   ## …
  }

In addition, since this Packer template uses the Docker v0.0.7 plugin, add the following packer block to the top of docker-ubuntu.json.pkr.hcl. This ensures Packer will retrieve the Docker plugin that fulfills the version contraint, so you can consistently generate an image from this template.

packer {
  required_plugins {
    docker = {
      version = ">= 0.0.7"
      source = "github.com/hashicorp/docker"
    }
  }
}

packer {
  required_plugins {
    docker = {
      version = ">= 0.0.7"
      source = "github.com/hashicorp/docker"
    }
  }
}

Build Packer image

First, initialize your template.

$ packer init .

$ packer init .

Then, build your image. The packer build . command loads all the contents in the current directory. You can also build specific images by specifying the template file directly.

$ packer build .
docker.ubuntu: output will be in this color.

==> docker.ubuntu: Creating a temporary directory for sharing data...
==> docker.ubuntu: Pulling Docker image: ubuntu:xenial
    docker.ubuntu: xenial: Pulling from library/ubuntu
    docker.ubuntu: Digest: sha256:eed7e1076bbc1f342c4474c718e5438af4784f59a4e88ad687dbb98483b59ee4
    docker.ubuntu: Status: Image is up to date for ubuntu:xenial
    docker.ubuntu: docker.io/library/ubuntu:xenial
==> docker.ubuntu: Starting docker container...
    docker.ubuntu: Run command: docker run -v /Users/youruser/.packer.d/tmp435754690:/packer-files -d -i -t --entrypoint=/bin/sh -- ubuntu:xenial
    docker.ubuntu: Container ID: bc3bcc9ed53bc2291fe2c31a93521ccd3ff45da38e2781ff9a57fe313f297dfc
==> docker.ubuntu: Using docker communicator to connect: 172.17.0.3
==> docker.ubuntu: Provisioning with shell script: /var/folders/s6/m22_k3p11z104k2vx1jkqr2c0000gp/T/packer-shell800525815
    docker.ubuntu: Hello world!
==> docker.ubuntu: Committing the container
    docker.ubuntu: Image ID: sha256:e763bc0ceb4873075a28b80986aa344d489861f4237a2049bd95025c1ec21882
==> docker.ubuntu: Killing the container: bc3bcc9ed53bc2291fe2c31a93521ccd3ff45da38e2781ff9a57fe313f297dfc
Build 'docker.ubuntu' finished after 4 seconds 487 milliseconds.

==> Wait completed after 4 seconds 487 milliseconds

==> Builds finished. The artifacts of successful builds are:
--> docker.ubuntu: Imported Docker image: sha256:e763bc0ceb4873075a28b80986aa344d489861f4237a2049bd95025c1ec21882

$ packer build .
docker.ubuntu: output will be in this color.

==> docker.ubuntu: Creating a temporary directory for sharing data...
==> docker.ubuntu: Pulling Docker image: ubuntu:xenial
    docker.ubuntu: xenial: Pulling from library/ubuntu
    docker.ubuntu: Digest: sha256:eed7e1076bbc1f342c4474c718e5438af4784f59a4e88ad687dbb98483b59ee4
    docker.ubuntu: Status: Image is up to date for ubuntu:xenial
    docker.ubuntu: docker.io/library/ubuntu:xenial
==> docker.ubuntu: Starting docker container...
    docker.ubuntu: Run command: docker run -v /Users/youruser/.packer.d/tmp435754690:/packer-files -d -i -t --entrypoint=/bin/sh -- ubuntu:xenial
    docker.ubuntu: Container ID: bc3bcc9ed53bc2291fe2c31a93521ccd3ff45da38e2781ff9a57fe313f297dfc
==> docker.ubuntu: Using docker communicator to connect: 172.17.0.3
==> docker.ubuntu: Provisioning with shell script: /var/folders/s6/m22_k3p11z104k2vx1jkqr2c0000gp/T/packer-shell800525815
    docker.ubuntu: Hello world!
==> docker.ubuntu: Committing the container
    docker.ubuntu: Image ID: sha256:e763bc0ceb4873075a28b80986aa344d489861f4237a2049bd95025c1ec21882
==> docker.ubuntu: Killing the container: bc3bcc9ed53bc2291fe2c31a93521ccd3ff45da38e2781ff9a57fe313f297dfc
Build 'docker.ubuntu' finished after 4 seconds 487 milliseconds.

==> Wait completed after 4 seconds 487 milliseconds

==> Builds finished. The artifacts of successful builds are:
--> docker.ubuntu: Imported Docker image: sha256:e763bc0ceb4873075a28b80986aa344d489861f4237a2049bd95025c1ec21882

List the images to verify Packer successfully built the image. The image ID should match the SHA produced in the Packer output.

$ docker images
REPOSITORY                        TAG                          IMAGE ID       CREATED          SIZE
<none>                            <none>                       e763bc0ceb48   3 minutes ago    134MB
## ...

$ docker images
REPOSITORY                        TAG                          IMAGE ID       CREATED          SIZE
<none>                            <none>                       e763bc0ceb48   3 minutes ago    134MB
## ...

Modify template

Now that you have built an image using a HCL2 template, you will now add a provisioner and post-processor to your template.

First, add the following provisioner in the build block after the existing provisioner.

provisioner "shell" {
  environment_vars = [
    "FOO=hello world",
  ]
  inline = [
    "echo Adding file to Docker Container",
    "echo \"FOO is $FOO\" > example.txt",
  ]
}

provisioner "shell" {
  environment_vars = [
    "FOO=hello world",
  ]
  inline = [
    "echo Adding file to Docker Container",
    "echo \"FOO is $FOO\" > example.txt",
  ]
}

This block defines a shell provisioner which sets an environment variable named FOO in the shell execution environment and runs the commands in the inline attribute. This provisioner will create a file named example.txt that contains FOO is hello world.

Next, add the following post-processor in the build block after the provisioner blocks.

post-processor "docker-tag" {
  repository = "learn-packer"
  tags       = ["ubuntu-xenial"]
}

post-processor "docker-tag" {
  repository = "learn-packer"
  tags       = ["ubuntu-xenial"]
}

This post-processor block will tag the newly created image with ubuntu-xenial.

After adding these provisioner and post-processor blocks, your build block should look like the following.

build {
  sources = ["source.docker.ubuntu"]

  provisioner "shell" {
    inline = ["echo Hello world!"]
  }

  provisioner "shell" {
    environment_vars = [
      "FOO=hello world",
    ]
    inline = [
      "echo Adding file to Docker Container",
      "echo \"FOO is $FOO\" > example.txt",
    ]
  }

  post-processor "docker-tag" {
    repository = "learn-packer"
    tags       = ["ubuntu-xenial"]
  }
}

build {
  sources = ["source.docker.ubuntu"]

  provisioner "shell" {
    inline = ["echo Hello world!"]
  }

  provisioner "shell" {
    environment_vars = [
      "FOO=hello world",
    ]
    inline = [
      "echo Adding file to Docker Container",
      "echo \"FOO is $FOO\" > example.txt",
    ]
  }

  post-processor "docker-tag" {
    repository = "learn-packer"
    tags       = ["ubuntu-xenial"]
  }
}

Build modified Packer image

Build your image.

$ packer build .
docker.ubuntu: output will be in this color.

==> docker.ubuntu: Creating a temporary directory for sharing data...
==> docker.ubuntu: Pulling Docker image: ubuntu:xenial
    docker.ubuntu: xenial: Pulling from library/ubuntu
    docker.ubuntu: Digest: sha256:eed7e1076bbc1f342c4474c718e5438af4784f59a4e88ad687dbb98483b59ee4
    docker.ubuntu: Status: Image is up to date for ubuntu:xenial
    docker.ubuntu: docker.io/library/ubuntu:xenial
==> docker.ubuntu: Starting docker container...
    docker.ubuntu: Run command: docker run -v /Users/youruser/.packer.d/tmp051358398:/packer-files -d -i -t --entrypoint=/bin/sh -- ubuntu:xenial
    docker.ubuntu: Container ID: e2e739467b24f21d113d37cf1dd709a8038e0f891cc6d34f56e50e229010d9e2
==> docker.ubuntu: Using docker communicator to connect: 172.17.0.3
==> docker.ubuntu: Provisioning with shell script: /var/folders/s6/m22_k3p11z104k2vx1jkqr2c0000gp/T/packer-shell915176051
    docker.ubuntu: Hello world!
==> docker.ubuntu: Provisioning with shell script: /var/folders/s6/m22_k3p11z104k2vx1jkqr2c0000gp/T/packer-shell946919744
    docker.ubuntu: Adding file to Docker Container
==> docker.ubuntu: Committing the container
    docker.ubuntu: Image ID: sha256:513c63d6a34abaf658faf709ad6e3bf24c07f232be1fec7e7aa2ef5a00f11921
==> docker.ubuntu: Killing the container: e2e739467b24f21d113d37cf1dd709a8038e0f891cc6d34f56e50e229010d9e2
==> docker.ubuntu: Running post-processor:  (type docker-tag)
    docker.ubuntu (docker-tag): Tagging image: sha256:513c63d6a34abaf658faf709ad6e3bf24c07f232be1fec7e7aa2ef5a00f11921
    docker.ubuntu (docker-tag): Repository: learn-packer:ubuntu-xenial
Build 'docker.ubuntu' finished after 19 seconds 647 milliseconds.

==> Wait completed after 19 seconds 648 milliseconds

==> Builds finished. The artifacts of successful builds are:
--> docker.ubuntu: Imported Docker image: sha256:513c63d6a34abaf658faf709ad6e3bf24c07f232be1fec7e7aa2ef5a00f11921
--> docker.ubuntu: Imported Docker image: learn-packer:ubuntu-xenial with tags learn-packer:ubuntu-xenial

$ packer build .
docker.ubuntu: output will be in this color.

==> docker.ubuntu: Creating a temporary directory for sharing data...
==> docker.ubuntu: Pulling Docker image: ubuntu:xenial
    docker.ubuntu: xenial: Pulling from library/ubuntu
    docker.ubuntu: Digest: sha256:eed7e1076bbc1f342c4474c718e5438af4784f59a4e88ad687dbb98483b59ee4
    docker.ubuntu: Status: Image is up to date for ubuntu:xenial
    docker.ubuntu: docker.io/library/ubuntu:xenial
==> docker.ubuntu: Starting docker container...
    docker.ubuntu: Run command: docker run -v /Users/youruser/.packer.d/tmp051358398:/packer-files -d -i -t --entrypoint=/bin/sh -- ubuntu:xenial
    docker.ubuntu: Container ID: e2e739467b24f21d113d37cf1dd709a8038e0f891cc6d34f56e50e229010d9e2
==> docker.ubuntu: Using docker communicator to connect: 172.17.0.3
==> docker.ubuntu: Provisioning with shell script: /var/folders/s6/m22_k3p11z104k2vx1jkqr2c0000gp/T/packer-shell915176051
    docker.ubuntu: Hello world!
==> docker.ubuntu: Provisioning with shell script: /var/folders/s6/m22_k3p11z104k2vx1jkqr2c0000gp/T/packer-shell946919744
    docker.ubuntu: Adding file to Docker Container
==> docker.ubuntu: Committing the container
    docker.ubuntu: Image ID: sha256:513c63d6a34abaf658faf709ad6e3bf24c07f232be1fec7e7aa2ef5a00f11921
==> docker.ubuntu: Killing the container: e2e739467b24f21d113d37cf1dd709a8038e0f891cc6d34f56e50e229010d9e2
==> docker.ubuntu: Running post-processor:  (type docker-tag)
    docker.ubuntu (docker-tag): Tagging image: sha256:513c63d6a34abaf658faf709ad6e3bf24c07f232be1fec7e7aa2ef5a00f11921
    docker.ubuntu (docker-tag): Repository: learn-packer:ubuntu-xenial
Build 'docker.ubuntu' finished after 19 seconds 647 milliseconds.

==> Wait completed after 19 seconds 648 milliseconds

==> Builds finished. The artifacts of successful builds are:
--> docker.ubuntu: Imported Docker image: sha256:513c63d6a34abaf658faf709ad6e3bf24c07f232be1fec7e7aa2ef5a00f11921
--> docker.ubuntu: Imported Docker image: learn-packer:ubuntu-xenial with tags learn-packer:ubuntu-xenial

In the output, you will find Adding file to Docker Container that confirms that the Packer ran the second provision block.

==> docker.ubuntu: Provisioning with shell script: /var/folders/s6/m22_k3p11z104k2vx1jkqr2c0000gp/T/packer-shell946919744
    docker.ubuntu: Adding file to Docker Container

==> docker.ubuntu: Provisioning with shell script: /var/folders/s6/m22_k3p11z104k2vx1jkqr2c0000gp/T/packer-shell946919744
    docker.ubuntu: Adding file to Docker Container

Verify Docker image

List the images to verify Packer successfully tagged the image.

$ docker images learn-packer
REPOSITORY     TAG                          IMAGE ID       CREATED         SIZE
learn-packer   ubuntu-xenial                513c63d6a34a   2 minutes ago   134MB

$ docker images learn-packer
REPOSITORY     TAG                          IMAGE ID       CREATED         SIZE
learn-packer   ubuntu-xenial                513c63d6a34a   2 minutes ago   134MB

To verify the provisioner created the example.txt file in the new image, first launch the newly created Docker image. Replace IMAGE_ID with the value shown above in your terminal (your local equivalent of 513c63d6a34a as shown in the output above).

$ docker run -it IMAGE_ID

$ docker run -it IMAGE_ID

In the Docker container shell, print the contents of the example.txt file. This should return FOO is hello world as expected.

$ cat example.txt
FOO is hello world

$ cat example.txt
FOO is hello world

Type exit to leave the container.

$ exit

$ exit

Next steps

In this tutorial, you upgraded a Packer JSON template to HCL2. Then, you added provisioners and post processors to the upgraded HCL2 template.

Refer to the following resources for additional details on the concepts covered in this tutorial:

Read more about the hcl2_upgrade command and Packer HCL2 syntax.
Read more about the init command.
Learn how to provision infrastructure with Packer and Terraform.
Read more about why Packer switched from JSON to HCL2 in HashiCorp Packer with HCL Configs blog post.

Back to Collection

Next Collection