Provision Infrastructure with Packer

Packer is HashiCorp's open-source tool for creating machine images from source configuration. You can configure Packer images with an operating system and software for your specific use-case.

Terraform configuration for a compute instance can use a Packer image to provision your instance without manual configuration.

In this tutorial, you will create a Packer image with a user group, a new user with authorized SSH keys, and a Go web app. Then, you will deploy this image using Terraform. Finally, you will access the instance via SSH to deploy the Go web app.

Prerequisites

To follow along with this tutorial, you will need:

• Packer 1.6.6 or later
• Terraform
• An AWS account
• Local environment variables for your AWS account.

Clone the example repository here.

$ git clone -b packer https://github.com/hashicorp/learn-terraform-provisioning

$ git clone -b packer https://github.com/hashicorp/learn-terraform-provisioning

Change into your cloned repo directory.

$ cd learn-terraform-provisioning

$ cd learn-terraform-provisioning

Create a local SSH key

For this tutorial, create a local SSH key to pair with the new terraform user you create on this instance.

$ ssh-keygen -t rsa -C "your_email@example.com" -f ./tf-packer

$ ssh-keygen -t rsa -C "your_email@example.com" -f ./tf-packer

Review the shell script

Packer's configuration will pass it a shell script to run when it builds the image. For more information on the other methods of delivering provisioning instructions to your image, visit the Packer provisioners documentation.

The script for this tutorial updates the default instance software, installs necessary apps, and creates a user with your SSH key created above.

Change directories into the scripts directory.

$ cd scripts

$ cd scripts

Open setup.sh in your file editor and review the provisioning instructions. This script installs the necessary dependencies, adds the terraform user to the sudo group, installs the previously created SSH key, and downloads the sample GoLang webapp. Use the comments in setup.sh to verify these steps before building the image.

#!/bin/bash
set -e

# Install necessary dependencies
sudo DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" dist-upgrade
sudo apt-get -y -qq install curl wget git vim apt-transport-https ca-certificates
sudo add-apt-repository ppa:longsleep/golang-backports -y
sudo apt -y -qq install golang-go

# Setup sudo to allow no-password sudo for "hashicorp" group and adding "terraform" user
sudo groupadd -r hashicorp
sudo useradd -m -s /bin/bash terraform
sudo usermod -a -G hashicorp terraform
sudo cp /etc/sudoers /etc/sudoers.orig
echo "terraform ALL=(ALL) NOPASSWD:ALL" | sudo tee /etc/sudoers.d/terraform

# Installing SSH key
sudo mkdir -p /home/terraform/.ssh
sudo chmod 700 /home/terraform/.ssh
sudo cp /tmp/tf-packer.pub /home/terraform/.ssh/authorized_keys
sudo chmod 600 /home/terraform/.ssh/authorized_keys
sudo chown -R terraform /home/terraform/.ssh
sudo usermod --shell /bin/bash terraform

# Create GOPATH for Terraform user & download the webapp from GitHub

sudo -H -i -u terraform -- env bash << EOF
whoami
echo ~terraform

cd /home/terraform

export GOROOT=/usr/lib/go
export GOPATH=/home/terraform/go
export PATH=$PATH:$GOROOT/bin:$GOPATH/bin
go get -d github.com/hashicorp/learn-go-webapp-demo
EOF

#!/bin/bash
set -e

# Install necessary dependencies
sudo DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" dist-upgrade
sudo apt-get -y -qq install curl wget git vim apt-transport-https ca-certificates
sudo add-apt-repository ppa:longsleep/golang-backports -y
sudo apt -y -qq install golang-go

# Setup sudo to allow no-password sudo for "hashicorp" group and adding "terraform" user
sudo groupadd -r hashicorp
sudo useradd -m -s /bin/bash terraform
sudo usermod -a -G hashicorp terraform
sudo cp /etc/sudoers /etc/sudoers.orig
echo "terraform ALL=(ALL) NOPASSWD:ALL" | sudo tee /etc/sudoers.d/terraform

# Installing SSH key
sudo mkdir -p /home/terraform/.ssh
sudo chmod 700 /home/terraform/.ssh
sudo cp /tmp/tf-packer.pub /home/terraform/.ssh/authorized_keys
sudo chmod 600 /home/terraform/.ssh/authorized_keys
sudo chown -R terraform /home/terraform/.ssh
sudo usermod --shell /bin/bash terraform

# Create GOPATH for Terraform user & download the webapp from GitHub

sudo -H -i -u terraform -- env bash << EOF
whoami
echo ~terraform

cd /home/terraform

export GOROOT=/usr/lib/go
export GOPATH=/home/terraform/go
export PATH=$PATH:$GOROOT/bin:$GOPATH/bin
go get -d github.com/hashicorp/learn-go-webapp-demo
EOF

Review the Packer image

Your Packer configuration defines the parameters of the image you want to build.

Change directories into images.

$ cd ../images

$ cd ../images

Open the image.pkr.hcl file in your file editor.

Review the variables block. This region must match the region where Terraform will build your AMI. If you customize it, you will need to customize the sample Terraform configuration to match (later on in the tutorial). The locals block creates a formatted timestamp to keep your AMI name unique.

variable "region" {
  type    = string
  default = "us-east-1"
}

locals { timestamp = regex_replace(timestamp(), "[- TZ:]", "") }

variable "region" {
  type    = string
  default = "us-east-1"
}

locals { timestamp = regex_replace(timestamp(), "[- TZ:]", "") }

The source block generates a template for your AMI. The source amazon-ebs declares this image will be created in AWS and uses Elastic Block Storage. This ami_name names the AMI learn-terraform-packer and searches for a base AMI in the source_ami_filter that matches your criteria of a t2.micro Ubuntu image with Elastic Block Storage (EBS) in your declared region.

source "amazon-ebs" "example" {
  ami_name      = "learn-terraform-packer-${local.timestamp}"
  instance_type = "t2.micro"
  region        = var.region
  source_ami_filter {
    filters = {
      name                = "ubuntu/images/*ubuntu-xenial-16.04-amd64-server-*"
      root-device-type    = "ebs"
      virtualization-type = "hvm"
    }
    most_recent = true
    owners      = ["099720109477"]
  }
  ssh_username = "ubuntu"
}

source "amazon-ebs" "example" {
  ami_name      = "learn-terraform-packer-${local.timestamp}"
  instance_type = "t2.micro"
  region        = var.region
  source_ami_filter {
    filters = {
      name                = "ubuntu/images/*ubuntu-xenial-16.04-amd64-server-*"
      root-device-type    = "ebs"
      virtualization-type = "hvm"
    }
    most_recent = true
    owners      = ["099720109477"]
  }
  ssh_username = "ubuntu"
}

Finally, the build block builds out your instances with specific scripts or files. Your build is based on the previously declared source as the type of AMI.

Next, the provisioner blocks copy your key to the image and run your setup script.

build {
  sources = ["source.amazon-ebs.example"]

  provisioner "file" {
    source      = "../tf-packer.pub"
    destination = "/tmp/tf-packer.pub"
  }
  provisioner "shell" {
    script = "../scripts/setup.sh"
  }
}

build {
  sources = ["source.amazon-ebs.example"]

  provisioner "file" {
    source      = "../tf-packer.pub"
    destination = "/tmp/tf-packer.pub"
  }
  provisioner "shell" {
    script = "../scripts/setup.sh"
  }
}

Build your Packer image

Run the Packer build command providing your image template file.

$ packer build image.pkr.hcl
amazon-ebs.example: output will be in this color.

==> amazon-ebs.example: Prevalidating any provided VPC information
==> amazon-ebs.example: Prevalidating AMI Name: learn-terraform-packer
    amazon-ebs.example: Found Image ID: ami-07f5c641c23596eb9
==> amazon-ebs.example: Creating temporary keypair: packer_601c660b-e33f-7875-ab03-ae9a42345198
==> amazon-ebs.example: Creating temporary security group for this instance: packer_601c660f-5a26-53d2-0321-7094c5044848
==> amazon-ebs.example: Authorizing access to port 22 from [0.0.0.0/0] in the temporary security groups...
==> amazon-ebs.example: Launching a source AWS instance...
## ...
==> amazon-ebs.example: Stopping the source instance...
    amazon-ebs.example: Stopping instance
==> amazon-ebs.example: Waiting for the instance to stop...
==> amazon-ebs.example: Creating AMI learn-terraform-packer from instance i-0054c6c995aacf330
    amazon-ebs.example: AMI: ami-0dbaca5d269497603
==> amazon-ebs.example: Waiting for AMI to become ready...
==> amazon-ebs.example: Terminating the source AWS instance...
==> amazon-ebs.example: Cleaning up any extra volumes...
==> amazon-ebs.example: No volumes to clean up, skipping
==> amazon-ebs.example: Deleting temporary security group...
==> amazon-ebs.example: Deleting temporary keypair...
Build 'amazon-ebs.example' finished after 5 minutes 42 seconds.

==> Wait completed after 5 minutes 42 seconds

==> Builds finished. The artifacts of successful builds are:
--> amazon-ebs.example: AMIs were created:
us-east-1: ami-0dbaca5d269497603

$ packer build image.pkr.hcl
amazon-ebs.example: output will be in this color.

==> amazon-ebs.example: Prevalidating any provided VPC information
==> amazon-ebs.example: Prevalidating AMI Name: learn-terraform-packer
    amazon-ebs.example: Found Image ID: ami-07f5c641c23596eb9
==> amazon-ebs.example: Creating temporary keypair: packer_601c660b-e33f-7875-ab03-ae9a42345198
==> amazon-ebs.example: Creating temporary security group for this instance: packer_601c660f-5a26-53d2-0321-7094c5044848
==> amazon-ebs.example: Authorizing access to port 22 from [0.0.0.0/0] in the temporary security groups...
==> amazon-ebs.example: Launching a source AWS instance...
## ...
==> amazon-ebs.example: Stopping the source instance...
    amazon-ebs.example: Stopping instance
==> amazon-ebs.example: Waiting for the instance to stop...
==> amazon-ebs.example: Creating AMI learn-terraform-packer from instance i-0054c6c995aacf330
    amazon-ebs.example: AMI: ami-0dbaca5d269497603
==> amazon-ebs.example: Waiting for AMI to become ready...
==> amazon-ebs.example: Terminating the source AWS instance...
==> amazon-ebs.example: Cleaning up any extra volumes...
==> amazon-ebs.example: No volumes to clean up, skipping
==> amazon-ebs.example: Deleting temporary security group...
==> amazon-ebs.example: Deleting temporary keypair...
Build 'amazon-ebs.example' finished after 5 minutes 42 seconds.

==> Wait completed after 5 minutes 42 seconds

==> Builds finished. The artifacts of successful builds are:
--> amazon-ebs.example: AMIs were created:
us-east-1: ami-0dbaca5d269497603

The final line of the output is the AMI ID you will pass into your Terraform configuration in the next step.

Deploy your Packer image with Terraform

The AMI is your artifact from the Packer run and is available in your AWS account in the EC2 Images section. You can visit the AWS Web Console to view this AMI ID again.

To use this AMI in your Terraform environment, navigate to the instances directory.

$ cd ../instances

$ cd ../instances

Open the main.tf file and navigate to the aws_instance resource. Edit the ami attribute with the AMI ID you received from your Packer build.

##...
resource "aws_instance" "web" {
-  ami                         = "ami-YOUR-AMI-ID"
+  ami                         = "ami-0dbaca5d269497603"
  instance_type               = "t2.micro"
  subnet_id                   = aws_subnet.subnet_public.id
  vpc_security_group_ids      = [aws_security_group.sg_22_80.id]
  associate_public_ip_address = true

  tags = {
    Name = "Learn-Packer"
  }
}
##...

##...
resource "aws_instance" "web" {
-  ami                         = "ami-YOUR-AMI-ID"
+  ami                         = "ami-0dbaca5d269497603"
  instance_type               = "t2.micro"
  subnet_id                   = aws_subnet.subnet_public.id
  vpc_security_group_ids      = [aws_security_group.sg_22_80.id]
  associate_public_ip_address = true

  tags = {
    Name = "Learn-Packer"
  }
}
##...

Save your configuration.

Create a new file called terraform.tfvars and add the Packer image's region as the variable definition. If you customized the region you gave to Packer you must change this region to match, or Terraform won't be able to access your image.

region = "us-east-1"

region = "us-east-1"

Save this file and then initialize and apply your configuration.

$ terraform init && terraform apply

$ terraform init && terraform apply

Type yes when prompted to create your instance. Your final output is your instance IP address. In the next section, you will SSH into this instance with your local key.

Your instance in this tutorial already contains the preferred SSH key because it uses the AMI you previously packaged with Packer. Using a Packer-packaged AMI makes deploying mass instances faster and more consistent than configuring the instances manually.

Verify your instance

Connect to your instance via SSH.

$ ssh terraform@$(terraform output -raw public_ip) -i ../tf-packer

$ ssh terraform@$(terraform output -raw public_ip) -i ../tf-packer

Now you have SSH access to your AWS instances without creating an SSH key in AWS. This is useful if your organization maintains keypairs outside of AWS.

Navigate to the Go directory.

$ cd go/src/github.com/hashicorp/learn-go-webapp-demo

$ cd go/src/github.com/hashicorp/learn-go-webapp-demo

Launch the demo webapp.

$ go run webapp.go

$ go run webapp.go

In your web browser, navigate your instance's IP address and port 8080 to see the app you deployed.

Destroy your instance

Avoid unnecessary charges in your AWS account by destroying your instance in Terraform.

$ terraform destroy

$ terraform destroy

Type yes when you are prompted in your terminal to delete your infrastructure.

This will not destroy your Packer image. Your Packer image will not incur costs in your AWS account. Most base Linux distributions have free image versions, but be sure to check on the cost of deploying and maintaining your images.

Next Steps

In this tutorial, you created a Packer image with your desired configuration and deployed it using Terraform.

• To learn about creating deployment scripts for Terraform, visit the cloud-init tutorial.
• For more information about creating images with Packer, visit the Packer tutorials.