Migrate CloudFormation templates to Terraform configurations

Author: Vijay Sharma

The purpose of this document is to outline patterns for migrating CloudFormation templates and AWS resources to Terraform. Mapping CloudFormation resources to Terraform requires manual effort due to the differences in design and usage between the two tools. CloudFormation is an AWS-specific UI tool, while Terraform is a CLI-based tool focused on automation.

Understand the differences

It is important for enterprises migrating from AWS CloudFormation to HCP Terraform to understand the differences between these two infrastructure as code offerings.

• Language: CloudFormation uses JSON or YAML to define infrastructure as code, while Terraform utilizes HCL (HashiCorp Configuration Language).
• State Management: Terraform maintains a state file to track infrastructure resources, enabling better management and collaboration. HCP Terraform simplifies this process by automatically handling state management, offering a secure and scalable solution for teams. AWS CloudFormation manages the state of your AWS infrastructure using a concept called stacks.
• Providers: Terraform supports a wide range of providers, not just AWS. This includes Google Cloud, Azure, and many others. This extensive provider support makes Terraform a versatile tool for multi-cloud and hybrid-cloud deployments. AWS CloudFormation is designed only to support AWS cloud infrastructure deployment.
• Modules: Terraform modules provide a way to create reusable and composable infrastructure components, promoting consistency and reducing duplication in your infrastructure code.

Please find more details on adopting Terraform in the HashiCorp Validated Design reference mentioned in the Related Resources section.

Target audience

The primary audience for this document is:

• Cloud architects and platform teams
• DevOps and infrastructure engineers

Prerequisites

To complete this guide, you will need the following:

• Review Terraform Solution Design Guide and Operating Guides.
• An active AWS account.
• A GitHub, BitBucket, or other source repository.
• An HCP Terraform account.

Background and best practices

Migrating from AWS CloudFormation to HCP Terraform involves a systematic approach to convert existing infrastructure as code (IaC) setups. The goal is to maintain or improve the current infrastructure state while leveraging the capabilities of Terraform for enhanced flexibility, provider support, and ecosystem integration.

This approach can be divided into three primary use cases.

1. Migrating existing CloudFormation templates into Terraform resources: This involves translating CloudFormation templates directly into Terraform configurations.
2. Executing complex CloudFormation runs using Terraform: This can be achieved by utilizing Terraform to run existing CloudFormation templates, which is especially useful for complex scenarios where existing CloudFormation scripts need to be maintained initially.
3. Migrating existing resources created Using CloudFormation into HCP Terraform using the Terraform import workflow: This method involves importing existing infrastructure resources into the Terraform state, allowing for seamless management and further modification using Terraform.

There are five key phases of migration.

1. Assessment and planning
   • Inventory current infrastructure: Document all resources managed by CloudFormation.
   • Identify dependencies: Map out resource dependencies and relationships.
   • Evaluate complexity: Categorize resources based on complexity (simple, medium, complex).
   • Set objectives: Define goals for the migration, such as improved maintainability, reduced deployment time, or better provider support.
2. Preparation
   • Skill building: Ensure the team knows Terraform, syntax, and best practices.
   • Tooling: Set up necessary tools, including Terraform CLI, AWS CLI, and version control systems like Git.
   • Environment setup: Prepare staging or a sandbox environment to test the migration process.
   • Set up an HCP Terraform account to take advantage of Terraform's advanced state management capabilities. Contact your account team for a trial account if needed.
3. Conversion
   • Resource mapping: Ensure each property in CloudFormation is mapped to a corresponding Terraform resource.
   • Referencing resources: Use Terraform resource references (e.g. aws_vpc.my_vpc.id) instead of CloudFormation intrinsic functions (e.g., !Ref).
   • Modularization: Break down infrastructure into reusable Terraform modules. Please reference the HashiCorp Validated Design Operating Guide for Adoption under the sections Consumption Model and Terraform Workflow for more details and examples.
   • Incremental conversion: Start with less critical resources to build confidence and refine processes.
   • Configuration management: It is important to manage state files securely, consider using HCP Terraform to securely manage state files.
4. Validation and testing
   • Static analysis: Use tools like terraform validate and terraform fmt to ensure code quality.
   • Plan and apply: Use terraform plan to preview changes and terraform apply to implement them.
   • Functional testing: Validate that resources are created and configured correctly.
     • Reference:
       • Testing HashiCorp Terraform
       • Terraform Tests
   • Rollback strategy: Plan for rollbacks in case of issues, keeping CloudFormation stacks as a fallback.
5. Deployment
   • Phased rollout: Deploy changes incrementally to reduce risk.
   • Monitoring and logging: Set up monitoring and logging to detect and troubleshoot issues.
   • Documentation: Document the new infrastructure setup and train relevant teams.

For Terraform best practices and implementation, we recommend you use HashiCorp Validated Designs. This resource provides comprehensive guides and examples for designing and implementing Terraform configurations effectively.

This document does not cover CloudFormation technical terms, but familiarity with CloudFormation Stacks, StackSets, and the export feature is helpful. Understanding these concepts will clarify the differences between CloudFormation and Terraform in managing infrastructure.

Migration strategy

• Incremental migration: Migrate resources incrementally rather than all at once. This reduces risk and allows for easier troubleshooting and rollback if issues arise.
• Parallel stacks: Run CloudFormation and Terraform in parallel during the transition period to validate that Terraform correctly manages the resources before fully decommissioning CloudFormation.

The following are CloudFormation to Terraform migration examples.

Simple EC2 instance migration

It is important to understand that CloudFormation YAML and Terraform are two distinct languages for infrastructure provisioning. CloudFormation is designed specifically from an AWS perspective, whereas Terraform is declarative and supports multiple providers. In this example, you can look at the basics of an EC2 instance configuration using CloudFormation YAML and compare it with how it would look in Terraform HCL.

Resources:
  MyEC2Instance:
    Type: AWS::EC2::Instance
    Properties:
      InstanceType: t2.micro
      ImageId: ami-0862be96e41dcbf74
      KeyName: ubuntu

provider "aws" {
  region = "us-east-2" # Specify your AWS region here
}

resource "aws_instance" "my_ec2_instance" {
  instance_type = "t2.micro"
  ami           = "ami-0862be96e41dcbf74"
  key_name      = "ubuntu"

  tags = {
    Name = "MyEC2Instance"
  }
}

Complex VPC migration

This is another example that demonstrates a VPC configuration in CloudFormation YAML and how you can define the same configuration using Terraform HCL.

Resources:
  MyVPC:
    Type: "AWS::EC2::VPC"
    Properties:
      CidrBlock: "10.0.0.0/16"
      Tags:
        - Key: "Name"
          Value: "demo-vpc"

  MySubnet:
    Type: "AWS::EC2::Subnet"
    Properties:
      VpcId: !Ref MyVPC
      CidrBlock: "10.0.1.0/24"
      Tags:
        - Key: "Name"
          Value: "demo-subnet1"

provider "aws" {
  region = "us-east-2"
}

resource "aws_vpc" "my_vpc" {
  cidr_block = "10.0.0.0/16"

  tags = {
    Name = "demo-vpc"
  }
}

resource "aws_subnet" "my_subnet" {
  vpc_id     = aws_vpc.my_vpc.id
  cidr_block = "10.0.1.0/24"

  tags = {
    Name = "demo-subnet1"
  }
}

IAM role migration

This example shows an IAM role migration from CloudFormation YAML to Terraform HCL. In Terraform, the actual IAM policy is JSON-encoded.

Resources:
  MyIAMRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Principal:
              Service: ec2.amazonaws.com
            Action: sts:AssumeRole

resource "aws_iam_role" "example" {
  name = "example-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Principal = {
          Service = "ec2.amazonaws.com"
        }
        Action = "sts:AssumeRole"
      }
    ]
  })
}

DynamoDB migration

This example shows the migration of an AWS DynamoDB table from a CloudFormation YAML template to a Terraform HCL configuration. Both configurations define a DynamoDB table with similar attributes, key schema, provisioned throughput, global secondary indexes, and local secondary indexes. The Terraform configuration maps each element from the CloudFormation template, ensuring a smooth migration.

AWSTemplateFormatVersion: "2010-09-09"
Resources: 
  myDynamoDBTable: 
    Type: AWS::DynamoDB::Table
    Properties: 
      AttributeDefinitions: 
        - 
          AttributeName: "Album"
          AttributeType: "S"
        - 
          AttributeName: "Artist"
          AttributeType: "S"
        - 
          AttributeName: "Sales"
          AttributeType: "N"
        - 
          AttributeName: "NumberOfSongs"
          AttributeType: "N"
      KeySchema: 
        - 
          AttributeName: "Album"
          KeyType: "HASH"
        - 
          AttributeName: "Artist"
          KeyType: "RANGE"
      ProvisionedThroughput: 
        ReadCapacityUnits: "5"
        WriteCapacityUnits: "5"
      TableName: "myTableName"
      GlobalSecondaryIndexes: 
        - 
          IndexName: "myGSI"
          KeySchema: 
            - 
              AttributeName: "Sales"
              KeyType: "HASH"
            - 
              AttributeName: "Artist"
              KeyType: "RANGE"
          Projection: 
            NonKeyAttributes: 
              - "Album"
              - "NumberOfSongs"
            ProjectionType: "INCLUDE"
          ProvisionedThroughput: 
            ReadCapacityUnits: "5"
            WriteCapacityUnits: "5"
        - 
          IndexName: "myGSI2"
          KeySchema: 
            - 
              AttributeName: "NumberOfSongs"
              KeyType: "HASH"
            - 
              AttributeName: "Sales"
              KeyType: "RANGE"
          Projection: 
            NonKeyAttributes: 
              - "Album"
              - "Artist"
            ProjectionType: "INCLUDE"
          ProvisionedThroughput: 
            ReadCapacityUnits: "5"
            WriteCapacityUnits: "5"
      LocalSecondaryIndexes: 
        - 
          IndexName: "myLSI"
          KeySchema: 
            - 
              AttributeName: "Album"
              KeyType: "HASH"
            - 
              AttributeName: "Sales"
              KeyType: "RANGE"
          Projection: 
            NonKeyAttributes: 
              - "Artist"
              - "NumberOfSongs"
            ProjectionType: "INCLUDE"

provider "aws" {
  region = "us-east-2" # Specify your AWS region here
}

resource "aws_dynamodb_table" "my_dynamodb_table" {
  name         = "myTableName"
  billing_mode = "PROVISIONED"

  read_capacity  = 5
  write_capacity = 5

  hash_key  = "Album"
  range_key = "Artist"

  attribute {
    name = "Album"
    type = "S"
  }

  attribute {
    name = "Artist"
    type = "S"
  }

  attribute {
    name = "Sales"
    type = "N"
  }

  attribute {
    name = "NumberOfSongs"
    type = "N"
  }

  global_secondary_index {
    name            = "myGSI"
    hash_key        = "Sales"
    range_key       = "Artist"
    read_capacity   = 5
    write_capacity  = 5
    projection_type = "INCLUDE"
    non_key_attributes = [
      "Album",
      "NumberOfSongs"
    ]
  }

  global_secondary_index {
    name            = "myGSI2"
    hash_key        = "NumberOfSongs"
    range_key       = "Sales"
    read_capacity   = 5
    write_capacity  = 5
    projection_type = "INCLUDE"
    non_key_attributes = [
      "Album",
      "Artist"
    ]
  }

  local_secondary_index {
    name            = "myLSI"
    range_key       = "Sales"
    projection_type = "INCLUDE"
    non_key_attributes = [
      "Artist",
      "NumberOfSongs"
    ]
  }

  tags = {
    Name = "myDynamoDBTable"
  }
}

Use Terraform to execute existing CloudFormation templates

Terraform can be used to execute existing CloudFormation runs. This is useful when you have completed runbooks in CloudFormation and are seeking an easier way to migrate existing CloudFormation scripts to HCP Terraform. This approach allows for a smoother transition while maintaining the integrity of your current infrastructure setup.

Execute CloudFormation templates stored in S3

Suppose you have a CloudFormation template stored in an S3 bucket. You can use the aws_cloudformation_stack resource in Terraform to deploy the template.

provider "aws" {
  region = "us-west-2"
}

resource "aws_cloudformation_stack" "example" {
  name          = "example-stack"
  template_url  = "https://my-bucket.s3.amazonaws.com/templates/my-template.yaml"

  parameters = {
    ParameterKey = "ParameterValue"
  }
  
  tags = {
    Name = "example-stack"
  }
}

Execute CloudFormation templates stored in GitHub

provider "aws" {
  region = "us-west-2"
}

resource "aws_cloudformation_stack" "example" {
  name          = "example-stack"
  template_url  = "https://raw.githubusercontent.com/my-repo/cloudformation-templates/main/templates/my-template.yaml"

  parameters = {
    ParameterKey = "ParameterValue"
  }
  
  tags = {
    Name = "example-stack"
  }
}

Using Terraform to manage and run your current CloudFormation templates makes it easy to switch to Terraform while keeping your existing infrastructure setup. This approach helps you stay consistent and efficient during the move to HCP Terraform.

Migrate CloudFormation resources using Terraform import workflow

Migrating CloudFormation resources into Terraform using the Terraform config-driven import workflow is similar to importing any AWS resource into Terraform. Terraform stores all information in the state file and checks the configuration against this state file to determine any necessary changes to the infrastructure.

Our recommended approach is to leverage Terraform's terraform import functionality to facilitate a seamless transition. By using terraform import, existing AWS resources defined in CloudFormation can be efficiently brought under Terraform management without disrupting current operations.

The migration process begins with the identification of the AWS resources managed by CloudFormation. Each resource is then imported into the corresponding Terraform state using the terraform import command. This method preserves the existing infrastructure while allowing it to be managed through Terraform's robust and flexible tooling.

For a more streamlined approach, the config-driven workflow is recommended, as it simplifies the migration by using a Terraform configuration block that clearly defines resource imports in a declarative manner. This workflow reduces manual steps and ensures a clear mapping between existing resources and their new Terraform-managed resources.

Key benefits of using terraform import include:

• Minimal disruption: The existing infrastructure remains intact during the migration process, avoiding downtime.
• State preservation: Terraform accurately reflects the current state of resources, providing a reliable foundation for further changes.
• Flexibility: Once migrated, the infrastructure can be easily extended and managed using Terraform's extensive features.

Conclusion

To successfully migrate from CloudFormation to Terraform, organizations must adopt a methodical approach that prioritizes planning, rigorous testing, and comprehensive documentation. By rigorously adhering to best practices and leveraging Terraform's strengths, organizations will ensure a seamless transition and gain significantly enhanced infrastructure management capabilities.

When migrating from CloudFormation to Terraform, there are several tools available that can assist with the process. While none may fit every specific requirement, they can be very helpful:

• Terraformer: A CLI tool that generates Terraform configuration files and state from existing infrastructure (e.g., AWS, GCP, etc.). It can be found at Terraformer on GitHub.
• cf-to-tf: A tool designed to convert CloudFormation templates to Terraform configuration. It can be accessed at cf-to-tf on GitHub.
• Terraform import config block
• Terraform Config-driven imports and checks
• HVD: Terraform Import
• Import CloudFormation Resources into Terraform
• Migrate configuration to HCP Terraform.
• Manage resources in Terraform state