Manage Terraform provider upgrades

Author: Craig Sloggett

This guide provides a step-by-step workflow for upgrading Terraform provider versions, applicable to both HashiCorp Cloud Platform (HCP) and the Community Edition.

Upgrading a provider often presents challenges, such as maintaining infrastructure integrity and addressing unforeseen bugs. This workflow ensures a smooth and confident upgrade process by walking you through a detailed example involving an AWS provider version upgrade.

Upgrading the Terraform provider can provide several significant benefits:

• Enable support for new cloud provider features.
• Ensure infrastructure security with the latest patches.
• Improve deployment performance with optimizations.
• Resolve bugs and issues present in older provider versions.
• Benefit from the latest community knowledge, documentation, and support resources.
• Prevent the accumulation of technical debt with regular upgrades.

Understanding this workflow will equip you with the skills needed to handle any scenario during the provider upgrade process.

Target audience

This guide references the following roles:

• Platform operators
• Terraform producers and consumers

The workflow outlined in this document is useful to anyone required to deploy infrastructure in a safe and predictable way. Even with an automated pipeline, there will be circumstances where you have to refactor your code to support provider changes.

Prerequisites

To complete this workflow, you will need the following:

• Terraform has been installed and configured on your machine with the ability to run a plan against the code base being upgraded.
• Push access to a git repository containing Terraform configuration files.
• A GitHub account to receive notifications for new provider versions.
• Access to deploy an S3 bucket to AWS using their API.

In addition, when using either HCP Terraform or Terraform Enterprise, we recommend organizing your infrastructure resources into organizations, projects, and workspaces, with access managed through user accounts, teams, and permissions. Planning how to organize these objects are outlined in the Configuring for First Use section of the Terraform: Operating Guide for Adoption in detail.

This pattern assumes you understand the core Terraform workflow and use git for version control in a pull merge request workflow.

The term platform used throughout this document refers to a standardized set of tools, processes, and services to support application development and deployment across various environments.

Before starting this guide, please review the following resources:

• Build in AWS with Terraform
• Git Tutorial
• Configuring Notifications in GitHub
• How to Read a git diff
• Speculative Plans
• What Is a Platform Team and What Problems Do They Solve?

Background and best practices

The following are best practices for upgrading the Terraform provider:

• Upgrade providers often, ideally as they are published.
• Isolate provider upgrades into their own separate git branch and pull request.
• When tackling issues during an upgrade, refactor for one issue at a time.
• Continuously verify (small) changes by running a plan.
• Only run terraform apply once there are no errors, warnings, or actions in a plan.
• Ensure the only change is the provider version configuration when assessing the initial plan output.

Validated architecture

The main components of this workflow are:

• Monitoring for a new provider version.
• Updating Terraform configuration to use the new provider version.
• Verifying there are no changes using the plan output.
• Refactoring Terraform configuration.
• Verifying your deployment after upgrading.

When upgrading the Terraform provider in your code base, you may have to make changes to your configuration, depending on the output of a speculative plan. This guide outlines both the overall process to upgrade a Terraform provider, and a detailed refactoring workflow.

The following is an overview of the upgrade process:

The following is an overview of the refactoring process:

1## People and process considerations

The Terraform: Operating Guide for Adoption has extensive discussion on the people and process recommendations for platform operations. 

Anyone required to deploy infrastructure in a safe and predictable way will benefit from this guide. With that in mind, there is a slight difference in responsibilities between the platform team and those consuming the platform.

At a high level, the platform team is responsible for monitoring and publishing new versions, but it's up to the consumers to implement change.

Platform operators

Platform operators are responsible for:

• Staying informed about new releases of the Terraform providers in use on the platform.
• Testing new versions in a staging environment before deployment to production.
• Ensuring all relevant documentation is updated to reflect changes in the new versions.
• Communicating upcoming upgrades and potential impacts to all stakeholders.
• Providing training and support to ensure smooth adoption of new versions.

Terraform producers and consumers

Terraform producers and consumers are responsible for:

• Staying informed about new versions available for the Terraform providers used to deploy their application infrastructure.
• Testing their own configurations and modules with the new provider versions in a staging environment.
• Providing feedback to the platform team about any issues encountered during testing.
• Updating their own documentation and workflows to align with the new versions.
• Attending training sessions and actively engaging with support resources provided by the platform team.

Workflow

The following workflow uses the example of upgrading the AWS provider from v3.74.3 to v3.75.0 where configuration within the aws_s3_bucket resource is deprecated and moved to a separate resource. The new resource is then added to the configuration to ensure the end state is the same in AWS.

There will also be an "unexpected" change introduced that showcases the discovery of a bug in the AWS provider.

Deploy stable configuration

Terraform producers and consumers are responsible for deploying the stable configuration to AWS.

The following configuration uses the v3.74.3 AWS provider to create an S3 bukcet with versioning enabled using the versioning configuration block.

terraform {
  required_version = "~> 1.5"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "3.74.3"
    }
  }
}

provider "aws" {}

resource "aws_s3_bucket" "testing" {
  bucket = “testing-provider-upgrades-bucket-00001”

  versioning {
    enabled = true
  }

  # Matching the SSE defaults to eliminate diffs in the plan output.
  server_side_encryption_configuration {
    rule {
      bucket_key_enabled = false
      apply_server_side_encryption_by_default {
        sse_algorithm = "AES256"
      }
    }
  }
}

Initialize the Terraform configuration.

$ terraform init
Initializing the backend...
Initializing provider plugins...
- Finding hashicorp/aws versions matching "3.74.3"...
- Installing hashicorp/aws v3.74.3...
- Installed hashicorp/aws v3.74.3 (signed by HashiCorp)
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Apply the Terraform configuration. Confirm the deploy by entering yes when prompted.

$ terraform apply
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_s3_bucket.testing will be created
  + resource "aws_s3_bucket" "testing" {
      + acceleration_status         = (known after apply)
      + acl                         = "private"
      + arn                         = (known after apply)
      + bucket                      = "testing-provider-upgrades-bucket-00001"
      + bucket_domain_name          = (known after apply)
      + bucket_regional_domain_name = (known after apply)
      + force_destroy               = false
      + hosted_zone_id              = (known after apply)
      + id                          = (known after apply)
      + region                      = (known after apply)
      + request_payer               = (known after apply)
      + tags_all                    = (known after apply)
      + website_domain              = (known after apply)
      + website_endpoint            = (known after apply)
  
      + server_side_encryption_configuration {
          + rule {
              + bucket_key_enabled = false
  
              + apply_server_side_encryption_by_default {
                  + sse_algorithm = "AES256"
                }
            }
        }

      + versioning {
          + enabled = true
          + mfa_delete = false
        }
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

aws_s3_bucket.testing: Creating...
aws_s3_bucket.testing: Creation complete after 2s [id=testing-provider-upgrades-bucket-00001]
  
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Monitor for new provider versions

Terraform producers and consumers are responsible for monitoring for new provider versions.

To effectively monitor for new releases of a Terraform provider, we recommend using an automation tool like Renovate or GitHub's Dependabot. These tools will automatically generate a pull request against your project to upgrade the version of dependencies in use (like a Terraform provider).

Alternatively, developers can subscribe to provider release notifications by selecting Releases under the Watch menu of the upstream GitHub repository.

Now, when a new release is published, you will be notified by email (and in your GitHub notifications inbox).

Update provider configuration

Terraform producers and consumers are responsible for updating the provider configuration to include the latest version.

Once you have been notified about a new provider version, the next step is to update the provider configuration to include the latest version. This involves modifying the version argument for the aws block within the required_providers block.

Ensure you work in a new git branch to isolate changes.

$ git branch upgrade-aws-provider
$ git push -u origin upgrade-aws-provider
$ git checkout upgrade-aws-provider

Update the version argument for the AWS provider.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "3.75.0"
    }
  }
}

We recommend only udpating the provider version to isolate the changes in the plan. Introducing the least amount of change possible ensures that any detected modifications are solely due to the provider update, enabling a clear and accurate evaluation of the impact.

Install the new provider version as configured.

$ terraform init -upgrade
# This ensures Terraform downloads the provider version specified.  
Initializing the backend...
  
Initializing provider plugins...
- Finding hashicorp/aws versions matching "3.75.0"...
- Installing hashicorp/aws v3.75.0...
- Installed hashicorp/aws v3.75.0 (signed by HashiCorp)
  
Terraform has made some changes to the provider dependency selections recorded
in the .terraform.lock.hcl file. Review those changes and commit them to your
version control system if they represent changes you intended to make.
  
Terraform has been successfully initialized!
  
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
  
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Create a plan

Terraform producers and consumers are responsible for running a plan to test changes with the updated provider version.

After updating the provider version in the Terraform configuration, the next step is to run a plan. This step allows you to preview the potential changes without actually applying them, ensuring that the new provider version does not introduce unexpected modifications to the infrastructure. The goal is to have no errors, warnings, or actions when upgrading to a newer provider version.

If you follow the advice found in the Terraform Workflows section of the Terraform: Operating Guide for Adoption and configured a VCS driven workflow, you could open a pull request right now and git push your changes to run a speculative plan.

However, since this workflow guides you (a single developer) through a provider upgrade, by rapidly iterating through the core Terraform workflow, use the CLI to create a plan.

If you run terraform plan without the -out=FILE option then it will create a plan, which is a description of the effect of the plan but without any intent to actually apply it.

Run a plan.

$ terraform plan
  
aws_s3_bucket.testing: Refreshing state... [id=testing-provider-upgrades-bucket-00001]
  
No changes. Your infrastructure matches the configuration.
  
Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.
╷
│ Warning: Argument is deprecated
│
│   with aws_s3_bucket.testing,
│   on main.tf line 1, in resource "aws_s3_bucket" "testing":
│ 1: resource "aws_s3_bucket" "testing" {
│
│ Use the aws_s3_bucket_versioning resource instead
│
│ (and 3 more similar warnings elsewhere)
╵

The first warning after updating the provider version shows that the Terraform configuration uses a deprecated argument.

Identify and document errors, warnings, or actions

Terraform producers and consumers are responsible for identifying any errors, warnings, or actions as indicated by a plan.

With the S3 bucket configuration unchanged and the AWS provider upgraded to version v3.75.0, this version of the AWS provider deprecates the versioning argument.

You can lookup the deprecation warning in the AWS provider documentation and see the following notice:

versioning - (Optional, Deprecated) Configuration of the S3 bucket versioning state. See Versioning below for details. Terraform will only perform drift detection if a configuration value is provided. Use the resource aws_s3_bucket_versioning instead.

When the speculative plan identifies changes to the infrastructure, it is essential to document these changes and prioritize them for review and implementation. Creating a ticket in the backlog helps organize and track the necessary modifications, ensuring they are addressed in a timely and systematic manner.

Clearly document the changes identified in the speculative plan. Include details such as the specific resources affected, the nature of the actions (e.g., additions, modifications, deletions), and any potential impacts on the infrastructure.

Ensure that the ticket includes all relevant information and context to facilitate understanding and resolution by the assigned engineer (including your future self).

Refactor configuration

Terraform producers and consumers are responsible for refactoring the configuration to ensure the plan reflects no changes.

If there are errors, warnings, or changes detected in the plan output, it is essential to refactor the Terraform code to align with the latest provider recommendations. We recommend using the plan to ensure the refactored configuration still behaves as expected.

You will address the versioning argument deprecation warning by refactoring the Terraform configuration to use the aws_s3_bucket_versioning resource.

Deprecated configuration

Given that the versioning block in the aws_s3_bucket resource has been deprecated in favor of the aws_s3_bucket_versioning resource, you must determine what code change will be functionally equivalent to the already deployed infrastructure in AWS.

Remove the versioning argument from the aws_s3_bucket resource and add the aws_s3_bucket_versioning resource.

resource "aws_s3_bucket" "testing" {
   bucket = "testing-provider-upgrades-bucket-00001"

   server_side_encryption_configuration {
     rule {
       bucket_key_enabled = false
     }
   }
 }

resource "aws_s3_bucket_versioning" "testing" {
   bucket = aws_s3_bucket.testing.id

   versioning_configuration {
     status = "Enabled"
   }
}

When refactoring code, make each change as small as possible in order to address one issue at a time.

Next, verify that the updated configuration works as expected by creating another plan.

$ terraform plan
aws_s3_bucket.testing: Refreshing state... [id=testing-provider-upgrades-bucket-00001]
  
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create
  
Terraform will perform the following actions:
  
  # aws_s3_bucket_versioning.testing will be created
  + resource "aws_s3_bucket_versioning" "testing" {
      + bucket = "testing-provider-upgrades-bucket-00001"
      + id = (known after apply)
  
      + versioning_configuration {
          + mfa_delete = (known after apply)
          + status     = "Enabled"
        }
    }
  
Plan: 1 to add, 0 to change, 0 to destroy.
╷
│ Warning: Argument is deprecated
│
│   with aws_s3_bucket.testing,
│   on main.tf line 1, in resource "aws_s3_bucket" "testing":
│ 1: resource "aws_s3_bucket" "testing" {
│
│ Use the aws_s3_bucket_server_side_encryption_configuration resource instead
│
│ (and one more similar warning elsewhere)
╵

There are two changes to note:

1. A new deprecation warning that suggest you use the aws_s3_bucket_server_side_encryption_configuration resource instead.
2. Terraform will create the aws_s3_bucket_versioning resource that you have added to your configuration.

To ensure there are no errors, warnings, or actions when upgrading to a newer provider version, you need to address both of these changes.

First, you can address the new deprecation warning by updating your configuration to use the aws_s3_bucket_server_side_encryption_configuration resource. This change is similar to the first deprecation warning about the versioning argument.

Following the same process you did for the versioning deprecation warning, review the AWS provider documentation to determine a functionally equivalent refactor.

Remove the server_side_encryption_configuration argument and add the aws_s3_bucket_server_side_encryption_configuration resource.

@@ -1,14 +1,5 @@
 resource "aws_s3_bucket" "testing" {
   bucket = "testing-provider-upgrades-bucket-00001"
-
-  server_side_encryption_configuration {
-    rule {
-      bucket_key_enabled = false
-      apply_server_side_encryption_by_default {
-        sse_algorithm = "AES256"
-      }
-    }
-  }
 }
 resource "aws_s3_bucket_versioning" "testing" {
   bucket = aws_s3_bucket.testing.id

   versioning_configuration {
     status = "Enabled"
   }
}
+
+resource "aws_s3_bucket_server_side_encryption_configuration" "testing" {
+  bucket = aws_s3_bucket.testing.id
+
+  rule {
+    bucket_key_enabled = false
+
+    apply_server_side_encryption_by_default {
+      sse_algorithm = "AES256"
+    }
+  }
+}

Again, check the speculative plan output to see if you have addressed the deprecation warning.

$ terraform plan
aws_s3_bucket.testing: Refreshing state... [id=testing-provider-upgrades-bucket-00001]
  
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create
  ~ update in-place
  
Terraform will perform the following actions:
  
  # aws_s3_bucket.testing will be updated in-place
  ~ resource "aws_s3_bucket" "testing" {
        id                      = "testing-provider-upgrades-bucket-00001"
        tags                    = {}
        # (12 unchanged attributes hidden)
  
      - server_side_encryption_configuration {
          - rule {
              - bucket_key_enabled = false -> null
  
              - apply_server_side_encryption_by_default {
                  - sse_algorithm = "AES256" -> null
                    # (1 unchanged attribute hidden)
                }
            }
        }
  
        # (1 unchanged block hidden)
    }
  
  # aws_s3_bucket_server_side_encryption_configuration.testing will be created
  + resource "aws_s3_bucket_server_side_encryption_configuration" "testing" {
      + bucket = "testing-provider-upgrades-bucket-00001"
      + id = (known after apply)
  
      + rule {
          + bucket_key_enabled = false
  
          + apply_server_side_encryption_by_default {
              + sse_algorithm = "AES256"
                # (1 unchanged attribute hidden)
            }
        }
    }
  
  # aws_s3_bucket_versioning.testing will be created
  + resource "aws_s3_bucket_versioning" "testing" {
      + bucket = "testing-provider-upgrades-bucket-00001"
      + id = (known after apply)
  
      + versioning_configuration {
          + mfa_delete = (known after apply)
          + status     = "Enabled"
        }
    }
  
Plan: 2 to add, 1 to change, 0 to destroy.

You have successfully removed all of the deprecation warnings!

However, you now have three changes to address:

1. The creation of the aws_s3_bucket_server_side_encryption_configuration resource that you added to your configuration.
2. The creation of the aws_s3_bucket_versioning resource that you added to your configuration.
3. The aws_s3_bucket will be updated in-place.

You have swapped a deprecation warning for an add and a change action in your speculative plan output.

While the S3 bucket has versioning enabled, it is unclear whether adding the aws_s3_bucket_versioning resource with terraform apply will succeed without error. The AWS API might throw an error when you try to update this resource. The same is true for the aws_s3_bucket_server_side_encryption_configuration resource.

However, since you can be certain the deployed infrastructure matches your updated configuration, importing the resources will align the Terraform state with the infrastructure in AWS, avoiding potential conflicts.

Import resources

The Terraform Registry has a section under each resource which explains how to import it. For example, the aws_s3_bucket_versioning resource explains how to import using the terraform import command or an import block (if you are using Terraform v1.5.0 and later).

We recommend using the configuration-driven import using import blocks since it is predictable, can be automated, and lets you preview an import operation before modifying state.

Add the aws_s3_bucket_versioning resource import block to the configuration.

+import {
+  to = aws_s3_bucket_versioning.testing
+  id = "testing-provider-upgrades-bucket-00001"
+}
+
 resource "aws_s3_bucket" "testing" {
   bucket = "testing-provider-upgrades-bucket-00001"

Now, check the speculative plan output to see if you have addressed the aws_s3_bucket_versioning resource creation message.

$ terraform plan
aws_s3_bucket.testing: Refreshing state... [id=testing-provider-upgrades-bucket-00001]
aws_s3_bucket_versioning.testing: Preparing import... [id=testing-provider-upgrades-bucket-00001]
aws_s3_bucket_versioning.testing: Refreshing state... [id=testing-provider-upgrades-bucket-00001]
  
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create
  ~ update in-place
  
Terraform will perform the following actions:
  
  # aws_s3_bucket.testing will be updated in-place
  ~ resource "aws_s3_bucket" "testing" {
        id                      = "testing-provider-upgrades-bucket-00001"
        tags                    = {}
        # (12 unchanged attributes hidden)
  
      - server_side_encryption_configuration {
          - rule {
              - bucket_key_enabled = false -> null
  
              - apply_server_side_encryption_by_default {
                  - sse_algorithm = "AES256" -> null
                    # (1 unchanged attribute hidden)
                }
            }
        }
  
        # (1 unchanged block hidden)
    }
  
  # aws_s3_bucket_server_side_encryption_configuration.testing will be created
  + resource "aws_s3_bucket_server_side_encryption_configuration" "testing" {
      + bucket = "testing-provider-upgrades-bucket-00001"
      + id = (known after apply)
  
      + rule {
          + bucket_key_enabled = false
  
          + apply_server_side_encryption_by_default {
              + sse_algorithm = "AES256"
                # (1 unchanged attribute hidden)
            }
        }
    }
  
  # aws_s3_bucket_versioning.testing will be imported
    resource "aws_s3_bucket_versioning" "testing" {
        bucket            = "testing-provider-upgrades-bucket-00001"
        expected_bucket_owner = null
        id                = "testing-provider-upgrades-bucket-00001"
  
        versioning_configuration {
            mfa_delete = null
            status     = "Enabled"
        }
    }
  
Plan: 1 to import, 1 to add, 1 to change, 0 to destroy.

Great! You have successfully refactored the Terraform configuration to address having an add action in our speculative plan for the aws_s3_bucket_versioning resource. Now, you need to address the remaining add action.

Add the aws_s3_bucket_server_side_encryption_configuration resource import block to the configuration.

+import {
+  to = aws_s3_bucket_server_side_encryption_configuration.testing
+  id = "testing-provider-upgrades-bucket-00001"
+}
+
 resource "aws_s3_bucket" "testing" {
   bucket = "testing-provider-upgrades-bucket-00001"
 }

Again, check the speculative plan output to see if you have addressed the aws_s3_bucket_server_side_encryption_configuration resource creation message.

aws_s3_bucket.testing: Refreshing state... [id=testing-provider-upgrades-bucket-00001]
aws_s3_bucket_server_side_encryption_configuration.testing: Preparing import... [id=testing-provider-upgrades-bucket-00001]
aws_s3_bucket_versioning.testing: Preparing import... [id=testing-provider-upgrades-bucket-00001]
aws_s3_bucket_server_side_encryption_configuration.testing: Refreshing state... [id=testing-provider-upgrades-bucket-00001]
aws_s3_bucket_versioning.testing: Refreshing state... [id=testing-provider-upgrades-bucket-00001]
  
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place
  
Terraform will perform the following actions:
  
  # aws_s3_bucket.testing will be updated in-place
  ~ resource "aws_s3_bucket" "testing" {
        id                      = "testing-provider-upgrades-bucket-00001"
        tags                    = {}
        # (12 unchanged attributes hidden)
  
      - server_side_encryption_configuration {
          - rule {
              - bucket_key_enabled = false -> null
  
              - apply_server_side_encryption_by_default {
                  - sse_algorithm = "AES256" -> null
                    # (1 unchanged attribute hidden)
                }
            }
        }
  
        # (1 unchanged block hidden)
    }
  
  # aws_s3_bucket_server_side_encryption_configuration.testing will be imported
    resource "aws_s3_bucket_server_side_encryption_configuration" "testing" {
        bucket            = "testing-provider-upgrades-bucket-00001"
        expected_bucket_owner = null
        id                = "testing-provider-upgrades-bucket-00001"
  
        rule {
            bucket_key_enabled = false
  
            apply_server_side_encryption_by_default {
                kms_master_key_id = null
                sse_algorithm     = "AES256"
            }
        }
    }
  
  # aws_s3_bucket_versioning.testing will be imported
    resource "aws_s3_bucket_versioning" "testing" {
        bucket            = "testing-provider-upgrades-bucket-00001"
        expected_bucket_owner = null
        id                = "testing-provider-upgrades-bucket-00001"
  
        versioning_configuration {
            mfa_delete = null
            status     = "Enabled"
        }
    }
  
Plan: 2 to import, 0 to add, 1 to change, 0 to destroy.

Fantastic! There are no more add actions left in our speculative plan output.

However, there is still a unexpected change action (update-in-place) for the original S3 bucket after removing the server_side_encryption_configuration block.

Discover a bug

Now that you have exhausted your options, we recommend searching for similar issues posted to the provider's GitHub repository.

This shows that there is a bug in the AWS provider version that you just upgraded to. In this case, you could pause this effort and wait for the issues to be resolved, or you can add a lifecycle block to the S3 bucket resource that will ignore the changes.

Lifecycle ignore changes

Since you want to move forward with your changes, add a lifecycle block to ignore the changes, knowing that they do not represent actual changes to the S3 configuration and are incorrectly being shown in a speculative plan due to a bug in the provider.

Add a lifecycle block to ignore_changes for server_side_encryption_configuration within the aws_s3_bucket resource block:

 resource "aws_s3_bucket" "testing" {
   bucket = "testing-provider-upgrades-bucket-00001"
+
+  # Temporary fix for https://github.com/hashicorp/terraform-provider-aws/issues/24153.
+  lifecycle {
+    ignore_changes = [
+      server_side_encryption_configuration
+    ]
+  }
 }

Adding a lifecycle block to ignore changes is a temporary workaround and an additional ticket should be put into the platform team's backlog to remove this whenever the provider bug has been fixed.

Verify the speculative plan has all errors, warnings, and actions removed:

$ terraform plan
aws_s3_bucket.testing: Refreshing state... [id=testing-provider-upgrades-bucket-00001]
aws_s3_bucket_versioning.testing: Preparing import... [id=testing-provider-upgrades-bucket-00001]
aws_s3_bucket_server_side_encryption_configuration.testing: Preparing import... [id=testing-provider-upgrades-bucket-00001]
aws_s3_bucket_server_side_encryption_configuration.testing: Refreshing state... [id=testing-provider-upgrades-bucket-00001]
aws_s3_bucket_versioning.testing: Refreshing state... [id=testing-provider-upgrades-bucket-00001]
  
Terraform will perform the following actions:
  
  # aws_s3_bucket_server_side_encryption_configuration.testing will be imported
    resource "aws_s3_bucket_server_side_encryption_configuration" "testing" {
        bucket            = "testing-provider-upgrades-bucket-00001"
        expected_bucket_owner = null
        id                = "testing-provider-upgrades-bucket-00001"
  
        rule {
            bucket_key_enabled = false
  
            apply_server_side_encryption_by_default {
                kms_master_key_id = null
                sse_algorithm     = "AES256"
            }
        }
    }
  
  # aws_s3_bucket_versioning.testing will be imported
    resource "aws_s3_bucket_versioning" "testing" {
        bucket            = "testing-provider-upgrades-bucket-00001"
        expected_bucket_owner = null
        id                = "testing-provider-upgrades-bucket-00001"
  
        versioning_configuration {
            mfa_delete = null
            status     = "Enabled"
        }
    }
  
Plan: 2 to import, 0 to add, 0 to change, 0 to destroy.

You no longer have any errors, warnings, or infrastructure changes after upgrading to a newer provider version.

Final Terraform configuration

The following is the final Terraform configuration after you refactored it.

terraform {
  required_version = "~> 1.5"
  
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "3.75.0"
    }
  }
}
  
provider "aws" {}
  
import {
  to = aws_s3_bucket_versioning.testing
  id = "testing-provider-upgrades-bucket-00001"
}
  
import {
  to = aws_s3_bucket_server_side_encryption_configuration.testing
  id = "testing-provider-upgrades-bucket-00001"
}
  
resource "aws_s3_bucket" "testing" {
  bucket = "testing-provider-upgrades-bucket-00001"
  
  # Temporary fix for https://github.com/hashicorp/terraform-provider-aws/issues/24153.
  lifecycle {
    ignore_changes = [
      server_side_encryption_configuration
    ]
  }
}
  
resource "aws_s3_bucket_versioning" "testing" {
  bucket = aws_s3_bucket.testing.id
  
  versioning_configuration {
    status = "Enabled"
  }
}
  
resource "aws_s3_bucket_server_side_encryption_configuration" "testing" {
  bucket = aws_s3_bucket.testing.id
  
  rule {
    bucket_key_enabled = false
  
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}

After deploying the changes, we recommend you remove import blocks from your configuration. Alternatively, you can leave them as a record of the resource's origin.

Review changes

Terraform producers and consumers are responsible for reviewing the changes to the Terraform code.

After refactoring the Terraform code to address deprecated attributes and to ensure there are no actions in the speculative plan, the next step is to submit the updated code for team review. This ensures that the changes are reviewed by peers for accuracy, adherence to best practices, and to catch any potential issues before deployment.

Push the refactored code to the upgrade-aws-provider branch we checked out in Step 3 and create a pull request for the team to review in your VCS provider (for example, GitHub).

Provide a clear and concise description of the changes made, including the reason for the refactor and any relevant context. Mention that you've ensured there are no errors, warnings, or actions in the speculative plan.

For example,

This PR refactors the Terraform code to replace the deprecated `versioning` block in the `aws_s3_bucket` resource with the recommended `aws_s3_bucket_versioning` resource. While testing, I have found an additional deprecation warning that was unexpected.

To address these warnings, two resources were added to align with the latest AWS provider recommendations. These resources will be imported during the next `terraform apply` to ensure there are no unexpected errors.

I've also found a [bug](https://github.com/hashicorp/terraform-provider-aws/issues/28701) in this provider version so a `lifecycle` block has been added to ignore misleading changes in the plan output.

Please review for accuracy and adherence to best practices.

Respond to any comments or feedback provided by the reviewers. Make necessary adjustments to the code based on the review and update the pull request accordingly.

Merge changes

Terraform producers and consumers are responsible for merging the refactored code into the main branch.

After the refactored Terraform code has been reviewed and approved by the team, the next step is to merge the changes into the main branch. This step finalizes the code changes, making them part of the main codebase and preparing them for deployment.

Apply terraform configuration

After merging the refactored code into the main branch, the final step is to run terraform apply to deploy the changes using the upgraded provider version. This ensures that the infrastructure is updated and aligned with the new provider version, and any configuration changes are applied.

Depending on how your infrastructure is managed, your changes might be deployed automatically as part of merging your pull request into the main branch. Alternatively, there might be some other process that needs to be approved before deployment.

Verify deployment

Terraform producers and consumers are responsible for verifying the deployment has succeeded without issues.

This involves checking that all resources are correctly provisioned and configured according to the updated Terraform state. Ensure that there are no errors or discrepancies, and confirm that the infrastructure behaves as expected with the new provider version.

Conclusion

In conclusion, by following this guide, you have successfully navigated the process of upgrading the AWS provider including:

• Monitoring for new provider releases.
• Updating the provider configuration to use the new version.
• Run speculative plans to identify potential issues.

The example workflow showed common issues that arise:

• Deprecated arguments in the existing configuration.
• Aligning your configuration with the latest provider recommendations.
• Importing resources to maintain state consistency.
• Finding bugs in the provider by searching the repository issues.

Finally, you now understand the importance of peer reviews followed by deployment verification of the upgraded configuration, ensuring a smooth transition to the new provider version.