Acquisition complete HashiCorp officially joins the IBM family. Learn more
Terraform
AWS Services

Upgrade RDS major version

  • 16min
  • |
  • Terraform

Terraform enables you to manage your Amazon Relational Database Service (RDS) instances over their lifecycle. Using Terraform's built-in lifecycle arguments, you can manage the dependency and upgrade ordering for tightly coupled resources like RDS instances and their parameter groups. You will also use Terraform to securely set your database password and store it in AWS Systems Manager (SSM).

In this tutorial, you will configure an RDS instance with Terraform, storing the password in SSM. Next, you will perform a major version upgrade on your RDS instance using Terraform and review how Terraform handles dependencies when you use it to manage resources that depend on other resources in your configuration.

Prerequisites

This tutorial assumes that you are familiar with the standard Terraform workflow. If you are new to Terraform, complete the Get Started tutorials first.

For this tutorial, you will need:

Clone example repository

Clone the example repository for this tutorial, which contains configuration for an RDS instance and parameter group.

$ git clone https://github.com/hashicorp-education/learn-terraform-rds-upgrade

Change into the repository directory.

$ cd learn-terraform-rds-upgrade

Review example configuration

Open main.tf in your code editor to review the resources you will provision. This configuration defines the following resources:

  • The provider block which configures your AWS region and default tags.
  • A data source to load the availability zones for your configured region.
  • An AWS VPC to provision your RDS instance in.
  • An RDS subnet group, which designates a collection of subnets for RDS placement.
  • A security group that will allow access to your RDS instance on port 5432.
  • An RDS parameter group.
  • An aws_db_instance, configured with PostgreSQL 15.

Note

The example configuration allows access to your RDS instance from the public internet, so that you can connect to it later in this tutorial. In production scenarios, we recommend you follow security best practices, such as placing your RDS instance in a private subnet and restricting access to it only from subnets you control.

The aws_db_parameter_group resource's family attribute configures the major version of your database instance. In this case, the parameter group family is postgres15, so the RDS engine will be PostgreSQL v15.

main.tf

resource "aws_db_parameter_group" "education" {
  name_prefix = "${random_pet.name.id}-education"
  family      = "postgres15"

  parameter {
    name  = "log_connections"
    value = "1"
  }

  lifecycle {
    create_before_destroy = true
  }
}

While you could use a default AWS parameter group for your database, we recommend that you maintain a custom one for your RDS instances. You cannot modify the parameters on the default parameter groups maintained by AWS. If you need to update an RDS setting in the future, you can modify your custom parameter group rather than creating a new one at that time.

The configuration generates a random password for your RDS instance using an ephemeral resource. The configuration then sets this password for your database using a write-only argument. The configuration also stores and encrypts the generated password in AWS SSM using another write-only argument. for your RDS instance, and stores the password as a parameter in AWS SSM, encrypted with your default SSM key. The configuration sets the password for your database and stores it in SSM using write-only arguments.

Review the ephemeral resource for the database password in main.tf.

main.tf

ephemeral "random_password" "db_password" {
  length = 16
}

The random_password.db_password is an ephemeral resource. Terraform does not store ephemeral resources in its state or plan files.

Note

Ephemeral resources and values are available in Terraform 1.10 and later.

The configuration uses random_password.db_password to set the value of two write-only arguments. A resource's write-only arguments are only available during the current operation, and Terraform does not store the argument's values in state or plan files. Terraform providers define write-only arguments for values that you do not want to store in Terraform's state, such as passwords or other secrets.

The first write-only argument, aws_db_instance.password_wo, sets the password on the RDS instance. The second write-only argument, aws_ssm_parameter.value_wo, stores the password value as an AWS SSM secret. With this configuration, Terraform does not store your database password, and the only way to retrieve that password is by querying the AWS SSM parameter.

main.tf

locals {
  # Increment db_password_version to update the DB password and store the new
  # password in SSM.
  db_password_version = 1
}

resource "aws_db_instance" "education" {
  identifier                  = "${random_pet.name.id}-education"
  instance_class              = "db.t3.micro"
  allocated_storage           = 10
  apply_immediately           = true
  engine                      = "postgres"
  engine_version              = "15"
  username                    = "edu"
  password_wo                 = ephemeral.random_password.db_password.result
  password_wo_version         = local.db_password_version
## ...
}

resource "aws_ssm_parameter" "secret" {
  name             = "/education/database/password/master"
  description      = "Password for RDS database."
  type             = "SecureString"
  value_wo         = ephemeral.random_password.db_password.result
  value_wo_version = local.db_password_version
}

Because Terraform does not store the value of write-only arguments, it cannot detect if the value of a write-only argument changed in your configuration. To track whether a write-only argument changes, the AWS provider includes accompanying versioning arguments: password_wo uses password_wo_version and value_wo uses value_wo_version.

Versioning arguments are tracked in state. You can indicate to Terraform and providers that a write-only arguments has changed by incrementing the corresponding _version argument. For example, incrementing password_wo_version lets Terraform know the value of password_wo has changed. Terraform then records that change in its plan, notifying the provider that password_wo has a new value it can use.

The example configuration sets both password_wo_version and value_wo_version to the same local value, local.db_password_version. If the values were hard-coded, a user might update one of these values but not the other, and cause the database password to become out of sync with the password stored in SSM.

Note

Write-only arguments are available in Terraform 1.11 and later.

Create RDS instance

In your terminal, initialize the Terraform configuration to install the module and providers used in this tutorial.

$ terraform init
Initializing the backend...
Initializing modules...
Downloading registry.terraform.io/terraform-aws-modules/vpc/aws 5.19.0 for vpc...
- vpc in .terraform/modules/vpc
Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
- Reusing previous version of hashicorp/random from the dependency lock file
- Installing hashicorp/aws v5.88.0...
- Installed hashicorp/aws v5.88.0 (signed by HashiCorp)
- Installing hashicorp/random v3.7.0...
- Installed hashicorp/random v3.7.0 (signed by HashiCorp)

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Next, apply your configuration to create your RDS instance and other resources. Enter yes when prompted to confirm the operation. Note that it can take up to 10 minutes to create an RDS instance.

$ terraform apply
ephemeral.random_password.db_password: Opening...
ephemeral.random_password.db_password: Opening complete after 0s
data.aws_availability_zones.available: Reading...
data.aws_availability_zones.available: Read complete after 1s [id=us-east-2]
ephemeral.random_password.db_password: Closing...
ephemeral.random_password.db_password: Closing complete after 0s

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_db_instance.education will be created
  + resource "aws_db_instance" "education" {
      + address                               = (known after apply)
      + allocated_storage                     = 10

## ...

Plan: 19 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + random_pet_name = (known after apply)
  + rds_hostname    = (sensitive value)
  + rds_port        = (sensitive value)
  + rds_username    = (sensitive value)
  + region          = "us-east-2"

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

## ...

random_pet_name = "pelican"
rds_hostname = <sensitive>
rds_port = <sensitive>
rds_username = <sensitive>
region = "us-east-2"

Notice that the RDS hostname, port, and username are marked as sensitive. The example configuration sets the sensitive attribute to true for these outputs so that Terraform won't include those values in its output by default. For example, the rds_hostname output block is designated as sensitive.

outputs.tf

output "rds_hostname" {
  description = "RDS instance hostname."
  value       = aws_db_instance.education.address
  sensitive   = true
}

Unlike ephemeral resources and write-only arguments, Terraform stores sensitive values in its state file, and will output these values as plain text if you specify the -raw flag for the terraform output command, or the -json flag to print out your workspace's output values in JSON format. Terraform stores sensitive values unencrypted in its state file, so you must keep this file secure.

Seed database with mock data

Next, connect to the database with the psql command line utility, and seed it. The coffees.sql file in the repository contains commands that populate your database with mock data about HashiCorp-themed coffee beverages.

psql can access your password using thr PGPASSWORD environment variable. Set your PostgreSQL password as an environment variable by retrieving the parameter from AWS SSM and using jq to extract the password from the response.

$ export PGPASSWORD=$( \
  aws ssm get-parameter \
    --region=$(terraform output -raw region) \
    --name=/education/database/password/master \
    --with-decryption \
  | jq --raw-output '.Parameter.Value' \
  )

Note

The previous command will save your database password unencrypted in the PGPASSWORD environment variable in your shell session. For production use cases, you may wish to unset this value once you are done using it.

Then, execute the script.

$ psql -h $(terraform output -raw rds_hostname) -U $(terraform output -raw rds_username) postgres -f coffees.sql
SET
CREATE EXTENSION
CREATE TABLE
CREATE TABLE
CREATE TABLE
## ...

Connect to your database to inspect your records.

$ psql -h $(terraform output -raw rds_hostname) -U $(terraform output -raw rds_username) postgres
psql (16.3, server 15.5)
SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384, compression: off)
Type "help" for help.

postgres=>

At the postgres prompt, list all of the coffees in your database.

$ SELECT * FROM coffees;
1 | Packer Spiced Latte | Packed with goodness to spice up your images                 |             |   350 | /packer.png    | 2021-08-30 00:00:00 | 2021-08-30 00:00:00 |
2 | Vaulatte            | Nothing gives you a safe and secure feeling like a Vaulatte  |             |   200 | /vault.png     | 2021-08-30 00:00:00 | 2021-08-30 00:00:00 |
3 | Nomadicano          | Drink one today and you will want to schedule another        |             |   150 | /nomad.png     | 2021-08-30 00:00:00 | 2021-08-30 00:00:00 |
4 | Terraspresso        | Nothing kickstarts your day like a provision of Terraspresso |             |   150 | /terraform.png | 2021-08-30 00:00:00 | 2021-08-30 00:00:00 |
5 | Vagrante espresso   | Stdin is not a tty                                           |             |   200 | /vagrant.png   | 2021-08-30 00:00:00 | 2021-08-30 00:00:00 |
6 | Connectaccino       | Discover the wonders of our meshy service                    |             |   250 | /consul.png    | 2021-08-30 00:00:00 | 2021-08-30 00:00:00 |

Type exit to exit psql.

Upgrade RDS instance

When managing an RDS instance, a common task is to upgrade the database to a new major version. To do so, you will upgrade the database engine version and the parameter group family in your Terraform configuration, and apply the change.

Take a snapshot

Before you upgrade your database, create a backup snapshot of your data. It is good practice to back up your data when you perform operations on your databases so that you have a point of recovery in the event of data loss or other error.

Add the following configuration to main.tf to create a snapshot of your database.

main.tf

resource "aws_db_snapshot" "pre_16_upgrade" {
  db_instance_identifier = aws_db_instance.education.identifier
  db_snapshot_identifier = "pre16upgradebackup"
}

Add the following to outputs.tf to report the identifier and status of your DB snapshot.

outputs.tf

output "rds_pre_16_backup_identifier" {
  description = "Identifier of the snapshot created before upgrading RDS instance to PostgreSQL 16."
  value = aws_db_snapshot.pre_16_upgrade.db_snapshot_identifier
}

output "rds_pre_16_backup_status" {
  description = "Status of the snapshot created before upgrading RDS instance to PostgreSQL 16."
  value = aws_db_snapshot.pre_16_upgrade.status
}

Apply your configuration to create the snapshot. Enter yes when prompted to confirm the operation.

$ terraform apply
random_pet.name: Refreshing state... [id=pelican]
ephemeral.random_password.db_password: Opening...
ephemeral.random_password.db_password: Opening complete after 0s
data.aws_availability_zones.available: Reading...

## ...

Terraform used the selected providers to generate the following execution plan.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_db_snapshot.pre_16_upgrade will be created
  + resource "aws_db_snapshot" "pre_16_upgrade" {
      + allocated_storage             = (known after apply)
      + availability_zone             = (known after apply)

## ...

Plan: 1 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + rds_pre_16_backup_identifier = "pre16upgradebackup"
  + rds_pre_16_backup_status     = (known after apply)

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

ephemeral.random_password.db_password: Opening...
ephemeral.random_password.db_password: Opening complete after 0s
aws_db_snapshot.pre_16_upgrade: Creating...

## ...

aws_db_snapshot.pre_16_upgrade: Still creating... [2m0s elapsed]
aws_db_snapshot.pre_16_upgrade: Creation complete after 2m1s [id=pre-16-upgrade-backup-pelican]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Outputs:

random_pet_name = "pelican"
rds_hostname = <sensitive>
rds_port = <sensitive>
rds_pre_16_backup_identifier = "pre16upgradebackup"
rds_pre_16_backup_status = "available"
rds_username = <sensitive>
region = "us-east-2"

Update RDS instance version

In this Terraform configuration, the aws_db_instance resource references the aws_db_parameter_group, creating an implicit dependency between the two. As a result, Terraform would first try to upgrade the parameter group, but would error out because the destructive update would attempt to remove a parameter group associated with a running RDS instance.

Terraform offers lifecycle meta-arguments to help you manage more complex resource dependencies such as this one. In this case, the aws_db_parameter_group in the example configuration includes the create_before_destroy argument to ensure that Terraform provisions the new parameter group and upgrades your RDS instance before destroying the original parameter group.

In your main.tf file, make the following changes:

In the aws_rds_parameter_group resource definition, update the family argument to postgres16 as shown below.

main.tf

resource "aws_db_parameter_group" "education" {
  name_prefix = "${random_pet.name.id}-education"
  family      = "postgres16"

  parameter {
    name  = "log_connections"
    value = "1"
  }

  lifecycle {
    create_before_destroy = true
  }
}

Update the version argument for the aws_db_instance resource to 16.

main.tf

resource "aws_db_instance" "education" {
  identifier                  = "${random_pet.name.id}-education"
  instance_class              = "db.t3.micro"
  allocated_storage           = 10
  apply_immediately           = true
  engine                      = "postgres"
  engine_version              = "16"
## ...
}

Upgrade RDS instance

In your terminal, apply your configuration changes to replace the parameter group and upgrade the engine version of your RDS instance. Enter yes when prompted to approve the operation.

Note

Major version upgrades to RDS are a destructive change. AWS will remove the existing data from your database, and you will need to reload it.

$ terraform apply

## ...

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place
+/- create replacement and then destroy

Terraform will perform the following actions:

 # aws_db_instance.education will be updated in-place
 ~ resource "aws_db_instance" "education" {
     ~ engine_version                        = "15" -> "16"
       id                                    = "db-RBX4IELIDWELBC3JXRPYXKANHU"
     ~ parameter_group_name                  = "halibut-education20240510184235754100000001" -> (known after apply)
       tags                                  = {}
       # (52 unchanged attributes hidden)
   }

 # aws_db_parameter_group.education must be replaced
/- resource "aws_db_parameter_group" "education" {
     ~ arn         = "arn:aws:rds:us-east-2:561656980159:pg:halibut-education20240510184235754100000001" -> (known after apply)
     ~ family      = "postgres15" -> "postgres16" # forces replacement
     ~ id          = "halibut-education20240510184235754100000001" -> (known after apply)
     ~ name        = "halibut-education20240510184235754100000001" -> (known after apply)
     - tags        = {} -> null
       # (3 unchanged attributes hidden)

       # (1 unchanged block hidden)
   }

Plan: 1 to add, 1 to change, 1 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

## ...

Apply complete! Resources: 1 added, 1 changed, 1 destroyed.

Outputs:

random_pet_name = "pelican"
rds_hostname = <sensitive>
rds_port = <sensitive>
rds_pre_16_backup_identifier = "pre16upgradebackup"
rds_pre_16_backup_status = "available"
rds_username = <sensitive>
region = "us-east-2"

Note

This upgrade may take up to 20 minutes.

Verify upgrade

Verify that the RDS instance is using Postgres 16.

$ psql -h $(terraform output -raw rds_hostname) -U $(terraform output -raw rds_username) postgres -c "SELECT version()"
                                                 version
---------------------------------------------------------------------------------------------------------
 PostgreSQL 16.3 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 7.3.1 20180712 (Red Hat 7.3.1-17), 64-bit
(1 row)

Destroy infrastructure

Once you have completed the tutorial, destroy your infrastructure to avoid incurring unnecessary costs. Type yes when prompted to confirm the operation.

$ terraform destroy
random_pet.name: Refreshing state... [id=pelican]

## ...

aws_db_snapshot.pre_16_upgrade: Refreshing state... [id=pre-16-upgrade-backup-pelican]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  - destroy

Terraform will perform the following actions:

  # aws_db_instance.education will be destroyed
  - resource "aws_db_instance" "education" {
      - address                               = "pelican-education.c0gk8jn5j2r9.us-east-2.rds.amazonaws.com" -> null
      - allocated_storage                     = 10 -> null
      - allow_major_version_upgrade           = true -> null
      - apply_immediately                     = true -> null

## ...

Plan: 0 to add, 0 to change, 20 to destroy.

Changes to Outputs:
  - random_pet_name              = "pelican" -> null
  - rds_hostname                 = (sensitive value) -> null
  - rds_port                     = (sensitive value) -> null
  - rds_pre_16_backup_identifier = "pre-16-upgrade-backup-pelican" -> null
  - rds_pre_16_backup_status     = "available" -> null
  - rds_username                 = (sensitive value) -> null
  - region                       = "us-east-2" -> null

Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: yes

module.vpc.aws_route_table_association.public[1]: Destroying... [id=rtbassoc-0f201820cea0f3d14]
module.vpc.aws_default_security_group.this[0]: Destroying... [id=sg-065248f22c8ea3239]
module.vpc.aws_route_table_association.public[2]: Destroying... [id=rtbassoc-00cba7a3c2fb4da35]

## ...

random_pet.name: Destroying... [id=pelican]
random_pet.name: Destruction complete after 0s

Destroy complete! Resources: 20 destroyed.

Next steps

In this tutorial, you learned how you can use Terraform's lifecycle arguments to manage a major version upgrade of your RDS instances. To learn more about the concepts used in this configuration, review the following tutorials:

Previous

Cloud Control

 Next

AssumeRole