Implement automated testing

11min
|
Terraform

In this tutorial, you will add automated acceptance testing capabilities to the data source and resource of a provider that interacts with the API of a fictional coffee-shop application called HashiCups. To do this, you will:

Implement data source id attribute.
This ensures the data source is compatible with the testing framework.
Implement data source acceptance testing.
This automates the end-to-end testing of the data source.
Run data source acceptance testing.
This ensures that the data source testing works as expected.
Implement resource acceptance testing.
This automates the end-to-end testing of the resource.
Run resource acceptance testing.
This ensures that the resource testing works as expected.

The terraform-plugin-testing Go module helper/resource package enables providers to implement automated acceptance testing. The testing framework is built on top of standard go test command functionality and calls actual Terraform commands, such as terraform apply, terraform import, and terraform destroy. Unlike manual testing, you do not have to locally reinstall the provider on code updates or switch directories to use the expected Terraform configuration when you run the automated tests.

Prerequisites

To follow this tutorial, you need:

Go 1.19+ installed and configured.
Terraform v1.0.3+ installed locally.
Docker and Docker Compose to run an instance of HashiCups locally.

Navigate to your terraform-provider-hashicups-pf directory.

Your code should match the import-order branch from the example repository.

Clone the import-order branch of the Terraform HashiCups Provider repository.

$ git clone --branch import-order https://github.com/hashicorp/terraform-provider-hashicups-pf

Change into the cloned repository.

$ cd terraform-provider-hashicups-pf

Then, install all the provider's dependencies.

$ go mod tidy

Build the provider

Terraform verifies provider versions and checksums at the terraform init phase. Terraform downloads your providers from either the provider registry, or from a local registry. However, while building your provider, you will want to try a test configuration against a development build of a provider that doesn't have an associated version number yet, and doesn't have an official set of checksums listed in a provider registry.

As a convenience for provider development, Terraform supports a special additional block dev_overrides in a CLI configuration file called .terraformrc. This block overrides all other configured installation methods.

Terraform searches for the .terraformrc file in your home directory and applies any configuration settings you set.

First, find the GOBIN path where Go installs your binaries. Your path may vary depending on how your Go environment variables are configured.

$ go env GOBIN
/Users/<Username>/go/bin

If the GOBIN go environment variable is not set, use the default path, /Users/<Username>/go/bin.

Create a new file called .terraformrc in the root directory (~), then add the dev_overrides block below. Change the <PATH> to the value returned from the echo $GOBIN command above.

~/.terraformrc

provider_installation {

  dev_overrides {
    "hashicorp.com/edu/hashicups-pf" = "<PATH>"
  }

  # For all other providers, install them directly from their origin provider
  # registries as normal. If you omit this, Terraform will _only_ use
  # the dev_overrides block, and so no other providers will be available.
  direct {}
}

You must create your CLI configuration file in your user %APPDATA% root directory. The physical location of this directory depends on your Windows version and system configuration.

Search for the user environment variables in PowerShell to find its location on your system.

$ $env:APPDATA

In another terminal, change into the directory returned from the command above.

Find the GOBIN path where go installs your binaries. Your path may vary depending on how your Go environment variables are configured.

$ go env GOBIN
/Users/<Username>/go/bin

If the GOBIN go environment variable is not set, use the default path, /Users/<Username>/go/bin.

Create a new file called terraform.rc and add the dev_overrides block below. Change the <PATH> to the value returned from the go env GOBIN command above.

terraform.rc

provider_installation {

  dev_overrides {
    "hashicorp.com/edu/hashicups-pf" = "<PATH>"
  }

  # For all other providers, install them directly from their origin provider
  # registries as normal. If you omit this, Terraform will _only_ use
  # the dev_overrides block, and so no other providers will be available.
  direct {}
}

Start HashiCups locally

Your HashiCups provider requires a running instance of HashiCups. Navigate to the docker_compose directory.

$ cd docker_compose

Run docker-compose up to spin up a local instance of HashiCups on port 19090.

$ docker-compose up

Leave this process running in your terminal window. The HashiCups service will print out log messages in this terminal.

In the original terminal window, verify that HashiCups is running by sending a request to its health check endpoint. The HashiCups service will respond with ok.

$ curl localhost:19090/health/readyz
ok

Create a HashiCups user

HashiCups requires a username and password to generate a JSON web token (JWT) which is used to authenticate against protected endpoints. You will use this user to authenticate to the HashiCups provider to manage your orders.

Create a user on HashiCups named education with the password test123.

$ curl -X POST localhost:19090/signup -d '{"username":"education", "password":"test123"}'
{"UserID":1,"Username":"education","token":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE1OTEwNzgwODUsInVzZXJfaWQiOjIsInVzZXJuYW1lIjoiZWR1Y2F0aW9uIn0.CguceCNILKdjOQ7Gx0u4UAMlOTaH3Dw-fsll2iXDrYU"}

Set the HASHICUPS_TOKEN environment variable to the token you retrieved from invoking the /signup endpoint. You will use this later in the tutorial to verify that Terraform has created your HashiCups order.

$ export HASHICUPS_TOKEN=ey...

The terminal containing your HashiCups logs will record the sign up operation.

api_1  | 2020-12-10T09:19:50.601Z [INFO]  Handle User | signup

Now that the HashiCups app is running, you are ready to start working on the Terraform provider.

If you're stuck at any point during this tutorial, refer to the acceptance-tests branch to see the changes implemented in this tutorial.

Implement data source id attribute

The testing framework requires an id attribute to be present in every data source and resource. In order to run tests on data sources and resources that do not have their own ID, you must implement an ID field with a placeholder value.

Open the internal/provider/coffees_data_source.go file.

Add the id attribute to the Schema method with the following.

internal/provider/coffees_data_source.go

func (d *coffeesDataSource) Schema(_ context.Context, _ datasource.SchemaRequest, resp *datasource.SchemaResponse) {
    resp.Schema = schema.Schema{
        Attributes: map[string]schema.Attribute{
            "id": schema.StringAttribute{
                Computed: true,
            },
            "coffees": schema.ListNestedAttribute{
                //...

Replace the coffeesDataSourceModel data source model with the following.

internal/provider/coffees_data_source.go

// coffeesDataSourceModel maps the data source schema data.
type coffeesDataSourceModel struct {
    Coffees []coffeesModel `tfsdk:"coffees"`
    ID      types.String   `tfsdk:"id"`
}

Set a placeholder value near the end of the data source's Read method immediately before returning the state with the following.

internal/provider/coffees_data_source.go

func (d *coffeesDataSource) Read(ctx context.Context, req datasource.ReadRequest, resp *datasource.ReadResponse) {
    //...

    state.ID = types.StringValue("placeholder")

    // Set state
    diags := resp.State.Set(ctx, &state)
    resp.Diagnostics.Append(diags...)
    if resp.Diagnostics.HasError() {
        return
    }
}

Implement data source acceptance testing

Data source acceptance testing is used to verify the Terraform state contains data after being read from the API.

Most providers will manage some shared implementation details in a single testing file to simplify the data source and resource testing implementations.

Navigate to the internal/provider directory and remove the example scaffolding test files.

$ cd internal/provider && rm example_data_source_test.go; rm example_resource_test.go

Open the internal/provider/provider_test.go file and replace the existing code with the following.

internal/provider/provider_test.go

package provider

import (
    "github.com/hashicorp/terraform-plugin-framework/providerserver"
    "github.com/hashicorp/terraform-plugin-go/tfprotov6"
)

const (
    // providerConfig is a shared configuration to combine with the actual
    // test configuration so the HashiCups client is properly configured.
    // It is also possible to use the HASHICUPS_ environment variables instead,
    // such as updating the Makefile and running the testing through that tool.
    providerConfig = `
provider "hashicups" {
  username = "education"
  password = "test123"
  host     = "http://localhost:19090"
}
`
)

var (
    // testAccProtoV6ProviderFactories are used to instantiate a provider during
    // acceptance testing. The factory function will be invoked for every Terraform
    // CLI command executed to create a provider server to which the CLI can
    // reattach.
    testAccProtoV6ProviderFactories = map[string]func() (tfprotov6.ProviderServer, error){
        "hashicups": providerserver.NewProtocol6WithError(New("test")()),
    }
)

Create a new internal/provider/coffees_data_source_test.go file with the following.

internal/provider/coffees_data_source_test.go

package provider

import (
    "testing"

    "github.com/hashicorp/terraform-plugin-testing/helper/resource"
)

func TestAccCoffeesDataSource(t *testing.T) {
    resource.Test(t, resource.TestCase{
        ProtoV6ProviderFactories: testAccProtoV6ProviderFactories,
        Steps: []resource.TestStep{
            // Read testing
            {
                Config: providerConfig + `data "hashicups_coffees" "test" {}`,
                Check: resource.ComposeAggregateTestCheckFunc(
                    // Verify number of coffees returned
                    resource.TestCheckResourceAttr("data.hashicups_coffees.test", "coffees.#", "9"),
                    // Verify the first coffee to ensure all attributes are set
                    resource.TestCheckResourceAttr("data.hashicups_coffees.test", "coffees.0.description", ""),
                    resource.TestCheckResourceAttr("data.hashicups_coffees.test", "coffees.0.id", "1"),
                    resource.TestCheckResourceAttr("data.hashicups_coffees.test", "coffees.0.image", "/hashicorp.png"),
                    resource.TestCheckResourceAttr("data.hashicups_coffees.test", "coffees.0.ingredients.#", "1"),
                    resource.TestCheckResourceAttr("data.hashicups_coffees.test", "coffees.0.ingredients.0.id", "6"),
                    resource.TestCheckResourceAttr("data.hashicups_coffees.test", "coffees.0.name", "HCP Aeropress"),
                    resource.TestCheckResourceAttr("data.hashicups_coffees.test", "coffees.0.price", "200"),
                    resource.TestCheckResourceAttr("data.hashicups_coffees.test", "coffees.0.teaser", "Automation in a cup"),
                    // Verify placeholder id attribute
                    resource.TestCheckResourceAttr("data.hashicups_coffees.test", "id", "placeholder"),
                ),
            },
        },
    })
}

Verify data source testing functionality

Now that you have implemented the testing functionality to the data source, you can run the tests.

Run Go testing with the TF_ACC environment variable set. The test framework will report that your data source's test passed.

$ TF_ACC=1 go test -count=1 -v
=== RUN   TestAccCoffeesDataSource
--- PASS: TestAccCoffeesDataSource (1.23s)
PASS
ok      terraform-provider-hashicups-pf/internal/provider   2.120s

Implement resource testing functionality

Resource acceptance testing is used to verify the entire resource lifecycle, such as the Create, Read, Update, and Delete functionality, along with import capabilities. The testing framework automatically handles destroying test resources and returning any errors as a final step, regardless of whether there is a destroy step explicitly written.

Create a new internal/provider/order_resource_test.go file with the following.

internal/provider/order_resource_test.go

package provider

import (
    "testing"

    "github.com/hashicorp/terraform-plugin-testing/helper/resource"
)

func TestAccOrderResource(t *testing.T) {
    resource.Test(t, resource.TestCase{
        ProtoV6ProviderFactories: testAccProtoV6ProviderFactories,
        Steps: []resource.TestStep{
            // Create and Read testing
            {
                Config: providerConfig + `
resource "hashicups_order" "test" {
  items = [
    {
      coffee = {
        id = 1
      }
      quantity = 2
    },
  ]
}
`,
                Check: resource.ComposeAggregateTestCheckFunc(
                    // Verify number of items
                    resource.TestCheckResourceAttr("hashicups_order.test", "items.#", "1"),
                    // Verify first order item
                    resource.TestCheckResourceAttr("hashicups_order.test", "items.0.quantity", "2"),
                    resource.TestCheckResourceAttr("hashicups_order.test", "items.0.coffee.id", "1"),
                    // Verify first coffee item has Computed attributes filled.
                    resource.TestCheckResourceAttr("hashicups_order.test", "items.0.coffee.description", ""),
                    resource.TestCheckResourceAttr("hashicups_order.test", "items.0.coffee.image", "/hashicorp.png"),
                    resource.TestCheckResourceAttr("hashicups_order.test", "items.0.coffee.name", "HCP Aeropress"),
                    resource.TestCheckResourceAttr("hashicups_order.test", "items.0.coffee.price", "200"),
                    resource.TestCheckResourceAttr("hashicups_order.test", "items.0.coffee.teaser", "Automation in a cup"),
                    // Verify dynamic values have any value set in the state.
                    resource.TestCheckResourceAttrSet("hashicups_order.test", "id"),
                    resource.TestCheckResourceAttrSet("hashicups_order.test", "last_updated"),
                ),
            },
            // ImportState testing
            {
                ResourceName:      "hashicups_order.test",
                ImportState:       true,
                ImportStateVerify: true,
                // The last_updated attribute does not exist in the HashiCups
                // API, therefore there is no value for it during import.
                ImportStateVerifyIgnore: []string{"last_updated"},
            },
            // Update and Read testing
            {
                Config: providerConfig + `
resource "hashicups_order" "test" {
  items = [
    {
      coffee = {
        id = 2
      }
      quantity = 2
    },
  ]
}
`,
                Check: resource.ComposeAggregateTestCheckFunc(
                    // Verify first order item updated
                    resource.TestCheckResourceAttr("hashicups_order.test", "items.0.quantity", "2"),
                    resource.TestCheckResourceAttr("hashicups_order.test", "items.0.coffee.id", "2"),
                    // Verify first coffee item has Computed attributes updated.
                    resource.TestCheckResourceAttr("hashicups_order.test", "items.0.coffee.description", ""),
                    resource.TestCheckResourceAttr("hashicups_order.test", "items.0.coffee.image", "/packer.png"),
                    resource.TestCheckResourceAttr("hashicups_order.test", "items.0.coffee.name", "Packer Spiced Latte"),
                    resource.TestCheckResourceAttr("hashicups_order.test", "items.0.coffee.price", "350"),
                    resource.TestCheckResourceAttr("hashicups_order.test", "items.0.coffee.teaser", "Packed with goodness to spice up your images"),
                ),
            },
            // Delete testing automatically occurs in TestCase
        },
    })
}

Verify resource testing functionality

Now that you have implemented the testing functionality for the order resource, run the tests.

Run Go testing with the TF_ACC environment variable set and only running the resource tests. The test framework will report that your resource's test passed.

$ TF_ACC=1 go test -count=1 -run='TestAccOrderResource' -v
=== RUN   TestAccOrderResource
--- PASS: TestAccOrderResource (2.01s)
PASS
ok      terraform-provider-hashicups-pf/internal/provider   2.754s

Navigate to the terraform-provider-hashicups-pf directory.

$ cd ../..

Next steps

Congratulations! You have enhanced the provider with acceptance testing capabilities.

If you were stuck during this tutorial, checkout the acceptance-tests branch to see the changes implemented in this tutorial.

To learn more about the Terraform Plugin Framework, refer to the Terraform Plugin Framework documentation.
For a full capability comparison between the SDKv2 and the Plugin Framework, refer to the Which SDK Should I Use? documentation.
The Terraform HashiCups (plugin-framework) provider's main branch contains the complete HashiCups provider. It includes a data source written with the plugin framework and implements create, read, update and delete functionality for the order resource.
Submit any Terraform Plugin Framework bug reports or feature requests to the development team in the Terraform Plugin Framework Github repository.
Submit any Terraform Plugin Framework questions in the Terraform Plugin Framework Discuss forum.

Import

Generate documentation