Perform CRUD operations with providers
Note: We recommend using the Terraform Plugin Framework for new provider development because it offers significant advantages compared to the SDKv2. Refer to the Plugin Framework tutorials to learn how to create providers using the framework.
In this tutorial, you will use a Terraform provider to interact with a fictional coffee-shop application called Hashicups. In the process, you will learn how providers map target APIs to Terraform in order to create, read, update, and delete resources.
Later in the track, you will re-create the HashiCups provider discussed in this tutorial based on the Terraform Plugin SDK v2.
Terraform plugins
Terraform is comprised of Terraform Core and Terraform Plugins.
- Terraform Core reads the configuration and builds the resource dependency graph.
- Terraform Plugins (providers and provisioners) bridge Terraform Core and their respective target APIs. Terraform provider plugins implement resources via basic CRUD (create, read, update, and delete) APIs to communicate with third party services.
Tip
We recommend Terraform plugins consume an external API client library, as shown in the diagram above. If one doesn't exist, you should create one. This is aligned with modern coding practices of keeping software projects modular.
Upon terraform plan
or terraform apply
, Terraform Core asks the Terraform provider to perform an action via a RPC interface. The provider attempts to fulfill the request by invoking a CRUD operation against the target API's client library. This process enforces a clear separation of concerns. Providers are able to serve as an abstraction of a client library.
While most Terraform providers manage cloud infrastructure (e.g. AWS, Azure and GCP providers), providers can serve as an interface to any API and allow Terraform to potentially manage any resource.
Introduction to HashiCups
HashiCups is a demo application that allows you to view and order customized HashiCorp branded coffee.
The API has a number of unprotected and protected endpoints. You can list all coffees and ingredients for a particular coffee without authentication. Once authenticated, you can create, read, update and delete (CRUD) orders.
The Terraform HashiCups provider interfaces with the product's REST API through a GoLang client library. This allows you to manage HashiCups orders through Terraform.
Prerequisites
To follow this tutorial, you need:
- the Terraform 0.14+ CLI installed locally.
- Docker and Docker Compose to run an instance of HashiCups locally.
It also assumes that you are familiar with the usual Terraform plan/apply workflow. If you're new to Terraform itself, refer first to the Getting Started track.
Initialize HashiCups locally
The HashiCups provider requires a running instance of HashiCups. In your terminal, clone the HashiCups Provider repository and navigate to it.
Navigate to the docker_compose
directory and run docker compose up
to spin up a local instance of HashiCups on port :19090
.
Leave this process running in your terminal window. This will contain HashiCups' log messages.
In another terminal window, verify that HashiCups is running by sending a request to its health check endpoint.
Install HashiCups provider
With Terraform 0.13+, you must specify all required providers and their respective source in your Terraform configuration. A provider source string is comprised of [hostname]/[namespace]/[name]
.
- Both
hostname
andnamespace
are optional for providers in thehashicorp
namespace in the Terraform Registry. - If
hostname
is omitted, Terraform will use the Terraform Registry hostname as the default hostname. namespace
is always required whenhostname
is set.
For third party Terraform providers, you can choose any arbitrary identifier (comprised of letters and hyphens) for the hostname
and namespace
.
When you run terraform init
, Terraform will attempt to download the provider from the Terraform Registry.
If Terraform can't download the provider from the Terraform Registry (for example if the provider is local, or because of firewall restrictions), you can specify the installation method configuration explicitly. Otherwise, Terraform will implicitly attempt to find the provider locally in the appropriate subdirectory within the user plugins directory, ~/.terraform.d/plugins/${host_name}/${namespace}/${type}/${version}/${target}
or %APPDATA%\terraform.d\plugins\${host_name}/${namespace}/${type}/${version}/${target}
.
Note For Terraform versions before v0.13.0, Terraform will attempt to locate the provider by binary name in the root user plugins directory.
In order to use any third-party provider not published to the registry, you must download the binary then move it into the appropriate subdirectory within the user plugins directory. Because HashiCups is a third-party provider used primarily for demos and education, you will do this. The provider source string for the HashiCups provider will be hashicorp.com/edu/hashicups
.
First, download the HashiCups provider binary. Because providers are Go binaries, you can download a pre-compiled binary or build it directly from source. Building from source requires some Go experience.
Download the HashiCups provider v0.3.1 binary that matches your system. A full list of binaries can be found on the Terraform HashiCups Provider release page. Rename the file to terraform-provider-hashicups
.
Download the HashiCups provider for MacOS.
Create the appropriate subdirectory within the user plugins directory for the HashiCups provider.
Then, unzip the downloaded binary into the appropriate user plugins directory.
Finally, make the binary executable.
Now that the provider is in your user plugins directory, you can use the provider in your Terraform configuration.
Create new HashiCups user
HashiCups requires a username and password to generate a JSON web token (JWT) which is used to authenticate against protected endpoints. You will use this user to authenticate to the HashiCups provider to manage your orders.
Create a user on HashiCups named education
with the password test123
.
Then, authenticate to HashiCups. This will return the userID, username, and a JSON Web Token (JWT) authorization token. You will use the JWT authorization token later to retrieve your orders.
Set the HASHICUPS_TOKEN
environment variable to the token you retrieved from invoking the /signin
endpoint. You will use this later in the tutorial to verify your HashiCups order has been created, updated and deleted.
The terminal containing your HashiCups logs will have recorded both operations.
Now that the HashiCups app is running, you're ready to interact with it using the Terraform provider.
Initialize workspace
Add the following to your main.tf
file. This is required for Terraform 0.13+.
Since HashiCups is a third-party provider, the hostname
and namespace
values in the source string are arbitrary. For more information on provider source, reference the provider source documentation.
Then, initialize your Terraform workspace. If your Hashicups provider is located in the correct directory, it should successfully initialize. Otherwise, move your HashiCups provider to the correct directory: ~/.terraform.d/plugins/${name}/${version}/${os}_${arch}
.
Create order
Add the following to your main.tf
file.
This authenticates the HashiCups provider, creates an order and returns the
order details as a Terraform output value. The order contains a total of 4
coffees: 2 of each coffee_id 3
and 2
.
Run terraform apply
to create the order. Notice how the execution plan shows a proposed order, with additional information about the order. Confirm the apply step with a yes
.
Once the apply completes, the provider saves the resource's state. If you're running Terraform locally, your terraform.tfstate
file contains this state. You can also view the state by running terraform state show <resource_name>
.
The (known after apply)
values in the execution plan during the terraform apply
step have all been populated, since the order was successfully created.
Verify order created
When you created an order in HashiCups using Terraform, the terminal containing your HashiCups logs will have recorded operations invoked by the HashiCups provider.
The provider invoked a total of 4 operations.
- The provider invoked the first
signin
operation when you ranterraform apply
to retrieve the current state of the resources. Because there are no resources, it only authenticated the user. - The provider invoked the second
signin
operation after you confirmed the apply run. The provider authenticated using the provided credentials to retrieve and save the JWT token. - The provider invoked the
CreateOrder
operation to create the order defined by the Terraform configuration. Since this is a protected endpoint, it used the saved JWT token from thesignin
operation. - After the order was created, the provider invoked the
GetUserOrder
operation to retrieve the order detail. Since this is a protected endpoint, it used the saved JWT token from thesignin
operation.
Verify the order was created by retrieving the order details via the API.
The order's properties should be the same as that of your hashicups_order.edu
resource.
Update order
Now, change your order by updating the order resource through Terraform. In your main.tf
, update the coffee quantities in the hashicups_order.edu
block.
Run terraform apply
to update the order. Notice how the execution plan reflects the order change. Terraform is able to determine that these changes could be done in place rather than destroying the current order and creating a new one. This is because the modified property (quantity
) does not have ForceNew
set to true
.
Confirm the apply step with a yes
.
Once the apply completes, view the output. Notice that while the order_id remained the same (1
), the provider updated the order's coffee quantity and assigned the current time to the newly created last_updated
field.
Verify order updated
Check the terminal containing your HashiCups logs for the recorded operations invoked by the HashiCups provider.
The provider invoked a total of 5 operations.
- The provider invoked the first
signin
operation when you ranterraform apply
to retrieve the current state of the resources. Because there are resources in the state... - The provider invoked the
GetUserOrder
operation to reconcile any potential differences between the current state (state in HashiCups) and the state stored by Terraform. - The provider invoked the second
signin
operation after Terraform's state was updated. - The provider invoked the
UpdateOrder
operation to update the order to the one defined by the Terraform configuration. Since this is a protected endpoint, it used the saved JWT token from thesignin
operation. - After the order was updated, the provider invoked the
GetUserOrder
operation to retrieve the order details. Since this is a protected endpoint, it used the saved JWT token from thesignin
operation.
Verify the order was updated by retrieving the order details via the API.
The order's properties should be the same as that of your updated hashicups_order.edu
resource. There should be 3
Nomadicano and 1
Vaulatte.
Read coffee ingredients
Add a data
block and an output
block to your main.tf
file. This will retrieve ingredients for the first coffee from your order. The data
block retrieves additional information about a resource, which enables it to be referenced by another Terraform resource. In this example, it's used by an output
block to display the coffee ingredients.
Run terraform apply
to retrieve the ingredients. Confirm the apply step with a yes
.
As you can see, ingredients of a Nomadicano is 20 ml espresso and 100 ml hot water.
Verify operation
Check the terminal containing your HashiCups logs for the recorded operations invoked by the HashiCups provider.
After the provider reconciles the state (first two operations), it invokes Handle Coffee Ingredients
which retrieves the coffee ingredients.
Delete order
Finally, you can delete resources in Terraform. Run terraform destroy
. Remember to confirm the destroy step with a yes
.
When the destroy step completes, the provider has destroyed the order resource, reflected by an empty state file.
Verify order deleted
Check the terminal containing your HashiCups logs for the recorded operations invoked by the HashiCups provider.
After the provider reconciles the state (first three operations), it destroys the order.
Verify the order was deleted by retrieving the order details via the API.
The order is blank, as expected.
Next steps
In this tutorial, you created, read, updated, and deleted an order resource in HashiCorp's demo application. In the process, you've learned how Terraform interacts with target APIs via providers.
A full list of official, partner, and community Terraform providers can be found on the Terraform Provider Registry. We encourage you to find a provider you're interested in and experiment!
To learn more about provider source, refer to Automatic Installation of Third-Party Providers with Terraform 0.13 blog post.
To learn more about interpolation, refer to the Configuration Language documentation.
In the following tutorials, you will learn how to write a Terraform provider using the Plugins SDK v2 by re-implementing the HashiCups provider. In the process, you will create data sources, authenticate the provider to the HashiCups client, and create resources with CRUD functionality.