Terraform
Deploy a Kubernetes cluster and application
Terraform Stacks allow you to automatically manage dependencies within complex infrastructure deployments. In addition to explicit orchestration rules, a Stack recognizes when a component requires attributes that are not yet available, and HCP Terraform defers those changes until it can apply them. For example, when managing Kubernetes workloads that use custom resources, you cannot provision both the custom resource definition (CRD) APIs and resources that use those APIs in a single step, because the Terraform provider relies on the existence of the APIs to create and apply your plan. In these scenarios, you must first deploy your CRD, and then deploy the resources that use it in a second plan and apply, usually in a separate workspace.
When you deploy a Stack that includes resources that depend on the availability of APIs provisioned by other components in your Stack, HCP Terraform recognizes the dependency between components, and automatically defers the plan and apply steps for your components until they can complete successfully.
In this tutorial, you will use an HCP Terraform Stack to deploy a Kubernetes cluster with an example service. When you deploy your Stack, HCP Terraform will automatically detect that the service you are deploying on your Kubernetes cluster needs attributes that aren't available during the initial plan. HCP Terraform will defer those changes until the custom resource definition is created and the API becomes available. Then, HCP Terraform will load the attribute values and proceed with the plan and apply steps for your service.
Prerequisites
This tutorial assumes that you are familiar with the Terraform and HCP Terraform workflows. If you are new to Terraform, complete the Get Started collection first. If you are new to HCP Terraform, complete the HCP Terraform Get Started tutorials first.
In order to complete this tutorial, you will need the following:
- An AWS account.
- An HCP Terraform account.
- An HCP Terraform variable set configured with your AWS credentials.
Create example repository
Navigate to the template
repository
for this tutorial. Click the Use this template button and select Create a
new repository. Choose a GitHub account to create the repository in and name
the new repository learn-terraform-stacks-eks-deferred
. Leave the rest of the
settings at their default values.
Clone your example repository, replacing USER
with your own GitHub username.
$ git clone https://github.com/USER/learn-terraform-stacks-eks-deferred.git
Change to the repository directory.
$ cd learn-terraform-stacks-eks-deferred
Review components and deployment
Explore the example configuration to review how this Terraform Stack's configuration is organized.
$ tree
.
├── CODEOWNERS
├── LICENSE
├── README.md
├── cluster
│ ├── iam.tf
│ ├── main.tf
│ ├── outputs.tf
│ ├── variables.tf
│ └── vpc.tf
├── components.tfstack.hcl
├── copywrite.hcl
├── deployments.tfdeploy.hcl
├── kube
│ ├── crd.tf
│ └── kube.tf
├── providers.tfstack.hcl
└── variables.tfstack.hcl
The example repository contains two directories containing Terrraform modules to
define the Stack’s components, kube
, and cluster
. It also contains
.tfstack.hcl
files to define your Stack, and a deployments.tfdeploy.hcl
to
define your Stack’s deployment.
Review components
Open the providers.tfstack.hcl
file. This file contains the provider
configuration for your Stack.
providers.tfstack.hcl
provider "aws" "main" {
config {
region = var.region
assume_role_with_web_identity {
role_arn = var.role_arn
web_identity_token = var.identity_token
}
default_tags {
tags = var.default_tags
}
}
}
provider "kubernetes" "main" {
config {
host = component.cluster.cluster_url
cluster_ca_certificate = component.cluster.cluster_ca
token = component.cluster.cluster_token
}
}
provider "random" "main" {}
This configuration file defines provider blocks for AWS and Kubernetes. The AWS provider block sets the role ARN and web identity token that your AWS provider will use to authenticate with AWS. Each deployment can configure a different role ARN for its instance of the Stack.
The Kubernetes provider configuration uses output values from the cluster
component, which provisions your Kubernetes cluster on AWS. Since HCP Terraform
will not know about these values until after it plans and applies cluster
component’s configuration, HCP Terraform automatically defers initialization of
your Kubernetes provider until after those values become available. Then, it
plans and applies the configuration that relies on the Kubernetes provider. This
process is similar to the way Terraform automatically determines dependencies
manages dependencies within a given Stack even across multiple plan/apply steps.
between resources and provisions them in the correct order. Stacks, however,
Next, review the components.tfstack.hcl
file. This file contains all of the
components for your Stack. Like Terraform configuration, you can organize your
Stacks configuration into multiple files without affecting the resulting
infrastructure.
components.tfstack.hcl
component "cluster" {
source = "./cluster"
providers = {
aws = provider.aws.main
random = provider.random.main
}
inputs = {
cluster_name = var.cluster_name
kubernetes_version = var.kubernetes_version
region = var.region
}
}
component "kube" {
source = "./kube"
providers = {
kubernetes = provider.kubernetes.main
}
}
This file includes configuration for two components, one for the EKS cluster itself, and another for the example API for this tutorial.
The configuration in the cluster
directory defines an EKS cluster
Review deployment
deployments.tfdeploy.hcl
identity_token "aws" {
audience = ["aws.workload.identity"]
}
deployment "development" {
inputs = {
cluster_name = "stacks-demo"
kubernetes_version = "1.30"
region = "us-east-1"
role_arn = "<YOUR_ROLE_ARN>"
identity_token = identity_token.aws.jwt
default_tags = { stacks-preview-example = "eks-deferred-stack" }
}
}
This Stack includes a single deployment of your Kubernetes cluster on AWS.
Replace the <YOUR_ROLE_ARN>
for both deployments with the role_arn
output
value from your identity token workspace.
Commit the role ARN to your git repository.
$ git add deployments.tfdeploy.hcl && git commit -m "Configure role ARN"
Push the change to Github.
$ git push
Note
For this tutorial, we are configuring the same role ARN for each deployment, but in a real-world setting, you should consider using a different role for each deployment, with appropriately scoped permissions.
Fork identity token repository
Your Stack uses an identity token to authenticate the AWS provider in each region. To do so, you will establish a trust relationship between your AWS account and HCP Terraform, and an AWS role with the appropriate permissions to create and manage your Stack.
Navigate to the template
repository
for the identity token. Click the Use this template button and select
Create a new repository. Choose a GitHub account to create the repository in
and name the new repository learn-terraform-stacks-identity-tokens
. Leave the
rest of the settings at their default values.
Note
If you have already created a repository based off of this template, you can re-use your existing repository for this tutorial.
Create a project
Create an HCP Terraform project for your identity token workspace and Kubernetes Stack.
To do so, first log in to HCP Terraform, and select the organization you wish to use for this tutorial.
Next, ensure that Stacks is enabled for your organization by navigating to
Settings > General. Ensure that the box next to Stacks
is checked, and
click the Update organization` button.
Then navigate to Projects, click the + New Project button, name your
project Learn Terraform Stacks deferred actions
, and click the Create
button to create it.
Next, ensure that your AWS credentials variable set is configured for your project. Navigate back to < Projects, then to your organization’s **Settings
Variable sets page, and select your AWS credentials variable set. Under Variable set scope, select Apply to specific projects and workspaces, and add your
Learn Terraform Stacks deferred actions
project to the list under Apply to projects. Scroll to the bottom of the page and click the Save variable set** button to apply it to your new project.
Provision and set identity token
Next, provision the AWS identity token and role that HCP Terraform will use to authenticate with AWS when it deploys your Stack.
Navigate to Projects and select your Learn Terraform Stacks deployments
project. Create a workspace to provision your project's identity tokens:
- On the project overview page, select New > Workspace.
- On the next screen, select Version Control Workflow and select your GitHub account.
- On the Choose a repository page, select the
learn-terraform-stacks-identity-tokens
repository you created in the previous step. - On the next page, select Advanced options and enter
aws/
in the Terraform Working Directory field. - Scroll to the bottom of the page and click the Create button to create your identity token workspace.
Once you create the workspace, HCP Terraform will load the configuration for
your identity token. This process may take a few seconds to complete. Once it
does, HCP Terraform will prompt you to enter values for your organization and
project. Enter your organization name and the project name for this tutorial,
Learn Terraform Stacks deferred actions
and click the Save variables
button.
Next, click the Start new plan button, and then Sart, to plan your changes. HCP Terraform will plan your changes, and prompt you to apply them. Once the plan is complete, click the Confirm and apply button to create your OpenID provider, policy, and role.
Make a copy of the role_arn
output value. You will use it in the next step.
Set role ARN
In your Stack’s configuration on your local machine, replace the
<YOUR_ROLE_ARN>
your deployment with the role_arn
output value from your
identity token workspace.
deployments.tfdeploy.hcl
deployment "development" {
inputs = {
cluster_name = "stacks-demo"
kubernetes_version = "1.30"
region = "us-east-1"
role_arn = "<YOUR_ROLE_ARN>"
identity_token = identity_token.aws.jwt
default_tags = { stacks-preview-example = "eks-deferred-stack" }
}
}
Commit the role ARN to your git repository.
$ git add deployments.tfdeploy.hcl && git commit -m "Configure role ARN"
Push the change to Github
$ git push
Create Stack
Return to your project by navigating to Projects
and selecting your Learn
Terraform Stacks deferred actions
project. Select Stacks
from the left nav
and click + New Stack
.
On the Connect to VCS
page, select your GitHub account. Then, choose the
repository you created for this tutorial, learn-terraform-stacks-eks-deferred
.
On the next page, leave your Stack name the same as your repository name, and
click Create Stack
to create it.
HCP Terraform will load your Stack’s configuration from your VCS repository.
Provision cluster and example services
Once HCP Terraform loads your configuration, it will plan your changes. Select
your development
deployment to review the plan status. HCP Terraform will
notify you that the plan includes deferred changes. You can review the plan to
identify which parts of your Stack are parts of the deferred changes. Click
Approve plan to apply your changes. Once the plan is complete, you can
review the resources that HCP Terraform will create, and apply the plan.
Note
It may take a few minutes to provision the EKS cluster.
As it provisions your infrastructure, HCP Terraform will automatically detect that the API you are deploying on your Kubernetes cluster is waiting on the other Kubernetes components to deploy successfully, and will defer those changes until the components are available. Then it will load the attribute values and proceed with the plan.
Destroy infrastructure
Now you have used a Terraform Stack to deploy a Kubernetes cluster with an API. HCP Terraform was able to deploy the entire Stack as a single unit, despite the dependency between the API and the Kubernetes cluster.
Before finishing this tutorial, destroy your infrastructure.
Navigate to your Stack's Deployments page, and select the development
deployment. Navigate to the Destruction and Deletion page and click the
Create destroy plan button to create a destroy plan. Once the destroy plan
is complete, approve it to remove your resources.
Then, remove your Stack by navigating back to your Stack and selecting your
Stack's Destruction and Deletion page, and clicking the Force delete Stack
learn-terraform-stacks-deploy button. Confirm the action, and click the
Delete
button. Finally, navigate to your
learn-terraform-stacks-identity-token
workspace’s Settings > Destruction and
Deletion page, and follow the prompts to destroy your identity token
infrastructure and delete the workspace. You can also remove your project by
navigating to its settings page and following the steps to delete it.
Next steps
In addition to allowing you to manage complex deployments with hidden dependencies, Terraform Stacks include power orchestration and workflow features.
- Read the Terraform Stacks documentation for more details on Stacks features and workflow.
- Read the Terraform stacks, explained blog post.