IaC and cloud provisioning

HCP Terraform provides you with a robust and efficient cloud provisioning workflow. The choice of workflow is important, and engineering effort will be required to change it later. The high level points required to implement a cloud provisioning workflow are as follows.

1. Plan your project, covering installation and configuration of HCP Terraform or Terraform Enterprise, and how you intend to provide self-service capability to your internal customers. This is detailed further in the Terraform Operating Guide: Standardization HVD. Consideration now of how you will onboard, and work with, early-adopter teams, and later how to onboard gradually larger groups of teams until the platform is generally available will save a lot of time and effort.
2. Consider the size and bandwidth of your platform team. Consider what would happen to their workload if any of the onboarding experience of your internal customers was not fully automated.
3. Plan a landing zone provisioning workflow.
4. Ensure your platform team is fully trained on IaC with Terraform and the cloud provider(s) you are using. All contributors should be expected to read, understand, and adhere to the HashiCorp Terraform language style guide(opens in new tab). Facility in your documentation should be given to guiding all developers to write Terraform code that is compliant with the style guide.
5. Set up a Terraform workspace with cloud credentials.
6. Store your Terraform code in your strategic version control system.
7. Arrange for the IaC in the VCS to be run in the workspace in order to provision cloud resources.
8. Consider the configuration required for HCP Terraform and for the workspace therein used to enable the deployment end-to-end in the context of your organization's security and compliance requirements. Revisit each step in the workflow, and consider the requirements in terms of engineering content and effort, and focus on the manual steps and what you would need to do to automate each step. Also consider what would be necessary in a disaster recovery scenario.
9. If your deployment is intended to scale to a large number of teams, ensure to read the respective HVDs in this series.

The landing zone provisioning workflow is a crucial process that you implement for your application or development teams to ensure infrastructure provisioning in the appropriate landing zones. Each cloud platform has its own recommendations for implementing these landing zones, and for Kubernetes-based teams, a Kubernetes namespace can also serve as a component of a landing zone.

To achieve end-to-end cloud provisioning, developers will commit Terraform code to the designated repository, triggering a plan in HCP Terraform workspaces which will utilize cloud credentials to complete the plan phase. Once approved, the workspaces will apply the plan, provisioning the desired resources in the cloud.

The landing zone workflow should be implemented and owned by the platform team, partnering the networking, security, and finance teams, adhering to the guidance provided by the cloud vendors' well-architected framework. The landing zone must be implemented by your team before any application teams can initiate infrastructure provisioning in the cloud.

Landing zones

A cloud landing zone is a foundational concept in cloud computing and a common deployment pattern for scaled customers. Defined as a standardized environment deployed in a cloud to facilitate secure, scalable, and efficient cloud operations, it forms a blueprint for setting up workloads in the cloud and serves as a launching pad for broader cloud adoption.

Cloud landing zones address several important areas:

• Networking: Configured with the network resources, such as virtual private clouds, subnets, and connectivity settings needed for operating workloads in the cloud.
• Identity and access management: Helps to enforce IAM policies, RBAC, and other permissions settings.
• Security and compliance: Security configurations such as encryption settings, security group rules, and logging settings.
• Operations: This includes operational tools and services, such as monitoring, logging, and automation tools. These help to manage and optimize cloud resources effectively.
• Cost management: Enables cost monitoring and optimization by facilitating automation, tagging policies, budget alerts, and cost reporting tools.

Major cloud providers offer their own solutions for this approach:

• AWS Control Tower
• Azure Landing Zone Accelerator
• Cloud Foundation Toolkit for GCP.

Landing zones are commonly deployed by HCP Terraform, but the landing zone concept is also applicable to HCP Terraform by which a management workspace is deployed by the Platform Team during the onboarding of an internal customer using automation and which is then used to deploy the operational environment needed by that customer to support their development remit; this is akin to the equivalent CSP account and configuration needed by the same team. The recommended way to do this is:

• Complete preparative reading: Read about cloud landing zone patterns applicable to the CSPs with whom work partner.
• Define the HCP Terraform landing zone: There are a number of ways to apply the LZ concept, but core requirements include:
  • Use of the Terraform Enterprise provider(opens in new tab) so that there is state representation for the configuration - this applies to both HCP Terraform and Terraform Enterprise.
  • Definition of a VCS template which contains boilerplate Terraform code for the LZ and a directory structure designed and managed by the Platform Team.
  • Creation of a Terraform module for your private registry - this will be called by the Platform Team's automated onboarding process when an application team wants to onboard. This module should nominally create the following:
    • A management workspace for the application team.
    • A VCS repository for the application team to store the code which will run in the management workspace. This will use the VCS repository template above.
    • Variables/sets as needed for efficient deployment.
    • Public and/or private cloud resources for the team to use as necessary.
  • Augment your onboarding pipeline to make use of the module. The process should hook into other organizational platforms for credential generation and observability etc.
  • Dedicate a HCP Terraform project to house the landing zone management workspaces separately from others the Platform Team own. Use the Terraform Enterprise provider or HCP Terraform API to deploy this. The onboarding process will direct creation of management workspaces into this project.
• Test: Create landing zones using the process
• Update the module and VCS template: Expect to do this a number of times before you are satisfied that the process scales. This is a key area which must not be underestimated. Depending on the onboarding design, if you onboard ten internal customers using the process, then need to change the process, this will apply from that point onwards for the next set of customers, but depending on the implementation, those already deployed may need to be updated manually such that standardization can be applied across all customers. This is also why it is important to funnel sets of early adopters onto the platform as you develop the solution.
• Ensure smooth onboarding: The process should be as smooth as possible for the application teams. If using Service Now as front door to the onboarding process, the Platform Team should liaise with the Service Now team as early as possible as there will be a lead time for remediation.

Landing zone workflow

Via the tasks above (irrespective of onboarding platform technology), the workflow can be summarized as follows:

• Ticket raised: Audit trail created, approval acquired as appropriate.
• Landing zone child module call code added: The top-level workspace and its VCS repository is where the Platform Team collect and manage onboarded teams.
  • We recommend automating the addition of this child module call in order to ensure the solution is scalable. We expect your strategic orchestration technology would be used as part of this automation.
  • If your business expects to onboard thousands of teams, then planning of the Platform Team landing zone deployment workspace should be handled more carefully to avoid performance issues with the workspace run as discussed at the end of the previous section.
• Run the top-level workspace: This will create the management workspace for the application team using the Terraform landing zone module.
• Application team management workspace created: The template VCS should contain Terraform code to use child modules to create the application team's workspaces. In a franchise model, the application team would have write access to this repository so that they can configure it to deploy all the required workspaces to support their needs. In a service catalog model, the management workspace would then also be run to create the application team's workspaces to hook into subsequent request processes for pre-canned infrastructure requests.

Workload identity

With dynamic credentials, users are able to authenticate their Vault providers using OpenID Connect (OIDC) compliant identity tokens to provide just-in-time credentials and eliminate the need to store long lived secrets on HCP Terraform/Enterprise. Some of the benefits of workload identity include the following.

• No more cloud credentials: Removes the need to store long-lived cloud credentials in HCP Terraform. Temporary cloud credentials are instead retrieved either from the CSP or via Vault on the fly.
• No more secret rotation logic: Credentials issued by cloud platform are temporary, removing the need to rotate secrets.
• Granular permissions control: Allows for using authentication and authorization tools from the cloud service provider to scope permissions based on HCP Terraform metadata such as run phase, workspace, or organization.

We recommend the implementation of workload identity to ensure that your cloud provisioning workflow adopts the most secure posture possible at all times. You will likely start by using fixed cloud credentials in order to authenicate your provider configuration and this is expected. Workflow identity should be part of your plans as the project proceeds and should be well-understood and fully-implemented prior to production deployment.