Infrastructure-as-code and cloud provisioning
Terraform is an Infrastructure as Code (IaC) tool that enables you to manage and build your infrastructure using codified definitions instead of manual processes. With Terraform, you interact with various APIs through the use of providers, which are plugins that communicate instructions to create entities. Common examples of Providers include Azure, AWS, Google Cloud Platform, Kubernetes, and Oracle, while more unique examples include Akamai, Cloudflare, Cobbler/Foreman (for bare metal provisioning), and PrismaCloud.
As a coding language/DSL, Terraform offers flexibility, but it also benefits from prescriptive patterns that have emerged over time. This composition section presents dogmatic patterns that promote straightforward, scalable, and value-oriented consumption of Terraform. By following these patterns, you can leverage the full potential of Terraform and position yourself, your team, and your organization to adopt and derive value from HCP Terraform or Terraform Enterprise.
The purpose of this section is to provide you with a foundational understanding of the fundamental concepts and terminology. Familiarizing yourself with these concepts is crucial before diving into the composition recommendations and guidance.
Accelerated basics and definitions
Terraform infrastructure is defined as code written in HashiCorp Configuration Language (HCL) built on top of JSON. As a user, you author Terraform code in a text editor on your local machine and then use the Terraform binary to plan and apply that code to the target system.Terraform code is typically written in text files with the extension ".tf". One commonly used file is "main.tf," which serves as a central location to define all the entities you want to create. This file resides in a non-recursive directory, which acts as the fundamental container for your code moving forward.
When starting to write Terraform code, the first step is to instantiate one or more Providers. Providers are responsible for interacting with APIs and managing the lifecycle of the associated resources.The next step is to determine which Resources you want to codify with Terraform. A Resource Block describes one or more infrastructure objects, such as virtual networks, compute instances, or higher-level components like DNS records. By defining Resource Blocks in your code, you can precisely specify the desired state of your infrastructure.
While the file structure and items mentioned above are straightforward, personal preferences can influence decision-making. To ensure reusability, clarity, and standardization, consider the following guidelines:
- Naming
.tf
files: Determine a consistent naming convention for your.tf
files. - Resource ordering: Establish a preferred order for writing resources in your configuration files.
- Inputs and outputs: Define inputs and outputs in a clear and structured manner.
- Code reusability: Explore the use of modules to facilitate code reuse.
- Code sharing: Determine the process for sharing your code with others.
- Consumer usage: Document guidelines on how other consumers can effectively utilize your code.
When transitioning to a more mature development model, utilizing a Version Control Solution (VCS) becomes essential. A VCS enables source code management, collaboration, and streamlined review processes.This topic is covered below in this section. As the usage of Terraform expands to involve more developers, contributors, and reviewers, standardization becomes crucial. Fortunately, Terraform addresses these concerns inherently.
Terraform leverages a Directed Acyclic Graph (DAG) to establish resource relationships, allowing for parallel processing and simplified code structure. Consequently, the order of code execution becomes irrelevant to the underlying system. This approach offers numerous benefits:
- Resource instantiation order: Terraform handles the sequencing of resource instantiation.
- File organization: Terraform combines all
.tf
files within a directory, enabling the division of large Terraform files into smaller, more manageable logical files. For instance:main.tf
: Central location for Providers and custom logic.frontend.tf
: Contains frontend resource configurations and definitions.storage.tf
: Houses storage-related configurations.
Additionally, Terraform provides built-in support for input and output management through two conventionally named files:
variables.tf
: Contains variable blocks in alphabetical order.output.tf
: Includes output blocks in alphabetical order.
Organizing and structuring Terraform
Overview
Cloud computing provides new opportunities to deploy scalable applications in an efficient way, allowing enterprise applications to dynamically adjust their computing resources on demand. In this paper we analyze and test the microservice architecture pattern, used during the last years by large Internet companies like Amazon, Netflix and LinkedIn to deploy large applications in the cloud as a set of small services that can be developed, tested, deployed, scaled, operated and upgraded independently, allowing these companies to gain agility, reduce complexity and scale their applications in the cloud in a more efficient way.
The above citation is an abstract from an early research paper on the transition from monolithic enterprise application stacks to microservices. Terraform, as an Infrastructure as Code (IaC) tool, follows this paradigm shift towards microservices. When writing Terraform code, it is important to adhere to the same principles: highly scoped components, simplicity, composability, and pragmatism. This approach improves velocity, reduces impact, and enhances overall consumption. It aligns with the functions and features of HCP Terraform and Terraform Enterprise, maximizing the value of these solutions.
In the context of Terraform repositories, it is advisable to avoid monolithic repositories. Instead, follow a modular structure. An example of a single Terraform repository is provided below, where customizable naming conventions are denoted with <example>
:
<MyAppStack>
main.tf
: Contains provider definitions for the entire scope of the Terraform repository and calls to child modules or submodules.<component>.tf
: Individual files for components or resource types, often separated for better organization.variables.tf
: An alphabetical list of variables, including sensible defaults and descriptions.output.tf
: An alphabetical list of outputs that provide information for other utilities and Terraform consumption, similar to return values in programming languages.README.md
: A critical file that introduces the module's purpose and provides a comprehensive understanding of the Terraform code, usage, and variable descriptions.CHANGELOG.md
: A log file documenting changes made to the Terraform code. Refer to https://keepachangelog.com/ for recommendations on maintaining a changelog.backend.tf
: Defines where Terraform stores its state data files, which are used to track managed resources. Non-trivial configurations integrate with HCP Terraform or use a backend to store state remotely, enabling collaborative work on infrastructure resources.terraform.tf
: Describes the Terraform and provider versions compatible with each module./modules
: Directory for holding submodules./examples
: Directory showcasing working examples of how to use the module./tests
: Directory for containing tests for Terraform modules.
When organizing Terraform code files, it is recommended to follow a relatively linear order that aligns with the logical flow of the provisioning process. Although Terraform itself can execute in parallel, it's important to structure the code in a way that is easily understandable for humans. Here are some general best practices to consider:
- Define dependent resources before those that depend on them to ensure proper ordering.
- Place provider configurations at the top of the file to set the context for the resources.
- Include globally used items, such as randomly generated strings for naming or globally useful data sources, after the providers.
- Utilize locals to define and manage variables that are used within the file.
- Arrange the Terraform code files in a way that logically represents the building process.
- Group similar resources together, such as IAM policies for the same resources, while keeping distinct sections separate.
By organizing the code in a structured and logical manner, you can enhance readability and maintainability, making it easier to understand and manage your Terraform configurations.
Code formatting
The logical structuring of Terraform code is closely tied to adhering to Terraform's idiomatic style conventions, which promote consistency and readability. To ensure consistency across configuration files and modules, it is recommended to use automatic source code formatting tools like terraform fmt
. These tools apply the style conventions automatically, enabling you to maintain a clean and organized codebase. Below are the key style conventions for formatting your Terraform code:
- Indentation: Use two spaces for each nesting level.
- Curly Brace Placement: Place the closing brace of non-empty, multiline curly braces or square braces on its own line.
- Snake Case: Use Snake Case for terraform resource/datasource/local names.
- Alignment: Align equals signs for single-line arguments at the same nesting level.
Example:
resource "aws_instance" "example" {
count = 2 # Meta-argument first
ami = "abc123"
instance_type = "t2.micro"
network_interface {
# ...
}
lifecycle { # Meta-argument block last
create_before_destroy = true
}
}
- Block Separation: Separate top-level blocks with one blank line. Separate nested blocks with blank lines, except when grouping related blocks of the same type.
- Block Grouping: Avoid separating multiple blocks of the same type with different blocks, unless they form a defined family.
Comments are crucial for understanding complex code but should not be overly verbose. Terraform itself serves as documentation, so it's recommended to follow concise and clear comment guidelines. Below are the recommended guidelines for using comments:
Consider the audience when writing comments in your Terraform code. It is recommended to write code that is understandable to someone with a solid understanding of Terraform, but not necessarily complete mastery. Avoid sacrificing information in the comments by catering sole-ly to the least experienced readers.
Comments should provide additional details, such as the "why" behind certain decisions, rather than restating information already evident from the resource itself.
Maintain consistency in the types of comment delimiters used. The hash symbol (
#
) is preferred over double slashes (//
). Multiline comments (/* */
) should be used sparingly, primarily for longer comments or when commenting out resources.Metadata often requires comments to provide additional context. For example, lifecycle or count statements may require an explanation of their purpose.
Complicated dynamic code, such as loops and counts, almost always benefits from comments to aid understanding.
A well-written Terraform code should be understandable within a minute by someone proficient in Terraform but unfamiliar with the specific codebase.
Variables should be in a separate file, provide a sane default value and validate input with informative error messages
Introducing modules
To scale and optimize the consumption of Terraform, the necessary next step is to introduce 'components' of infrastructure called modules. Modules in Terraform serve a similar purpose to library functions in other programming languages. They provide consistent and standardized results for a specific component, ensuring uniformity across infrastructure deployments. Modules also contribute to efficiency by promoting standardization and enabling resource creation with predictable outcomes. At the same time, modules establish boundaries for customization, allowing users to control inputs and outputs within predefined limits. This approach strikes a balance between flexibility and stability, ensuring that infrastructure can be customized while maintaining overall consistency.
To ensure efficient management and reuse of resource configurations in Terraform, modules are used as containers for multiple resources that are designed to be used together. Understanding the different types of modules is essential. Please review the definitions below:
Module: A module consists of
.tf
or.tf.json
files organized within a directory and intended to be managed in version control. The directory may also contain subfolders for related artifacts such as*.ci
,.gitignore
, and*.md
files, used for Terraform code parsing, version control, or workflow.Root Module: The root module is constructed from the configuration files found in the current working directory when Terraform is executed. This module may reference child modules located in other directories or sourced remotely, allowing for the creation of a hierarchical structure.
Child Module: A child module is not typically run directly from the Terraform CLI but is instead consumed by another module. It focuses on completing a specific function to create a "component" used within the broader infrastructure configuration.
Generalized Child Module: A specialized type of child module designed for broad consumption. Well-written Terraform Public Registry modules often fall into this category. Generalized child modules provide a wider range of inputs and outputs and incorporate switching logic within the code to accommodate various use cases for the specific component.
In the world of programming, maturity can be compared to a large fast food franchise. Initially, a concept or small experiment, it eventually demands standardization and repeatability for scalability. Franchisees adhere to a strict contract model with robust supervision, aiming to serve customers the same product globally. Terraform naturally aligns with this pattern, providing exponential value. To structure your code and modules effectively, consider the following key questions:
- How are the boundaries of responsibility defined within your organization for owning specific components?
- Are there opportunities to break down barriers and build collaborative co-authoring teams for cross-functional requirements? For example, can a compute team, storage team, and web team come together to build a "full-stack" module?
- What opportunities exist in your organization for standardization that you can enforce through code? -Can you develop an optimized or express path to encourage further standardization?
- Which exceptions fall outside the scope of automation or are unnecessary to consider?
- What are the significant automation opportunities in your organization, and what are the small opportunities? Automating a "full-stack" deployment used by more than 30% of your company holds exponentially more value than automating a single application's unique stack.
Module best practices
The defining characteristic of a module is its encapsulation within a folder. However, it's important for you to evaluate the value and usefulness of creating a module, rather than doing so out of convenience. Consider the following fundamental principles of modules:
Define a Distinct Architectural Concept: Modules raise the level of abstraction by describing a new concept in your architecture, constructed from resource types offered by providers. For example, "acmecorp_standard_application_stack" could represent a suitable module concept.
Contains complex or business-specific requirements: When creating modules, they should cover multiple resources while staying focused and proportionate in scope. If there are complex or business-specific requirements, it's recommended to create modules for standardization. For example, an "aws_db_module" can address various tasks like managing backups, configuring CloudWatch monitoring, setting up alerts, log ingestion, and enabling Performance Insights. However, for cases where standardization is unnecessary, you can handle the requirements through policy implementation. Modules representing application stacks such as LAMP Stack, Landing Zone or Project Factory are good examples of useful modules. Conversely, modules encompassing the entire Quality Assurance environment, including Github accounts, Ingress/Egress rules, and all compute-related aspects, are considered excessive.
Embody Best Practices: Modules serve as a platform for enforcing naming conventions for cloud resources. They utilize variables to retrieve information required for constructing cloud resource names (or tags) in accordance with the naming convention. While best practices may be nebulous, subject to interpretation, and subject of debate, HashiCorp Validated Designs recommend specific patterns.
The primary focus of modules is to cater to the needs of the customer. It aims to provide them with the value of lightweight abstractions, considering factors such as impact radius, usability, and convenience. From the customer's perspective, they should only be concerned with providing the necessary inputs and receiving the desired outputs for operations, without having to worry about the internal workings of the module. On the other hand, as writers or operators of modules, you need to ensure that the module follows standard consumption practices, guarantees security and correct deployment of entities. As a business objective, modules should:
- Enable consumers to easily provide inputs and obtain desired outputs related to compute, storage, and networking access.
- Establish a standard and consumable entity for producers, auditors, and security.
- Instill confidence in consumers by offering a standardized component within the enterprise.
- Facilitate rapid development of new shared best practices and infrastructure.
- Separate production and consumption maintenance cycles, allowing for the creation of newer versions while maintaining a versioned object for consumers to utilize.
Module production
The Private Registry in HCP Terraform and Terraform Enterprise is a crucial component that allows organizations to create, manage, and share reusable modules within their environment. As we've previously discussed the hierarchical structure of organizations, projects, and workspaces, it's essential to understand how the private registry fits into this setup.
Within the confines of a single HCP Terraform and Terraform Enterprise Organization, the private registry enables teams to publish and consume modules. Organizations serve as the highest-level entity, encompassing multiple projects and their respective workspaces. Projects, on the other hand, provide a structured approach to organizing workspaces and controlling access to resources. Workspaces act as isolated environments where infrastructure code can be applied and managed independently.
The private registry comes into play within this organizational hierarchy, providing a way to share modules securely within an organization. As of now, the private registry's scope is limited to a single organization, meaning that modules can be shared between workspaces within the same organization. However, HCP Terraform does not support private registry module sharing across different organizations. For Terraform Enterprise users, administrators have the option to configure module sharing across organizations in the Enterprise version. This means that if you have multiple HCP Terraform and Terraform Enterprise Organizations and wish to leverage the private registry, a separate private registry will need to be created for each Organization, and each shared module must be individually published to each private registry.
While workspaces within a single HCP Terraform and Terraform Enterprise Organization can cross-reference or source modules from a private registry within the same organization, this process becomes streamlined if you have a supported Version Control System (VCS) connected to HCP Terraform and Terraform Enterprise. When module source code resides in a singular VCS repository, it becomes easier to manage and update the modules across private registry's.
In such a scenario, each module must be individually published into every private registry as a separate operation initially. Once this is done and provided that the VCS repositories are correctly tagged, the private registry's will automatically identify new versions of the module. However, if HCP Terraform and Terraform Enterprise is being used with an unsupported VCS, the publishing and updating of all private registry modules must be performed manually using the HCP Terraform and Terraform Enterprise API, which involves separate operations. This process introduces additional operational overhead.
Module production overview
In this section, we will establish the fundamental principles for module production within an enterprise. Determining the precise boundaries of modules can be challenging, and it is important to base this decision on business and organizational factors rather than solely on Terraform capabilities. To ensure clarity and manageability, Terraform code should have defined owners and lifecycles. When different parts of the organization handle related but distinct tasks, such as creating underlying networking versus compute resources, it is often beneficial to use separate modules. Below is a recap the two key considerations before proceeding:
Terraform consolidates all dependent folders, resources, and entities, creating an implicit connection within the codebase. This becomes especially evident when modules call other modules.
Code segmentation should align with an organization's separation of duties and prioritize readability and usability above all else.
To put the previously discussed best practices into action, you will explore a sample software stack called the Acme Application. Through this stack, you'll gain insights into optimizing various components following Terraform's best practices. The Acme Application consists of the following components:
- Virtual Private Cloud (VPC) with ingress and egress rules.
- Frontend compute infrastructure for hosting the web configuration.
- Middleware running on a compute instance.
- Backend utilizing a hosted database provided by a Cloud Service Provider (CSP).
As you review the components of the Acme Application, you can begin to apply your Terraform knowledge to identify areas that can be optimized and transformed into modules by taking into the following steps presented below.
The first component to evaluate is the Virtual Private Cloud (VPC):
Determine relevant background information: At Acme Corp, the architecture operates within a single VPC. As of now, there is no need to segregate applications into separate VPCs. Customization is achieved using variables for quality assurance, staging, and production environments. If other entities or components require information or data from the VPC (e.g., allocated IPs), they can access them through Remote State.
Ask the question(s): Does this VPC contain complex or business-driven requirements?
Make a decision: No, this VPC does not contain complex or business-driven requirements, and there is no need to modularize it.
The second component to evaluate is the compute for Acme Applications' frontend and middleware:
Determine relevant background information: At Acme Corp, the compute is customized primarily at runtime and uses a standardized golden image. Most of this configuration is done post-provisioning using a configuration management software. The underlying compute is typically modified via an inlet form from the customer requesting it (for example if someone requests an application, the standard Quality Assurance/Staging environment gets 4G of RAM and 2 CPUs, but Production gets double).
Ask the question(s): Does this component represent a distinct architectural concept?
Make a decision: No, these components do not represent a distinct enough requirement to modularize and change from a basic resource definition.
The final component to evaluate is the hosted data storage layer:
Determine relevant background information: At Acme Corp, their applications operate in a highly regulated and complex data-driven environment. The databases require stringent security measures and regular backups. Additionally, when new features are added to the Acme Application, staging data is copied to a lower environment for thorough developer testing before considering further deployments.
Ask the question(s): Does this component represent a distinct architectural concept? Does it encapsulate complex or business requirements? Does it embody best practices?
Make a decision: Yes, maybe. This component has rigorous requirements, including complex business needs, but it could potentially be standardized on a single resource. However, considering the constant reprovisioning into a testing environment and the self-service nature of Acme Corp's cloud requirements, modularizing this component could provide benefits.
Public module registry
If your goal is to understand the basics and kickstart your automation journey, this section will guide you through the final steps to achieve initial success. Terraform benefits from a thriving community with extensive support and resources. Currently, there are over 3000 published providers and 13,000 modules available. Leveraging existing modules and development patterns can provide a valuable shortcut to jumpstart the provisioning process for your organization. The Terraform Registry serves as a public platform where contributions from various users are shared and reviewed. It's worth noting that some modules in the marketplace have been utilized over 50,000,000 times. While it's important to exercise caution and possess a certain level of understanding when using community-contributed resources, they serve as an excellent starting point for organizations venturing into Infrastructure as Code and are highly recommended for initiating your automation journey.
To curate and mirror existing Terraform code from the public Terraform Registry, it is essential to analyze the quality of the code and verify its intended purpose. Follow these guidelines to evaluate whether the code meets basic quality standards and aligns with the desired outcomes.
- Evaluate the author:
- Check the "Partner content" checkbox in the Module Registry to filter for stricter requirements and maintenance cycles.
- Review the author's profile (e.g., Anton) and consider the number of downloads as an indicator of usage and quality.
- Take note of the author's affiliation and association (e.g., Eric) for certifications or involvement with reputable companies.
- Evaluate the module:
Assess if the module follows a consistent update cadence and adheres to proper semantic versioning.
- Review the number of downloads for the module, both within the current week, month, year, and overall. Compare these statistics with other modules of the same type.
- Look for a detailed and exhaustive Readme. A good Readme should provide clear documentation on the module's usage, inputs, outputs, dependencies, and resources, offering comprehensive guidance to end users.
- Check if the module includes examples. Examples provide detailed demonstrations on how to use the module effectively.
Once you've identified a suitable module for use, you can mirror the module internally with a private registry (more details on Terraform private registry can be found in the forthcoming Terraform Standardize HVD). This method of validating external modules and curating them for internal use can be an extremely useful way to start building a library of objects for consumption in your organization. While it doesn't develop the knowledge base in the same way as learning the rest of the composition best practices, it has to be acknowledged as one of the fastest ways to productivity using open source programming languages. For the lifecycle management of these modules and further best practices, please refer to the rest of this guide.
Terraform versioning concepts
Semantic versioning is a crucial concept in Terraform that was briefly introduced in the previous section. It provides a structured approach to versioning, ensuring compatibility and predictability when making changes to your Terraform configurations. By following semantic versioning principles, you can communicate the impact of your changes through version numbers:
- MAJOR version when making incompatible API changes
- MINOR version when adding functionality in a backwards compatible manner
- PATCH version when making backwards compatible bug fixes
Terraform supports semantic versioning for the binary, providers, and modules. It is crucial to evaluate compatibility between the Terraform binary and individual providers by referring to their compatibility matrix. For example, versions 4.0.0 and 4.x.x of the AWS Provider will be the last versions compatible with Terraform 0.12-0.15.
When upgrading the Terraform binary, provider versions, and module versions, it is important to avoid making code changes to the codebase (except for changes required to support the upgrade). Isolating changes to individual variables is necessary to prevent cross-contamination of issues. Additionally, it is recommended to review the changes notes while validating your upgrade strategy.
In general, it is advisable to use version constraints on major versions, which restrict large breaking changes to occur only in major versions. The compatibility hierarchy should follow the order of Terraform Version, Provider Version, and then Module Version. The visual representation is below:
The practice of branching in Version Control Solutions aligns well with this model. Additionally, speculative plans take advantage of this capability by allowing you to perform plan-only runs against specific branches. For example, you can create a new branch called application_a_v_newtfversion_xyz
and conduct a speculative run with the next version you intend to introduce.
Utilizing the Dependency Lock File extensively is crucial for successful large-scale upgrades of Terraform Providers. It is important to be aware of certain limitations regarding version constraints, which can be found here.
During the installation of required providers for a configuration, the terraform init
command considers both the version constraints specified in the configuration and the version selections recorded in the lock file.
By combining the dependency lock file with a standard branching solution in Version Control, it becomes safer and more straightforward to introduce major changes.
Cloud provisioning
With the foundations of Infrastructure as Code (IaC) established, you can now proceed to the next stage of this operating guide, which focuses on establishing a workflow for cloud provisioning. Terraform Enterprise provides you with a robust and efficient "golden cloud provisioning" workflow. To successfully provision resources in the cloud, follow these steps:
- Implement the "Landing Zone Provisioning Workflow": You will implement this workflow to ensure that infrastructure is provisioned in the appropriate "landing zones."
- Set up a Terraform Workspace with Cloud Credentials: Create a Terraform Workspace that includes the necessary cloud credentials. Alternatively, you can integrate with Vault using Workload Identity to dynamically manage cloud credentials.
- Store your Terraform Code in a Version Control System (VCS): Use a Version Control System repository to store your Terraform code. In the upcoming Pipeline section, we will discuss the two main alternatives for executing Terraform code.
The "Landing Zone Provisioning Workflow" is a crucial process that you implement for the various business units to ensure infrastructure provisioning in the appropriate "landing zones." Each cloud platform has its own recommendations for implementing these landing zones, and for Kubernetes-based teams, a Kubernetes namespace can also serve as a component of a landing zone. Key outputs of the Landing Zone workflow process include:
- VCS Repo - Used to store Terraform code
- Workspace
- Cloud Credentials stored in Workspace Variables
To achieve end-to-end cloud provisioning, developers will commit Terraform code to the designated repository, triggering a plan in both HCP Terraform and Terraform Enterprise workspaces. The workspaces will utilize cloud credentials, either stored directly within the workspace or integrated with Vault, to perform the Terraform plan. Once approved, the workspaces will apply the plan, provisioning the desired resources in the cloud.
However, before cloud provisioning can commence and Terraform workspaces can be provisioned with cloud credentials, it is crucial that the "Landing Zone Workflow" mentioned in earlier sections is fully implemented.
The Landing Zone Workflow is implemented by you, the platform team, in collaboration with partners in networking, security, and finance, adhering to the guidance provided by the cloud vendors' Well-Architected Framework. The Landing Zone must be implemented by your team before any application teams can initiate infrastructure provisioning in the cloud.
Cloud landing zones
A cloud landing zone is a foundational concept in cloud computing. It refers to an environment or infrastructure that is set up in the cloud for an organization to facilitate secure, scalable, and efficient cloud operations.
A cloud landing zone provides a secure and compliant environment for an organization's cloud resources, including networking components, identity and access management, security measures, and baseline configurations. This forms a blueprint for setting up workloads in the cloud and serves as a launching pad for broader cloud adoption. Cloud landing zones address several important areas:
- Networking: Configured with the network resources, such as Virtual Private Clouds (VPCs), subnets, and connectivity settings needed for operating workloads in the cloud. This includes setting up appropriate firewall rules, network access control lists, and routes to control inbound and outbound traffic.
- Identity and Access Management (IAM): Helps to enforce IAM policies, role-based access control (RBAC), and other permissions settings. This allows an organization to control who can access their cloud resources and what actions they can perform.
- Security and Compliance: Security configurations like encryption settings, security group rules, and logging settings. This helps ensure that an organization's cloud environment complies with internal policies and external regulations.
- Operations: Includes operational tools and services, such as monitoring, logging, and automation tools. These help to manage and optimize cloud resources effectively.
- Cost Management: Enables cost monitoring and optimization by facilitating tagging policies, budget alerts, and cost reporting tools.
Major cloud providers like Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP) offer solutions to help organizations create and manage landing zones. For instance, AWS Control Tower for creating a landing zone in AWS, Azure Landing Zone Accelerator, and Cloud Foundation Toolkit for GCP.
In essence, a cloud landing zone is a best practice strategy that ensures an organization's cloud journey starts from a secure, manageable, and well-governed foundation.
Secure variable storage
An important remark about the use of variables:
Warning
Terraform Workspaces contain a high level of privileged information and these are either in the state file, variables or their ability to access sensitive information from external secret management tools. It is possible for these sensitive information to be exfiltrated (by accident or otherwise) either by changing debug log level or using outputs. We recommend that Platform Teams use stringent RBAC controls to control this access.
Importance of variables
One of the the most obvious capabilities to explore when it comes to introducing variables in a secure fashion is Terraform Variable properties and precedence. Terraform Enterprise supports configuring Variables in many different ways (API, CLI, GUI) following many different security policies.
Variables are how terraform users can modify what terraform configuration code produces without having to make underlying changes to the terraform provider itself. Variable Sets are groups of variables created in Terraform that can contain both terraform and environment categories of variables. A variable set can be configured as global for an organization, that applies to all of the workspaces in an organization. Variable sets can also be reused by specified workspaces.
Secure Variables are crucial for storing access credentials used in the Terraform plan/apply lifecycle. These credentials include Cloud Credentials for provisioning, Admin passwords for initializing VMs and databases, and API keys/Tokens for third-party tools.
Considerations
- Consider using Variable sets Project level in HCP Terraform. Benefits are:
- Projects have granular access to who can view and manage the variable sets.
- Ease of administration instead maintaining long lists of variables defined individually selected workspaces.
- Sensitive variables can be defined at the project level and can be inherited to the all or specific workspaces if required.
- Leverage integration with Vault using "Workspace Identity" to enhance security and protection of sensitive data.
Workspace identity
With Dynamic Credentials, users are able to authenticate their Vault providers using OpenID Connect (OIDC) compliant identity tokens to provide just-in-time credentials and eliminate the need to store long lived secrets on HCP Terraform/Enterprise. Some of the benefits of Workspace Identity include;
- No more “cloud credentials”: Removes the need to store long-lived Cloud credentials in HCP Terraform/Enterprise. Temporary cloud credentials are instead retrieved from a Vault on the fly.
- No more secret rotation logic: Credentials issued by cloud platform are temporary and short-lived, removing the need to rotate secrets.
- Granular permissions control: Allows for using a cloud platform’s authentication and authorization tools to scope permissions based on HCP Terraform metadata such as a run’s phase, its workspace, or its organization.