Nomad installation

You can deploy Terraform Enterprise to HashiCorp Nomad, which lets you control the deployment scaling of the application. Because Nomad and Terraform are HashiCorp products, you do not need not to depend on third-party tools and their support. Nomad has a lower learning curve compared to Kubernetes and is easier to manage.

Nomad in beta

Running Terraform Enterprise on Nomad is in beta. Do not deploy beta features in production environments. Provision a dedicated test environment before installing Terraform Enterprise on Nomad. If you have questions or feedback about the Nomad beta, contact your HashiCorp account representative.

Complete the following steps to deploy Terraform Enterprise to Nomad-orchestrated containers:

Parameterize the Terraform Enterprise license, host, and TLS encryption settings by adding Nomad variables to your job specifications. This enable you to use the same job specification with different configurations. Refer to Job Specification in the Nomad documentation for additional information.
Add Terraform Enterprise environment variables to your Nomad job specification to configure Terraform behavior. Refer to the Terraform Enterprise configuration reference for additional information.
Create a Nomad job specification for operating the Terraform Enterprise agent. Refer to Custom Worker Image for additional information about the Terraform Enterprise agent.
Run the Nomad command for pulling the Terraform Enterprise image and installing the binary.

Requirements

Before you begin, ensure you meet the Nomad requirements for installing Terraform Enterprise on Nomad.

Parameterize Terraform Enterprise settings

Add the following variables to your Terraform Enterprise Nomad job:

tfe_license: Specifies the Terraform Enterprise license key.
tfe_hostname: Specifies the hostname of the Terraform Enterprise instance.
tfe_tls_cert_file: Specifies the base64 encoded TLS certificate file.
tfe_tls_key_file: Specifies the base64 encoded TLS key file.
tfe_tls_ca_bundle_file: Specifies the base64 encoded TLS CA bundle file.

Create file var.hcl and add the following variables:

  path      = "nomad/jobs/tfe-job/tfe-group/tfe-task"
  namespace = "terraform-enterprise"

  items {
    # The field should contain the base64 encoded value of the cert. Mappped to the TFE_TLS_CERT_FILE environment variable.
    cert = ""

    # The field should contain the base64 encoded value of the bundle. Mapped to the TFE_TLS_CA_BUNDLE_FILE environment variable.
    bundle = ""

    # The field should contain the base64 encoded value of the key. Mappped to the TFE_TLS_KEY_FILE environment variable.
    key = ""

    # A valid TFE license. Mapped to the TFE_LICENSE environment variable.
    tfe_license = ""

    # The hostname of the TFE instance. Mapped to the TFE_HOSTNAME environment variable.
    tfe_hostname = ""
  }

path variable specifies the path where the Nomad variables will be stored. Update the path variable if default value of job_name is overridden in the var.hcl file.

Apply the Nomad variables to the job specification by running the following command:

$ nomad var put @var.hcl

Refer to Nomad Variables in the Nomad documentation for additional information.

Configure Terraform Enterprise Nomad job specification

This job is responsible for running the Terraform Enterprise image on Nomad.

Refer to the example Nomad job specification for TFE for a template that you can copy and modify.

Pass the variables that you defined in the Parameterize Terraform Enterprise settings section. Refer to Assigning Values to job Variables in the Nomad documentation for instructions. The following variables are required:

The following variables are optional:

Configure a Nomad batch job to run the Terraform Enterprise agent

Create a Nomad job specification that defines the behavior of Terraform Enterprise agent running on Nomad. Configure the job as a Nomad batch job so that Nomad starts a new agent job each time you run the terraform plan or terraform apply command. You do not need to run batch jobs manually after creating them. Refer to Batch Job in the Nomad documentation for additional information. Refer to the example Nomad batch job specification for TFE agent for a template that you can copy and modify.

Refer to Batch Job in the Nomad documentation for additional information. Refer to the example Nomad batch job specification for TFE agent for a template that you can copy and modify.

Run the Nomad jobs

Run the nomad run command to pull the Terraform Enterprise image and install the application. Pass the Terraform Enterprise job specification as the command argument. You must also provide the credentials for the registry to download the image:

$ nomad run -var="tfe_image_username=$TFE_REGISTRY_USERNAME" -var="tfe_image_password=$TFE_REGISTRY_PASSWORD" <path-to-tfe-job-spec>

Run the nomad run command and pass the Terraform agent job specification to register the batch job in Nomad :

$ nomad run <path-to-tfe-agent-job-spec>

Alternatively, you can pull and install the Terraform Enterprise image using the Terraform Enterprise On Nomad Pack tool. Refer to the terraform-enterprise-fdo-nomad-pack repository on GitHub for instructions.

Create initial admin user

Provision your first administrative user and start using Terraform Enterprise.

Examples

You can copy the following examples and modify the values to match your deployment.

Nomad job specification for TFE

The following example configuration defines a Terraform Enterprise job specification. You can copy the example and modify the values to match your deployment.

variable "tfe_image" {
  description = "The TFE image to use"
  type        = string
  default     = "images.releases.hashicorp.com/hashicorp/terraform-enterprise:v202406-1"
}

variable "tfe_image_username" {
  description = "Username for the registry to download TFE image"
  type        = string
}

variable "tfe_image_password" {
  description = "Password for the registry to download TFE image"
  type        = string
}

variable "namespace" {
  description = "The Nomad namespace to run the job"
  type        = string
  default     = ""
}

job "tfe-job" {
  datacenters = ["dc1"]
  namespace   = var.namespace
  type        = "service"

  group "tfe-group" {
    count = 1
    network {
      port "tfe" {
        static = 443
      }
      port "vault" {
        static = 8201
      }
    }

    service {
      name     = "tfe-svc"
      port     = "tfe"
      provider = "nomad"
      check {
        name     = "tfe_probe"
        type     = "http"
        protocol = "https"
        port     = "tfe"
        path     = "/_health_check"
        interval = "5s"
        timeout  = "2s"
        method   = "GET"
      }
    }

    task "tfe-task" {
      driver = "docker"

      identity {
        # Expose Workload Identity in NOMAD_TOKEN env var
        env = true
      }

      template {
        data        = <<EOF
              {{- with nomadVar "nomad/jobs/tfe-job/tfe-group/tfe-task" -}}
              TFE_LICENSE={{ .tfe_license }}
              TFE_HOSTNAME={{ .tfe_hostname }}
              {{- end -}}
              EOF
        destination = "secrets/env.env"
        env         = true
        change_mode = "restart"
      }

      template {
        data        = <<EOF
              {{- with nomadVar "nomad/jobs/tfe-job/tfe-group/tfe-task" -}}
              {{ base64Decode .tfe_tls_cert_file.Value }}
              {{- end -}}
              EOF
        destination = "secrets/cert.pem"
        env         = false
        change_mode = "restart"
      }

      template {
        data        = <<EOF
              {{- with nomadVar "nomad/jobs/tfe-job/tfe-group/tfe-task" -}}
              {{ base64Decode .tfe_tls_key_file.Value }}
              {{- end -}}
              EOF
        destination = "secrets/key.pem"
        env         = false
        change_mode = "restart"
      }

      template {
        data        = <<EOF
              {{- with nomadVar "nomad/jobs/tfe-job/tfe-group/tfe-task" -}}
              {{ base64Decode .tfe_tls_ca_bundle_file.Value }}
              {{- end -}}
              EOF
        destination = "secrets/bundle.pem"
        env         = false
        change_mode = "restart"
      }

      config {
        image = var.tfe_image
        ports = ["tfe", "vault"]

        auth {
          username = var.tfe_image_username
          password = var.tfe_image_password
        }

        volumes = [
          "secrets:/etc/ssl/private/terraform-enterprise",
        ]
      }

      resources {
        cpu    = 2500
        memory = 2048
      }

      env {
        TFE_DATABASE_HOST       = "<Database hostname and port e.g. postgres:5432>"
        TFE_DATABASE_USER       = "<Database user e.g. postgres>"
        TFE_DATABASE_PASSWORD   = "<Database password e.g. postgres>"
        TFE_DATABASE_NAME       = "<Database name e.g. hashicorp>"
        TFE_DATABASE_PARAMETERS = "<Database parameters e.g. sslmode=disable>"

        TFE_OBJECT_STORAGE_S3_ENDPOINT          = "<S3 hostname and port e.g. localhost:9000>"
        TFE_OBJECT_STORAGE_TYPE                 = "s3"
        TFE_OBJECT_STORAGE_S3_ACCESS_KEY_ID     = "<AWS Access Key ID>"
        TFE_OBJECT_STORAGE_S3_SECRET_ACCESS_KEY = "<AWS Secret Access Key>"
        TFE_OBJECT_STORAGE_S3_REGION            = "<AWS Region e.g.us-east-1>"
        TFE_OBJECT_STORAGE_S3_BUCKET            = "<Bucket name>"

        TFE_REDIS_HOST     = "<Redis hostname and port e.g. redis:6379>"
        TFE_REDIS_USER     = "<Redis username>"
        TFE_REDIS_PASSWORD = "<Redis password>"
        TFE_REDIS_USE_TLS  = "<To use tls? e.g. false>"
        TFE_REDIS_USE_AUTH = "<To use customized credential to authenticate? e.g. false>"

        TFE_RUN_PIPELINE_NOMAD_ADDRESS                = "${NOMAD_ADDR}"
        TFE_RUN_PIPELINE_NOMAD_TLS_CONFIG_CA_CERT     = "<path to CA certificate for mTLS/TLS communication with Nomad>"
        TFE_RUN_PIPELINE_NOMAD_TLS_CONFIG_CLIENT_CERT = "<path to Client certificate for mTLS/TLS communication with Nomad>"
        TFE_RUN_PIPELINE_NOMAD_TLS_CONFIG_CLIENT_KEY  = "<path to Client certificate's key for mTLS/TLS communication with Nomad>"
        TFE_RUN_PIPELINE_NOMAD_TLS_CONFIG_INSECURE    = "<if set SSL is disabled>"
        TFE_RUN_PIPELINE_DRIVER                       = "nomad"
        TFE_VAULT_DISABLE_MLOCK                       = "true"
        TFE_ENCRYPTION_PASSWORD                       = "<Encryption password>"
        TFE_OPERATIONAL_MODE                          = "active-active"

        # If you are using the default internal vault, this should be the private routable IP address of the node itself.
        TFE_VAULT_CLUSTER_ADDRESS = "http://${NOMAD_HOST_ADDR_vault}"

        TFE_TLS_CERT_FILE      = "/etc/ssl/private/terraform-enterprise/cert.pem"
        TFE_TLS_KEY_FILE       = "/etc/ssl/private/terraform-enterprise/key.pem"
        TFE_TLS_CA_BUNDLE_FILE = "/etc/ssl/private/terraform-enterprise/bundle.pem"
      }
    }
  }
}

Nomad batch job specification for TFE agent

The following example configuration defines a Terraform Enterprise agent job. You can copy the example and modify the values to match your deployment. You may use the TFE_RUN_PIPELINE_NOMAD_AGENT_JOB_ID configuration to specify a custom job ID for the agent job.

job "tfe-agent-job" {
  type        = "batch"
  namespace   = "tfe-agents"
  datacenters = ["dc1"]
  node_pool   = "node_pool_tfe_agents"

  constraint {
    attribute = "${attr.kernel.name}"
    value     = "linux"
  }

  parameterized {
    payload = "forbidden"

    meta_required = [
      "TFC_AGENT_TOKEN",
      "TFC_ADDRESS"
    ]

    meta_optional = [
      "TFE_RUN_PIPELINE_IMAGE",
      "TFC_AGENT_AUTO_UPDATE",
      "TFC_AGENT_CACHE_DIR",
      "TFC_AGENT_SINGLE",
      "HTTPS_PROXY",
      "HTTP_PROXY",
      "NO_PROXY"
    ]
  }

  group "tfe-agent-group" {
    task "tfc-agent-task" {

      driver = "docker"

      template {
        destination = "local/image.env"
        env         = true
        change_mode = "noop"
        data        = <<EOF
          {{ $image := env "NOMAD_META_TFE_RUN_PIPELINE_IMAGE" }}
          {{ if ne $image "" }}TFE_RUN_PIPELINE_IMAGE={{$image}} {{ else }}TFE_RUN_PIPELINE_IMAGE="hashicorp/tfc-agent:latest"  {{ end }}
          EOF
      }

      config {
        image = "${TFE_RUN_PIPELINE_IMAGE}"
      }

      env {
        TFC_ADDRESS           = "${NOMAD_META_TFC_ADDRESS}"
        TFC_AGENT_TOKEN       = "${NOMAD_META_TFC_AGENT_TOKEN}"
        TFC_AGENT_AUTO_UPDATE = "${NOMAD_META_TFC_AGENT_AUTO_UPDATE}"
        TFC_AGENT_CACHE_DIR   = "${NOMAD_META_TFC_AGENT_CACHE_DIR}"
        TFC_AGENT_SINGLE      = "${NOMAD_META_TFC_AGENT_SINGLE}"
        HTTPS_PROXY           = "${NOMAD_META_HTTPS_PROXY}"
        HTTP_PROXY            = "${NOMAD_META_HTTP_PROXY}"
        NO_PROXY              = "${NOMAD_META_NO_PROXY}"
      }

      resources {
        cpu    = 500
        memory = 2048
      }
    }
  }
}