Migrate to HCP Terraform Operator for Kubernetes v2

To upgrade the HCP Terraform Operator for Kubernetes from version 1 to the HCP Terraform Operator for Kubernetes (version 2), there is a one-time process that you need to complete. This process upgrades the operator to the newest version and migrate your custom resources.

Prerequisites

The migration process requires the following tools to be installed locally:

• kubectl
• Helm

Prepare for the upgrade

Configure an environment variable named RELEASE_NAMESPACE with the value of the namespace that the Helm chart is installed in.

$ export RELEASE_NAMESPACE=<NAMESPACE>

Next, create an environment variable named RELEASE_NAME with the value of the name that you gave your installation for the Helm chart.

$ export RELEASE_NAME=<INSTALLATION_NAME>

Before you migrate to HCP Terraform Operator for Kubernetes v2, you must first update v1 of the operator to the latest version, including the custom resource definitions.

$ helm upgrade --namespace ${RELEASE_NAMESPACE} ${RELEASE_NAME} hashicorp/terraform

Next, backup the workspace resources.

$ kubectl get workspace --all-namespaces -o yaml > backup_tfc_operator_v1.yaml

Manifest schema migration

Version 2 of the HCP Terraform Operator for Kubernetes renames and moves many existing fields. When you migrate, you must update your specification to match version 2's field names.

Workspace controller

The table below lists the field mapping of the Workspace controller between v1 and v2 of the operator.

Below is an example of configuring a variable in v1 of the operator.

apiVersion: app.terraform.io/v1alpha1
kind: Workspace
metadata:
  name: migration
  spec:
    variables:
      - key: username
        value: "user"
        hcl: true
        sensitive: false
        environmentVariable: false
      - key: SECRET_KEY
        value: "s3cr3t"
        hcl: false
        sensitive: false
        environmentVariable: true

In v2 of the operator, you must configure Terraform variables in spec.terraformVariables and environment variables spec.environmentVariables.

apiVersion: app.terraform.io/v1alpha2
kind: Workspace
metadata:
  name: migration
  spec:
    terraformVariables:
      - name: username
        value: "user"
        hcl: true
        sensitive: false
    environmentVariables:
      - name: SECRET_KEY
        value: "s3cr3t"
        hcl: false
        sensitive: false

Module controller

HCP Terraform Operator for Kubernetes v2 configures modules in a new Module controller separate from the Workspace controller. Below is a template of a custom resource manifest:

apiVersion: app.terraform.io/v1alpha2
kind: Module
metadata:
  name: <NAME>
spec:
  organization: <ORG-NAME>
  token:
    secretKeyRef:
      name: <SECRET-NAME>
      key: <KEY-NAME>
  name: operator

The table below describes the mapping between the Workspace controller from v1 and the Module controller in v2 of the operator.

Below is an example migration of a Module between v1 and v2 of the operator:

apiVersion: app.terraform.io/v1alpha1
kind: Workspace
metadata:
  name: migration
spec:
  module:
    source: app.terraform.io/org-name/module-name/provider
    version: 0.0.42
  variables:
    - key: username
      value: "user"
      hcl: true
      sensitive: false
      environmentVariable: false
    - key: SECRET_KEY
      value: "s3cr3t"
      hcl: false
      sensitive: false
      environmentVariable: true

In v2 of the operator, separate controllers manage workspace and modules.

apiVersion: app.terraform.io/v1alpha2
kind: Workspace
metadata:
  name: migration
  spec:
    terraformVariables:
      - name: username
        value: "user"
        hcl: true
        sensitive: false
    environmentVariables:
      - name: SECRET_KEY
        value: "s3cr3t"
        hcl: false
        sensitive: false

apiVersion: app.terraform.io/v1alpha2
kind: Module
metadata:
  name: migration
spec:
  name: operator
  module:
    source: app.terraform.io/org-name/module-name/provider
    version: 0.0.42
  workspace:
    name: migration

Upgrade the operator

Download Workspace CRD patch A:

$ curl -sO https://raw.githubusercontent.com/hashicorp/hcp-terraform-operator/main/docs/migration/crds/workspaces_patch_a.yaml

View the changes that patch A applies to the workspace CRD.

$ kubectl diff --filename workspaces_patch_a.yaml

Patch the workspace CRD with patch A. This patch adds app.terraform.io/v1alpha2 support, but excludes .status.runStatus because it has a different format in app.terraform.io/v1alpha1 and causes JSON un-marshalling issues.

$ kubectl patch crd workspaces.app.terraform.io --patch-file workspaces_patch_a.yaml

Install the Operator v2 Helm chart with the helm install command. Be sure to set the operator.watchedNamespaces value to the list of namespaces your Workspace resources are deployed to. If this value is not provided, the operator will watch all namespaces in the Kubernetes cluster.

$ helm install \
  ${RELEASE_NAME} hashicorp/hcp-terraform-operator \
  --version 2.4.0 \
  --namespace ${RELEASE_NAMESPACE} \
  --set 'operator.watchedNamespaces={white,blue,red}' \
  --set controllers.agentPool.workers=5 \
  --set controllers.module.workers=5 \
  --set controllers.workspace.workers=5

Next, create a Kubernetes secret to store the HCP Terraform API token following the Usage Guide. The API token can be copied from the Kubernetes secret that you created for v1 of the operator. By default, this is named terraformrc. Use the kubectl get secret command to get the API token.

$ kubectl --namespace ${RELEASE_NAMESPACE} get secret terraformrc -o json | jq '.data.credentials' | tr -d '"' | base64 -d

Update existing custom resources according to the schema migration guidance and apply your changes.

$ kubectl apply --filename <UPDATED_V2_WORKSPACE_MANIFEST.yaml>

Download Workspace CRD patch B.

$ curl -sO https://raw.githubusercontent.com/hashicorp/hcp-terraform-operator/main/docs/migration/crds/workspaces_patch_b.yaml

View the changes that patch B applies to the workspace CRD.

$ kubectl diff --filename workspaces_patch_b.yaml

Patch the workspace CRD with patch B. This patch adds .status.runStatus support, which was excluded in patch A.

$ kubectl patch crd workspaces.app.terraform.io --patch-file workspaces_patch_b.yaml

The v2 operator will fail to proceed if a custom resource has the v1 finalizer finalizer.workspace.app.terraform.io. If you encounter an error, check the logs for more information.

$ kubectl logs -f <POD_NAME>

Specifically, look for an error message such as the following.

ERROR   Migration   {"workspace": "default/<WORKSPACE_NAME>", "msg": "spec contains old finalizer finalizer.workspace.app.terraform.io"}

The finalizer exists to provide greater control over the migration process. Verify the custom resource, and when you’re ready to migrate it, use the kubectl patch command to update the finalizer value.

$ kubectl patch workspace migration --type=merge --patch '{"metadata": {"finalizers": ["workspace.app.terraform.io/finalizer"]}}'

Review the operator logs once more and verify there are no error messages.

$ kubectl logs -f <POD_NAME>

The operator reconciles resources during the next sync period. This interval is set by the operator.syncPeriod configuration of the operator and defaults to five minutes.

If you have any migrated Module custom resources, apply them now.

$ kubectl apply --filename <MIGRATED_V2_MODULE_MANIFEST.yaml>

In v2 of the operator, the applyMethod is set to manual by default. In this case, a new run in a managed workspace requires manual approval. Run the following command for each Workspace resource to change it to auto approval.

$ kubectl patch workspace <WORKSPACE_NAME> --type=merge --patch '{"spec": {"applyMethod": "auto"}}'