Partner Guide - Consul NIA, Terraform, and F5 BIG-IP

12min
Terraform
Consul
Video

In a previous tutorial, we demonstrated how to integrate the Consul catalog with an F5 BIG-IP load balancer by deploying applications with the Application Services 3 (AS3) extensions.

In a Day2+ scenario, F5 BIG-IP's automation tool chain can be used to automate the configuration of an F5 BIG-IP load balancer using F5 Application Services Templates (FAST) to deploy and manage Application Services 3 (AS3) applications.

HashiCorp and F5 collaborated on a strategy for this using HashiCorp's Network Infrastructure Automation (NIA).

This tutorial provides step-by-step instructions on how to automate the configuration update process for an AS3 application using Terraform and Consul. You can use the workflow presented as a blueprint to get familiar with the pattern and accelerate your networking infrastructure management.

Prerequisites

To complete this tutorial, you will need previous experience with F5 BIG-IP and Consul. You can either manually deploy the necessary infrastructure, or use the Terraform demo code.

To complete the steps listed in this tutorial you should configure the following infrastructure.

A single Consul datacenter with server and client nodes, and the configuration directory for Consul agents at /etc/consul.d/.
A running instance of the F5 BIG-IP platform. If you don’t already have one you can use a hosted AWS instance for this tutorial.
The AS3 package version 3.7.0 installed on your F5 BIG-IP platform.
Standard web server running on a node, listening on HTTP port 80. You will use NGINX in this tutorial.

You can set up the prerequisites on your own, or use the Terraform configuration in this repository to set up the entire tutorial environment.

The tutorial will provide the detailed steps to setup the environment using the Terraform code provided in the repository.

Terraform 0.13.x - To use the Terraform configuration in the repository, you need to have the terraform binary installed on your test machine.
Consul-Terraform-Sync 0.2.0+.

Watch the video - Optional

Consul's integration with F5 was demonstrated in a webinar. If you want to learn about the integration but aren't ready to try it out, you can watch the webinar recording to see the integration in action.

Deploy the demo environment using Terraform - Optional

The tutorial provides an example scenario that can be deployed on AWS using Terraform.

First, clone the repository.

$ git clone https://github.com/hashicorp/f5-terraform-consul-sd-webinar.git

Next, navigate to the repository's root folder.

$ cd f5-terraform-consul-sd-webinar

The demo code uses AWS to deploy the datacenter infrastructure, as well as the instance of the F5 BIG-IP platform. To continue with the deploy, configure AWS credentials for your environment so that Terraform can authenticate with AWS and create resources.

To do this with IAM user authentication, set your AWS access key ID as an environment variable.

$ export AWS_ACCESS_KEY_ID="<YOUR_AWS_ACCESS_KEY_ID>"

Now set your secret key.

$ export AWS_SECRET_ACCESS_KEY="<YOUR_AWS_SECRET_ACCESS_KEY>"

If you have temporary AWS credentials, you must also add your AWS_SESSION_TOKEN as an environment variable. See the AWS Provider Documentation for more details.

Tip

If you don't have access to IAM user credentials, use another authentication method described in the AWS provider documentation.

Deploy Consul datacenter and F5 BIG-IP

The Terraform code for deploying the Consul datacenter and the BIG-IP instance is under the terraform folder.

Move into the folder.

$ cd terraform/

Generate a random password for your F5 BIG-IP instance.

$ ./admin-shadow.sh

Use the terraform.tfvars.example template file to create a terraform.tfvars file. The example file is in the terraform folder.

$ cp terraform.tfvars.example terraform.tfvars

Edit the file to specify a prefix for the resources being created and an IP address to access the environment once deployed.

terraform.tfvars

#
## prefix will be added to resources created
#

prefix = "your-prefix"

# IP address to allow traffic from
# recommended to use a /32 (single IP address)

allow_from = "192.0.2.0/32"

#
# additional options 
#

# region = "us-east-1"
# f5_ami_search_name = "F5 BIGIP-15.1.2* PAYG-Good 25Mbps*"
# f5_username = "bigipuser"

Once the configuration is complete, you can deploy the infrastructure with Terraform.

First, initialize Terraform.

$ terraform init

Note

If terraform init fails with Error: Failed to install provider issue the command again.

Then, use terraform plan to check the resources that are going to be created.

$ terraform plan

...
Plan: 30 to add, 0 to change, 0 to destroy.
...

Finally, apply the changes.

$ terraform apply -auto-approve

...
Apply complete! Resources: 30 added, 0 changed, 0 destroyed.

Outputs:

Consul_UI = http://3.86.229.209:8500
F5_IP = 52.44.244.96
F5_Password = L5l81KPsxv
F5_UI = https://52.44.244.96:8443
F5_Username = admin
F5_ssh = ssh -i terraform-20210803123725302700000001.pem admin@52.44.244.96

The final part of the Terraform output provides you with the information to access your infrastructure.

You can access your Consul datacenter UI using the address specified by the Consul_UI value.

Consul UI showing services tab with two instances of NGINX

From the Consul UI you can verify the datacenter contains two instances of NGINX running on two different nodes.

By opening your browser at the URL specified by the F5_UI variable, you can access your F5 BIG-IP instance GUI.

BigIP web UI booting up

After the device finishes booting, use the F5_Username and F5_Password values to login.

Deploy FAST template

F5 Application Services Templates (FAST) are an easy and effective way to deploy applications on the BIG-IP system using AS3.

The FAST Extension provides a toolset for templating and managing AS3 Applications on BIG-IP.

To deploy the FAST template, you can use the code under the fast folder of the example scenario code.

The code contains a working template example that defines an AS3 application representing an NGINX instance.

Click here for the template example from the repository

ConsulWebinar.yaml

title: Consul Service Discovery
description: This template will create a virtual server that will use event driven service discovery
definitions:
  tenant:
    title: Name of tenant
    description: give a unique name for this tenant
  app:
    title: Application
    description: give a unique name for this application
  virtualAddress:
    title: Virtual Address
    description: IP addresses of virtual addresses (will create 80/443)
  virtualPort:
    title: Virtual Port
    description: Port that will be used
    type: integer
parameters:
  uri: "http://10.0.0.100:8500"
  service: "nginx"
  virtualAddress: 10.0.0.200
  virtualPort: 8080
  tenant: "Consul_SD"
  app: "Nginx"
template: |
  {
    "class": "AS3",
    "action": "deploy",
    "persist": true,
    "declaration": {
        "class": "ADC",
        "schemaVersion": "3.0.0",
        "id": "urn:uuid:940bdb69-9bcd-4c5c-9a34-62777210b581",
        "label": "Consul Webinar",
        "remark": "Consul Webinar",
        "{{tenant}}": {
          "class": "Tenant",
          "{{app}}": {
              "class": "Application",
              "webserver_vs": {
                "class": "Service_HTTP",
                "virtualPort": {{virtualPort}},
                "virtualAddresses": [
                    "{{virtualAddress}}"
                ],
                "pool": "nginx_pool",
                "persistenceMethods": [],
                "profileMultiplex": {
                "bigip": "/Common/oneconnect"
              }
              },
              "nginx_pool": {
                "class": "Pool",
                "monitors": [
                    "http"
                ],
                "members": [{
              "servicePort": 80,
              "addressDiscovery": "event"
            }]
              }
          }
        }
    }
  }

Move into the folder.

$ cd ../fast

First, initialize Terraform.

$ terraform init

Initializing the backend...

Initializing provider plugins...
- Finding f5networks/bigip versions matching "1.9.0"...
- Finding latest version of hashicorp/archive...
- Installing f5networks/bigip v1.9.0...
- Installed f5networks/bigip v1.9.0 (signed by a HashiCorp partner, key ID 8D69F031B13946D3)
- Installing hashicorp/archive v2.2.0...
- Installed hashicorp/archive v2.2.0 (signed by HashiCorp)

Partner and community providers are signed by their developers.
If you'd like to know more about provider signing, you can read about it here:
https://www.terraform.io/docs/plugins/signing.html

The following providers do not have any version constraints in configuration,
so the latest version was installed.

To prevent automatic upgrades to new major versions that may contain breaking
changes, we recommend adding version constraints in a required_providers block
in your configuration, with the constraint strings suggested below.

* hashicorp/archive: version = "~> 2.2.0"

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Note

Issue the command again if terraform init fails with Error: Failed to install provider.

Then, use terraform plan to check the resources that are going to be created.

$ terraform plan

...
Plan: 2 to add, 0 to change, 0 to destroy.
...

Finally, apply the changes.

$ terraform apply -auto-approve

data.archive_file.template_zip: Refreshing state...
bigip_fast_template.consul-webinar: Creating...
bigip_fast_template.consul-webinar: Still creating... [10s elapsed]
bigip_fast_template.consul-webinar: Creation complete after 10s [id=ConsulWebinar]
bigip_fast_application.nginx-webserver: Creating...
bigip_fast_application.nginx-webserver: Still creating... [10s elapsed]
bigip_fast_application.nginx-webserver: Creation complete after 11s [id=Nginx]

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

Outputs:

app_url = http://52.44.244.96:8080

Once the terraform command terminates, you can verify that the FAST template deployed on your F5 instance.

BigIP web UI F5 Application Services Templates

The URL provided by app_url in the output is the public endpoint that will grant you access to the services once consul-terraform-sync is started.

Network Infrastructure Automation

Once all infrastructure is deployed, you can use Consul-Terraform-Sync to monitor the changes in your NGINX instances and modify your F5 BigIP configuration accordingly, without the need for a network operator.

First, move into the nia directory in the repository.

$ cd ../nia/

Next, you will configure Consul-Terraform-Sync.

If you deployed the infrastructure using the provided scripts, the configuration file is produced automatically for you under the nia folder.

config.hcl

driver "terraform" {
  log = true
  required_providers {
    bigip = {
      source = "F5Networks/bigip"
    }
  }
}
#log_level = "trace"
consul {
  address = "3.86.229.209:8500"
}

terraform_provider "bigip" {
  address  = "52.44.244.96:8443"
  username = "admin"
  password = "L5l81KPsxv"
}

task {
  name = "AS3"
  description = "BIG-IP example"
  source = "./bigip-example"
  providers = ["bigip"]
  services = ["nginx"]
}

Finally, you can start Consul-Terraform-Sync.

$ consul-terraform-sync start -config-file config.hcl

...
[INFO] (cli) running controller in daemon mode
[INFO] (cli) setting up controller: readwrite
[INFO] (ctrl) initializing Consul client and testing connection
[INFO] (cli) initializing controller
[INFO] (ctrl) initializing driver
[INFO] (ctrl) initializing all tasks
[INFO] (client.terraformcli) Terraform logging is set, Terraform logs will output with Sync logs
[INFO] (driver.terraform) retrieved 0 Terraform handlers for task 'AS3'
[INFO] (templates.tftmpl) overwriting providers.tfvars in root module for task "AS3"
[INFO] (templates.tftmpl) overwriting main.tf in root module for task "AS3"
[INFO] (templates.tftmpl) overwriting variables.tf in root module for task "AS3"
[INFO] (templates.tftmpl) overwriting terraform.tfvars.tmpl in root module for task "AS3"
[INFO] running Terraform command: ./f5-terraform-consul-sd-webinar/nia/terraform version -json
{
  "terraform_version": "0.13.7",
  "terraform_revision": "",
  "provider_selections": {
    "registry.terraform.io/f5networks/bigip": "1.8.0"
  },
  "terraform_outdated": true
}
[INFO] running Terraform command: ./f5-terraform-consul-sd-webinar/nia/terraform init -no-color -force-copy -input=false -lock-timeout=0s -backend=true -get=true -upgrade=false -lock=true -get-plugins=true -verify-plugins=true
Initializing modules...

Initializing the backend...

Initializing provider plugins...
- Using previously-installed f5networks/bigip v1.8.0

Terraform has been successfully initialized!
[INFO] running Terraform command: ./f5-terraform-consul-sd-webinar/nia/terraform workspace new -no-color AS3
Workspace "AS3" already exists
[INFO] running Terraform command: ./f5-terraform-consul-sd-webinar/nia/terraform workspace select -no-color AS3
[INFO] running Terraform command: ./f5-terraform-consul-sd-webinar/nia/terraform validate -no-color -json
{
  "valid": true,
  "error_count": 0,
  "warning_count": 0,
  "diagnostics": []
}
[INFO] (ctrl) driver initialized
[INFO] (ctrl) executing all tasks once through
[INFO] (ctrl) executing task AS3
[INFO] running Terraform command: ./f5-terraform-consul-sd-webinar/nia/terraform apply -no-color -auto-approve -input=false -var-file=terraform.tfvars -var-file=providers.tfvars -lock=true -parallelism=10 -refresh=true
Acquiring state lock. This may take a few moments...
module.AS3.bigip_event_service_discovery.event_pools["nginx"]: Refreshing state... [id=~Consul_SD~Nginx~nginx_pool]
module.AS3.bigip_event_service_discovery.event_pools["nginx"]: Modifying... [id=~Consul_SD~Nginx~nginx_pool]
module.AS3.bigip_event_service_discovery.event_pools["nginx"]: Modifications complete after 1s [id=~Consul_SD~Nginx~nginx_pool]

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.
Releasing state lock. This may take a few moments...
[INFO] (ctrl) task completed AS3
[INFO] (ctrl) all tasks completed once
[INFO] (api) starting server at '8558'
...

Open the URL provided by app_url and verify that load balancing works.

BigIP Load Balancer redirecting traffic to NGINX instances

Test automation

To verify the integration is working, add more NGINX instances by editing the AWS Auto Scaling group configuration.

Edit the nginx.tf file inside the terraform folder to change the desired capacity from 2 to 4.

terraform/nginx.tf

resource "aws_autoscaling_group" "nginx" {
  name                 = "${var.prefix}-nginx-asg"
  launch_configuration = aws_launch_configuration.nginx.name
  desired_capacity     = 4
  min_size             = 1
  max_size             = 4
  vpc_zone_identifier  = [module.vpc.public_subnets[0]]

  lifecycle {
    create_before_destroy = true
  }

  tags = [
    {
      key                 = "Name"
      value               = "${var.prefix}-nginx"
      propagate_at_launch = true
    },
    {
      key                 = "Env"
      value               = "consul"
      propagate_at_launch = true
    },
  ]

}
...

Then, use terraform plan to check the resources that are going to be changed.

$ terraform plan

Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

...

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # aws_autoscaling_group.nginx will be updated in-place
  ~ resource "aws_autoscaling_group" "nginx" {

    ...

      ~ desired_capacity          = 2 -> 4

    ...

    }

Plan: 0 to add, 1 to change, 0 to destroy.
...

Finally, apply the changes.

$ terraform apply -auto-approve

...
Apply complete! Resources: 0 added, 1 changed, 0 destroyed.
...

Once the changes are applied on AWS, Consul will show the new instances on the Services tab.

Consul UI showing services tab with four instances of NGINX

Consul-Terraform-Sync will pick the change from the Consul catalog and modify the BIG-IP configuration to reflect the new NGINX instances.

...
[INFO] (ctrl) executing task AS3
[INFO] running Terraform command: ./f5-terraform-consul-sd-webinar/nia/terraform apply -no-color -auto-approve -input=false -var-file=terraform.tfvars -var-file=providers.tfvars -lock=true -parallelism=10 -refresh=true
Acquiring state lock. This may take a few moments...
module.AS3.bigip_event_service_discovery.event_pools["nginx"]: Refreshing state... [id=~Consul_SD~Nginx~nginx_pool]
module.AS3.bigip_event_service_discovery.event_pools["nginx"]: Modifying... [id=~Consul_SD~Nginx~nginx_pool]
module.AS3.bigip_event_service_discovery.event_pools["nginx"]: Modifications complete after 1s [id=~Consul_SD~Nginx~nginx_pool]

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.
Releasing state lock. This may take a few moments...
[INFO] (ctrl) task completed AS3
[INFO] (ctrl) executing task AS3
[INFO] running Terraform command: ./f5-terraform-consul-sd-webinar/nia/terraform apply -no-color -auto-approve -input=false -var-file=terraform.tfvars -var-file=providers.tfvars -lock=true -parallelism=10 -refresh=true
Acquiring state lock. This may take a few moments...
module.AS3.bigip_event_service_discovery.event_pools["nginx"]: Refreshing state... [id=~Consul_SD~Nginx~nginx_pool]
module.AS3.bigip_event_service_discovery.event_pools["nginx"]: Modifying... [id=~Consul_SD~Nginx~nginx_pool]
module.AS3.bigip_event_service_discovery.event_pools["nginx"]: Modifications complete after 0s [id=~Consul_SD~Nginx~nginx_pool]

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.
Releasing state lock. This may take a few moments...
[INFO] (ctrl) task completed AS3
[INFO] (ctrl) executing task AS3
[INFO] running Terraform command: ./f5-terraform-consul-sd-webinar/nia/terraform apply -no-color -auto-approve -input=false -var-file=terraform.tfvars -var-file=providers.tfvars -lock=true -parallelism=10 -refresh=true
Acquiring state lock. This may take a few moments...
module.AS3.bigip_event_service_discovery.event_pools["nginx"]: Refreshing state... [id=~Consul_SD~Nginx~nginx_pool]
module.AS3.bigip_event_service_discovery.event_pools["nginx"]: Modifying... [id=~Consul_SD~Nginx~nginx_pool]
module.AS3.bigip_event_service_discovery.event_pools["nginx"]: Modifications complete after 1s [id=~Consul_SD~Nginx~nginx_pool]

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.
Releasing state lock. This may take a few moments...
[INFO] (ctrl) task completed AS3
[INFO] (ctrl) executing task AS3
[INFO] running Terraform command: ./f5-terraform-consul-sd-webinar/nia/terraform apply -no-color -auto-approve -input=false -var-file=terraform.tfvars -var-file=providers.tfvars -lock=true -parallelism=10 -refresh=true
Acquiring state lock. This may take a few moments...
module.AS3.bigip_event_service_discovery.event_pools["nginx"]: Refreshing state... [id=~Consul_SD~Nginx~nginx_pool]
module.AS3.bigip_event_service_discovery.event_pools["nginx"]: Modifying... [id=~Consul_SD~Nginx~nginx_pool]
module.AS3.bigip_event_service_discovery.event_pools["nginx"]: Modifications complete after 0s [id=~Consul_SD~Nginx~nginx_pool]

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.
Releasing state lock. This may take a few moments...
[INFO] (ctrl) task completed AS3
...

Refresh the page to verify the traffic is being balanced across the four NGINX instances.

BigIP Load Balancer redirecting traffic to NGINX instances

Clean your environment

Once you have completed the tutorial, you can clean up your environment by stopping consul-terraform-sync and using terraform destroy from each of the folders you deployed from.

Stop Consul-Terraform-Sync

You can stop Consul-Terraform-Sync by either using CTRL+C in the shell running the daemon or by sending the SIGINT signal to the process.

Before stopping, the daemon will log the requested shutdown.

...
[INFO] (cli) signal received to initiate graceful shutdown: terminated
[INFO] (ctrl) stopping controller
[INFO] (api) shutdown api server
[INFO] (cli) graceful shutdown

Destroy resources

$ cd ../fast

$ terraform destroy -auto-approve

...
Destroy complete! Resources: 2 destroyed.

$ cd ../terraform

$ terraform destroy -auto-approve

...
Destroy complete! Resources: 29 destroyed.

Clean your repository

Finally, you can clean the Git repository to remove files created by the runs.

$ git clean -xdf

Next steps

In this tutorial, you learned how to automate the configuration of your F5 BIG-IP load balancer using Consul-Terraform-Sync and F5 Application Services Templates (FAST).

To learn more about Network Infrastructure Automation with Consul-Terraform-Sync, check the full documentation on the Consul website.

You can review the Secure Consul-Terraform-Sync for Production tutorial to learn how to secure the Consul-Terraform-Sync instance and other best practices to integrate it into a production environment.

For more info on F5 Application Services Templates (FAST) refer to the official documentation.