Partner Guide - Consul NIA, CTS, and Palo Alto Networks

As cloud adoption continues to grow, many organizations are moving towards hybrid cloud environments, gradually transferring some of their application services to public cloud platforms while also keeping their existing on-premises datacenters.

Hybrid-cloud environments increase complexity when designing and deploying application services globally, and they complicate the operations workflow by requiring collaboration among different groups (application, server, networking, and security teams) and among different platforms (cloud providers and datacenters). This motivates organizations to leverage DevOps approaches and tools that simplify and streamline the process, enabling self-service models and automating operational tasks.

A new outlook on network automation involves the creation of automation flows between your service networking control plane, your infrastructure provisioning platform, and your network devices.

HashiCorp and Palo Alto Networks recently collaborated on a strategy for this using HashiCorp's Network Infrastructure Automation (NIA). This works by triggering a Terraform workflow that automatically update Palo Alto resources based on changes it detects from the Consul catalog.

This tutorial demonstrates step-by-step how you can automate the configuration management process for Palo Alto Next-Generation Firewall (NGFW) using Terraform and Consul. You can use the workflow presented as a blueprint to get familiar with the pattern and accelerate your own networking infrastructure management.

Prerequisites

To complete the tutorial you need the following components:

1. A Palo Alto NGFW
2. A HashiCorp Consul datacenter for service networking and service mesh
3. A server with Consul-Terraform-Sync (CTS) installed. This server requires communication to the NGFW and the Consul datacenter for automation between those two products.
4. Two servers running a web service on port 80.

The image below shows the basic architecture of an NIA setup with Palo Alto NGFW.

Network device - Palo Alto NGFW

This tutorial uses the Palo Alto Next-Generation Firewall (NGFW) running PAN-OS for the network device. This workflow can also work with other Palo Alto Networks PAN-OS based devices such as the Panorama (as well as other vendors) that have built NIA integrations.

Below is an example Palo Alto configuration that shows a minimal Dynamic Address Group (DAG) configuration required for the Palo Alto Next-Generation Firewall (NGFW) and Network Infrastructure Automation (NIA) integration. This defines a Dynamic Address Group named web with a matching selection for 'web'. The web server resources will automatically be registered or de-registered via the NIA workflow process that is explained later in the tutorial.

Consul

With Network Infrastructure Automation (NIA), web servers and their services' status are monitored via Consul. Consul builds a service catalog by communicating with each service instance node via a local Consul agent.

$ consul members

Node              Address             Status  Type    Build   Protocol  DC   Segment
consul-server     10.88.0.100:8301    alive   server  1.10.3  2         dc1  <all>
web-1             10.88.0.102:8301    alive   client  1.10.3  2         dc1  <default>
web-2             10.88.0.103:8301    alive   client  1.10.3  2         dc1  <default> 

Once Consul is deployed and all agents have joined the datacenter, it is time to prepare Consul-Terraform-Sync (CTS).

Consul-Terraform-Sync requires access to the Consul catalog. If you're using Consul as a Terraform backend, you'll also need privileges to store the Terraform state into Consul's KV store.

In production environments, it is expected for Consul to be configured with Access Control Lists (ACLs) enabled. In order for Consul-Terraform-Sync to access information from Consul, a token needs to be passed to its configuration.

See the CTS tutorial for more information about ACL tokens and secure CTS configuration.

Once Consul is successfully deployed and all agents have joined the datacenter, it is time to prepare Consul-Terraform-Sync (CTS).

Consul-Terraform-Sync

Consul-Terraform-Sync (CTS) is a tool that uses Consul’s service catalog as a source of truth for all applications running in a given environment. When changes are detected to these applications (e.g. a new service node is added or deleted from the service list), CTS dynamically applies the necessary changes to your Dynamic Address Group of the Next-Generation Firewall (NGFW) using Terraform.

The Palo Alto Networks Terraform modules for Consul Terraform Sync enable network infrastructure automation for operators to dynamically configure address groups and dynamic address tags. Once configured, Consul-Terraform-Sync will register/de-register Address Group IP tags based on the respective services in Consul catalog.

The diagram below shows an example NIA workflow with Terraform, Consul, Consul-Terraform-Sync (CTS), and Palo Alto Next-Generation Firewall (NGFW).

Configure Consul services

On each web server, run a Consul agent with a service registration file where you will define the service details (IP, port, etc.) and health check information. Feel free to check out the service registration tutorial for more details.

Choose a tab to define these services on your web server nodes in either HCL or JSON.

$ cat /etc/consul.d/web.json

{
  "service": {
    "name": "web",
    "port": 80,
    "checks": [
      {
        "id": "web",
        "name": "web TCP Check",
        "tcp": "localhost:80",
        "interval": "10s",
        "timeout": "1s"
      }
    ]
  }
}

$ consul services register /etc/consul.d/web.json

$ cat /etc/consul.d/web.hcl

service {
  name = "web"
  port = 80
  check {
    id = "web
    name = "web TCP check"
    tcp = "localhost:80"
    interval = "10s"
    timeout = "1s"
  }
}

$ consul services register /etc/consul.d/web.hcl

After the reload, the Consul catalog will show the new instances of the web service in the Services tab:

Configure Consul-Terraform-Sync

You will define a set of tasks for CTS to execute whenever a service is registered or removed from Consul. The CTS configuration (in HCL format) below contains several blocks:

$ cat tasks.hcl 

log_level = "info"

driver "terraform" {
  log = true
  required_providers {
    panos = {
      source = "PaloAltoNetworks/panos"
    }
  }
}

consul {
  address = "localhost:8500"
}

terraform_provider "panos" {
  alias = "panos1" 
  address  = "10.64.4.104"
  username = "admin"
  api_key  = "778506df-a1b2-65e0-fe1e-eafd2d1162a8" 
}

task {
  name = "Create_DAG_on_PANOS1"
  description = "Automate Dynamic Address Groups based on service definition"
  module = "github.com/PaloAltoNetworks/terraform-panos-dag-nia"
  providers = ["panos.panos1"]
  condition "services" {
    names = ["web"]
  }
  variable_files = []
}

The driver defines all Terraform providers required to execute the task. In this case, source = "PaloAltoNetworks/panos" is listed.

The terraform_provider specifies the options and variables to interface with the Palo Alto Next-Generation Firewall (NGFW). The example above includes the IP address of the Palo Alto NGFW, an alias, and the login credentials.

The task block identifies a task to run as automation for the selected services. The task named Create_DAG_on_PANOS1 includes a list, under condition "services", of logical service names that should match the service name(s) registered on the Consul catalog. providers lists the network infrastructure (e.g., Palo Alto NGFW) with aliases (if applicable). module specifies a path to the Palo Alto Terraform NIA module defined for CTS that allows the Palo Alto NGFW to dynamically manage address group configuration for the services monitored on the Consul catalog.

Run Consul-Terraform-Sync

Once CTS is configured, start it using the consul-terraform-sync command. On startup, CTS will download and install the Terraform providers and modules according to the HCL config file, then create Terraform files for the tasks defined, and connect to Consul.

$ consul-terraform-sync start -config-file=cts-config.hcl

[INFO]  cli: v0.6.0 (7f0d7ba)
[INFO]  cli: running controller in daemon mode
[INFO]  cli: setting up controller: type=readwrite
[INFO]  ctrl: initializing Consul client and testing connection
[INFO]  driver.terraform: install terraform: install_path=/consul-terraform-sync/config
[INFO]  driver.terraform: successfully installed terraform

...

[INFO] running Terraform command: /consul-terraform-sync/config/terraform init -no-color -force-copy -input=false -backend=true -get=true -upgrade=false
Initializing modules...
Downloading github.com/PaloAltoNetworks/terraform-panos-dag-nia for Create_DAG_on_PANOS1...
- Create_DAG_on_PANOS1 in .terraform/modules/Create_DAG_on_PANOS1

Initializing the backend...

Successfully configured the backend "consul"! Terraform will automatically
use this backend unless the backend configuration changes.

Initializing provider plugins...
- Finding paloaltonetworks/panos versions matching "~> 1.8.3"...
- Installing paloaltonetworks/panos v1.8.3...
- Installed paloaltonetworks/panos v1.8.3 (signed by a HashiCorp partner, key ID D5D93F98EFA33E83)

...

Terraform has been successfully initialized!

[INFO] running Terraform command: /consul-terraform-sync/config/terraform workspace new -no-color Create_DAG_on_PANOS1
Created and switched to workspace "Create_DAG_on_PANOS1"!

...

[INFO]  ctrl: driver initialized
[INFO]  ctrl: executing all tasks once through
[INFO]  ctrl: executing task: task_name=Create_DAG_on_PANOS1
[INFO] running Terraform command: /consul-terraform-sync/config/terraform apply -no-color -auto-approve -input=false -var-file=terraform.tfvars -var-file=providers.tfvars -lock=true -parallelism=10 -refresh=true

Terraform used the selected providers to generate the following execution plan. Resource actions are
indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # module.Create_DAG_on_PANOS1.panos_ip_tag.this["web.web-1.dc1"] will be created
  + resource "panos_ip_tag" "this" {
      + id   = (known after apply)
      + ip   = "10.88.0.102"
      + tags = [
          + "web",
        ]
      + vsys = "vsys1"
    }

  # module.Create_DAG_on_PANOS1.panos_ip_tag.this["web.web-2.dc1"] will be created
  + resource "panos_ip_tag" "this" {
      + id   = (known after apply)
      + ip   = "10.88.0.103"
      + tags = [
          + "web",
        ]
      + vsys = "vsys1"
    }

Plan: 2 to add, 0 to change, 0 to destroy.
module.Create_DAG_on_PANOS1.panos_ip_tag.this["web.web-1.dc1"]: Creating...

...

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.
[INFO]  ctrl: task completed: task_name=Create_DAG_on_PANOS1
[INFO]  ctrl: all tasks completed once
[INFO]  api: starting server: port=8558

CTS will continue running as a daemon and will execute the Create_DAG_on_PANOS1 task automatically any time a change is triggered.

Verify NIA automation workflow

Once Consul-Terraform-Sync is configured and started, you can verify the workflow by clicking the more... link for your Address Group web.

Any time Consul detects changes to your web service, Consul-Terraform-Sync will automatically register/de-register the IP addresses tags in the Dynamic Address Group on your Palo Alto Next-Generation Firewall (NGFW).

Next steps

The Network Infrastructure Automation (NIA) solution using Consul-Terraform-Sync (CTS) is a powerful network automation enabler and works perfectly with Palo Alto Next-Generation Firewall (NGFW). The two resources allow organizations to achieve better automation workflows together.

This automates the operations necessary in case of infrastructure change or if any unexpected server failure happens. Furthermore, you can easily extend this NIA solution to support CI/CD operations including blue-green deployment by leveraging Terraform and service tags on Consul.

The Palo Alto Next-Generation Firewall and Palo Alto Panorama are compatible with various cloud providers, hypervisors, and containers. Combined with the extreme portability of Consul and Terraform makes the workflow portable with minimal changes, in any type of platform, increasing your flexibility and simplifying workflow migration across different solutions.

Links and references

For more resources on how to try NIA out yourself, check out:

• Consul documentation
• Try Palo Alto NGFW free for 30 days
• Get the Terraform NIA module for Palo Alto NGFW
• Webinar: Automating Network Infrastructure Tasks with Palo Alto and HashiCorp
• Whitepaper: Enabling Dynamic Firewalling with Palo Alto and HashiCorp