Deploy an App with Nomad Service Discovery

14min
Nomad

Service discovery is a method of automatically detecting services running on a host machine with the intent of making them available to other services connected to the host. Tools that offer these functionalities expose the service's address and port for communication by way of a service name or endpoint. Consul is one of these tools that takes care of routing requests sent to the service name endpoint and forwarding them on to the application running on a specific host and port.

Diagram showing service endpoint and Consul forwarding the request to the application host on a specific port

Nomad offers an easy-to-use integration with Consul that abstracts away a lot of the configuration necessary for service discovery, requiring only a few lines of configuration in the Nomad job spec file.

job "example-job" {
  task "simple-task" {
    service {
      name = "simple-service"
      port = "8080"
    }
  }
}

Consul service definition in a Nomad job file

job "example-job" {
  task "simple-task" {
    service {
      name = "simple-service"
      port = "8080"
    }
  }
}

However, this means that Consul must either be installed separately alongside Nomad on the host or running somewhere else, such as in HCP. Including Consul in your environment adds another layer of complexity to the deployment and, depending on the application, additional complexity that may not be worth the effort.

Nomad v1.3 added built-in native service discovery (SD), making Consul or other third-party tools unnecessary. Nomad's service discovery is not meant to be a replacement for those tools. Rather, it is an alternative for easier testing and deployments with simpler architectures.

Read on to see how you can easily connect your services together that are running in Nomad.

»Prerequisites

For this tutorial, you will need:

Nomad v1.3.3 or later installed locally
Terraform v1.2.0 or later installed locally
An AWS account with credentials set as local environment variables and an AWS keypair
An AMI available in your AWS region created by following the steps in the Nomad Cluster Setup on AWS tutorial

Note: This tutorial creates AWS resources that may not qualify as part of the AWS free tier. Be sure to follow the Cleanup process at the end of this tutorial so you don't incur any additional unnecessary charges.

»Service discovery usage

The Nomad job spec file defines services and service queries but you can also interact with the catalog and find information about services through the Nomad CLI tool. Each of these options are shown in the examples below.

»Service definition

The service block enables a Nomad service for discovery by setting the provider attribute to nomad. By default, the value is set to consul implicitly.

job "example-job" {
  task "simple-task" {
    service {
      name = "simple-service"
      port = "8080"
      provider = "nomad"
    }
  }
}

Service definition example

job "example-job" {
  task "simple-task" {
    service {
      name = "simple-service"
      port = "8080"
      provider = "nomad"
    }
  }
}

»Service query

A template block can specify a function to query a service's address and port. This is a similar procedure for querying services registered in Consul. The following example uses the nomadService function.

job "example-job" {
    group "db" {
        network {
            port "db" {
                static = 5432
            }
        }
        task "db" {
            service {
                name = "database"
                provider = "nomad"
                port = "db"
                address  = attr.unique.platform.aws.public-ipv4
            }
        }
    }
    group "product-api" {
        task "product-api" {
        ## ...
            template {
                data        = <<EOH
{{ range nomadService "database" }}
DB_CONNECTION="host={{ .Address }} port={{ .Port }} user=user password=password dbname=db_name"
{{ end }}
EOH
                destination = "local/env.txt"
                env         = true
            }
        }
    }
}

Service query example

job "example-job" {
    group "db" {
        network {
            port "db" {
                static = 5432
            }
        }
        task "db" {
            service {
                name = "database"
                provider = "nomad"
                port = "db"
                address  = attr.unique.platform.aws.public-ipv4
            }
        }
    }
    group "product-api" {
        task "product-api" {
        ## ...
            template {
                data        = <<EOH
{{ range nomadService "database" }}
DB_CONNECTION="host={{ .Address }} port={{ .Port }} user=user password=password dbname=db_name"
{{ end }}
EOH
                destination = "local/env.txt"
                env         = true
            }
        }
    }
}

»Service commands

Once a service is registered, you can interact with it via the service command of the Nomad CLI.

$ nomad service list
Service Name  Tags
database      []
frontend      []
nginx         []
payments-api  []
product-api   []
public-api    []

Listing registered services

$ nomad service list
Service Name  Tags
database      []
frontend      []
nginx         []
payments-api  []
product-api   []
public-api    []

$ nomad service info nginx
Job ID     Address                                        Tags  Node ID   Alloc ID
hashicups  ec2-18-233-100-132.compute-1.amazonaws.com:80  []    340e27c3  8e7e0db5

Getting more information about a registered service

$ nomad service info nginx
Job ID     Address                                        Tags  Node ID   Alloc ID
hashicups  ec2-18-233-100-132.compute-1.amazonaws.com:80  []    340e27c3  8e7e0db5

»Load balancing

Nomad balances service workloads by selecting an allocation of the referenced service (e.g. database) when the upstream service (e.g. product-api) allocation is created. Nomad selects from available allocations of the referenced service that have not previously been selected from any of the upstream service’s currently active allocations.

job "example-job" {
    group "db" {
        network {
            port "db" {
                static = 5432
            }
        }
        task "db" {
            service {
                name = "database"
                provider = "nomad"
                port = "db"
                address  = attr.unique.platform.aws.public-ipv4
            }
        }
    }
    group "product-api" {
        task "product-api" {
        ## ...
            template {
                data = <<EOH
{{ $allocID := env "NOMAD_ALLOC_ID" }}
{{ $service := nomadService 1 $allocID "database" }}
[product-api]
server = {{ $service.Address }}
ports = [ {{ $service.Port }} ]
                EOH

            destination = "config.toml"
            }
        }
    }
}

Load balancing setup example

job "example-job" {
    group "db" {
        network {
            port "db" {
                static = 5432
            }
        }
        task "db" {
            service {
                name = "database"
                provider = "nomad"
                port = "db"
                address  = attr.unique.platform.aws.public-ipv4
            }
        }
    }
    group "product-api" {
        task "product-api" {
        ## ...
            template {
                data = <<EOH
{{ $allocID := env "NOMAD_ALLOC_ID" }}
{{ $service := nomadService 1 $allocID "database" }}
[product-api]
server = {{ $service.Address }}
ports = [ {{ $service.Port }} ]
                EOH

            destination = "config.toml"
            }
        }
    }
}

»HashiCups with Nomad SD

In this tutorial, you will deploy HashiCups, a demo coffee shop application made up of several microservices running on Docker. Each of the services configured in the HashiCups job spec file use the native service discovery feature to retrieve downstream service connection information.

Additionally, you will modify the application after the initial deployment to integrate the load balancing feature of the native service discovery.

You can read more about the HashiCups application in the HashiCups overview.

HashiCups frontend and backend
services

»Clone the example repository

The example repository contains Terraform configuration to set up a cluster and a Nomad job spec file (hashicups.hcl) containing the entire HashiCups application.

Clone the example repository.

$ git clone https://github.com/hashicorp/learn-nomad-sd.git

$ git clone https://github.com/hashicorp/learn-nomad-sd.git

Navigate to the cloned repository folder.

$ cd learn-nomad-sd

$ cd learn-nomad-sd

Check out the v0.1 tag of the repository as a local branch named nomad-hashicups-sd.

$ git checkout tags/v0.1 -b nomad-hashicups-sd
Switched to a new branch 'nomad-hashicups-sd'

$ git checkout tags/v0.1 -b nomad-hashicups-sd
Switched to a new branch 'nomad-hashicups-sd'

»Deploy Nomad cluster

Rename the example variables file from terraform.tfvars.example to terraform.tfvars.

$ mv terraform.tfvars.example terraform.tfvars

$ mv terraform.tfvars.example terraform.tfvars

The module in main.tf requires an Amazon Machine Image (AMI) for provisioning the cluster's servers and clients. Copy the AMI ID that you created as part of the prerequisites and replace the placeholder value for ami in the terraform.tfvars with it.

name                      = "nomad"
key_name                  = "us-east-1-ssh-key"
region                    = "us-east-1"
ami                       = "ami-012345678901234567"
allowlist_ip              = "0.0.0.0/0"
nomad_consul_token_id     = "123e4567-e89b-12d3-a456-426614174000"
nomad_consul_token_secret = "123e4567-e89b-12d3-a456-426614174000"

terraform.tfvars

name                      = "nomad"
key_name                  = "us-east-1-ssh-key"
region                    = "us-east-1"
ami                       = "ami-012345678901234567"
allowlist_ip              = "0.0.0.0/0"
nomad_consul_token_id     = "123e4567-e89b-12d3-a456-426614174000"
nomad_consul_token_secret = "123e4567-e89b-12d3-a456-426614174000"

Replace the region and key_name values with the AWS region that contains the AMI from above and SSH key registered in that region. Replace the allowlist_ip value with a range of IP addresses if you want to limit access to the virtual machines.

name                      = "nomad"
key_name                  = "us-east-1-ssh-key"
region                    = "us-east-1"
ami                       = "ami-012345678901234567"
allowlist_ip              = "0.0.0.0/0"
nomad_consul_token_id     = "123e4567-e89b-12d3-a456-426614174000"
nomad_consul_token_secret = "123e4567-e89b-12d3-a456-426614174000"

terraform.tfvars

name                      = "nomad"
key_name                  = "us-east-1-ssh-key"
region                    = "us-east-1"
ami                       = "ami-012345678901234567"
allowlist_ip              = "0.0.0.0/0"
nomad_consul_token_id     = "123e4567-e89b-12d3-a456-426614174000"
nomad_consul_token_secret = "123e4567-e89b-12d3-a456-426614174000"

Finally, generate UUIDs for the nomad_consul_token_id and nomad_consul_token_secret and replace the placeholder values for them in terraform.tfvars.

name                      = "nomad"
key_name                  = "us-east-1-ssh-key"
region                    = "us-east-1"
ami                       = "ami-012345678901234567"
allowlist_ip              = "0.0.0.0/0"
nomad_consul_token_id     = "123e4567-e89b-12d3-a456-426614174000"
nomad_consul_token_secret = "123e4567-e89b-12d3-a456-426614174000"

terraform.tfvars

name                      = "nomad"
key_name                  = "us-east-1-ssh-key"
region                    = "us-east-1"
ami                       = "ami-012345678901234567"
allowlist_ip              = "0.0.0.0/0"
nomad_consul_token_id     = "123e4567-e89b-12d3-a456-426614174000"
nomad_consul_token_secret = "123e4567-e89b-12d3-a456-426614174000"

»Provision the Nomad cluster

Initialize Terraform so that it retrieves any required plugins and sets up the workspace.

$ terraform init
Initializing modules...
Downloading github.com/hashicorp/learn-nomad-cluster-setup for nomad-cluster...
- nomad-cluster in .terraform/modules/nomad-cluster

Initializing the backend...

Initializing provider plugins...
- Finding latest version of hashicorp/aws...
- Finding latest version of hashicorp/template...
- Installing hashicorp/template v2.2.0...
- Installed hashicorp/template v2.2.0 (signed by HashiCorp)
- Installing hashicorp/aws v4.25.0...
- Installed hashicorp/aws v4.25.0 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

$ terraform init
Initializing modules...
Downloading github.com/hashicorp/learn-nomad-cluster-setup for nomad-cluster...
- nomad-cluster in .terraform/modules/nomad-cluster

Initializing the backend...

Initializing provider plugins...
- Finding latest version of hashicorp/aws...
- Finding latest version of hashicorp/template...
- Installing hashicorp/template v2.2.0...
- Installed hashicorp/template v2.2.0 (signed by HashiCorp)
- Installing hashicorp/aws v4.25.0...
- Installed hashicorp/aws v4.25.0 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

Provision the resources. Respond yes to the prompt to confirm the operation and then press the Enter key to start the process. This will take a few minutes to provision.

$ terraform apply
Apply complete! Resources: 15 added, 0 changed, 0 destroyed.

Outputs:

IP_Addresses = <<EOT

Client public IPs: 3.98.57.19, 3.96.66.175, 35.183.78.6

Server public IPs: 99.79.73.235, 3.99.241.240, 15.223.77.39

The Consul UI can be accessed at http://ar-nomad-server-lb-2016482221.ca-central-1.elb.amazonaws.com:8500/ui
with the bootstrap token: 8f94ee77-bc50-4ba1-bf75-132ed6b9366e

EOT
consul_bootstrap_token_secret = "8f94ee77-bc50-4ba1-bf75-132ed6b9366e"
lb_address_consul_nomad = "http://ar-nomad-server-lb-2016482221.ca-central-1.elb.amazonaws.com"

$ terraform apply
Apply complete! Resources: 15 added, 0 changed, 0 destroyed.

Outputs:

IP_Addresses = <<EOT

Client public IPs: 3.98.57.19, 3.96.66.175, 35.183.78.6

Server public IPs: 99.79.73.235, 3.99.241.240, 15.223.77.39

The Consul UI can be accessed at http://ar-nomad-server-lb-2016482221.ca-central-1.elb.amazonaws.com:8500/ui
with the bootstrap token: 8f94ee77-bc50-4ba1-bf75-132ed6b9366e

EOT
consul_bootstrap_token_secret = "8f94ee77-bc50-4ba1-bf75-132ed6b9366e"
lb_address_consul_nomad = "http://ar-nomad-server-lb-2016482221.ca-central-1.elb.amazonaws.com"

»Set up access to Nomad

Run the post-setup.sh script to complete the setup process.

Note: The Consul UI may take some time to finish setting up and becoming responsive. If the post-setup.sh script fails, wait a minute, make sure the Consul UI is up, and run rm nomad.token && ./post-setup.sh to delete the empty token file and try again.

$ ./post-setup.sh
The Nomad user token has been saved locally to nomad.token and deleted from the Consul KV store.

Set the following environment variables to access your Nomad cluster with the user token created during setup:

export NOMAD_ADDR=$(terraform output -raw lb_address_consul_nomad):4646
export NOMAD_TOKEN=$(cat nomad.token)


The Nomad UI can be accessed at http://ar-nomad-server-lb-2016482221.ca-central-1.elb.amazonaws.com:4646/ui
with the bootstrap token: 0853c07b-5718-8b0a-dad8-e3d6715bbd9f

$ ./post-setup.sh
The Nomad user token has been saved locally to nomad.token and deleted from the Consul KV store.

Set the following environment variables to access your Nomad cluster with the user token created during setup:

export NOMAD_ADDR=$(terraform output -raw lb_address_consul_nomad):4646
export NOMAD_TOKEN=$(cat nomad.token)


The Nomad UI can be accessed at http://ar-nomad-server-lb-2016482221.ca-central-1.elb.amazonaws.com:4646/ui
with the bootstrap token: 0853c07b-5718-8b0a-dad8-e3d6715bbd9f

Copy the export commands from the output, paste them into your terminal, and press the Enter key.

$ export NOMAD_ADDR=$(terraform output -raw lb_address_consul_nomad):4646 && export NOMAD_TOKEN=$(cat nomad.token)

$ export NOMAD_ADDR=$(terraform output -raw lb_address_consul_nomad):4646 && export NOMAD_TOKEN=$(cat nomad.token)

Finally, verify connectivity to the cluster by running a Nomad command.

$ nomad node status
ID        DC   Name              Class   Drain  Eligibility  Status
0c40d12b  dc1  ip-172-31-12-109  <none>  false  eligible     ready
d51c20d9  dc1  ip-172-31-7-192   <none>  false  eligible     ready
bacfe7b3  dc1  ip-172-31-7-84    <none>  false  eligible     ready

$ nomad node status
ID        DC   Name              Class   Drain  Eligibility  Status
0c40d12b  dc1  ip-172-31-12-109  <none>  false  eligible     ready
d51c20d9  dc1  ip-172-31-7-192   <none>  false  eligible     ready
bacfe7b3  dc1  ip-172-31-7-84    <none>  false  eligible     ready

»Add security groups for HashiCups

The Nomad cluster module sets up the minimum amount of ingress rules necessary for Nomad and Consul to function by opening up access to Consul on port 8500 and Nomad on port 4646 through AWS security groups. Additional rules for the HashiCups ports (80, 3000, 5432, 8080, 8081, and 9090) need to be added for the application to run properly.

Open the secgroups.tf file, add the following configuration to the end of it, and save the file. This attaches the security group defined in the same file to the Nomad clients running in the cluster.

resource "aws_network_interface_sg_attachment" "sg_attachment" {
  for_each = toset(module.nomad-cluster.nomad_client_network_ids)
  security_group_id    = aws_security_group.clients_security_group.id
  network_interface_id = each.value
}

resource "aws_network_interface_sg_attachment" "sg_attachment" {
  for_each = toset(module.nomad-cluster.nomad_client_network_ids)
  security_group_id    = aws_security_group.clients_security_group.id
  network_interface_id = each.value
}

Run Terraform again to update the client machines. Respond yes to the prompt to confirm the operation.

$ terraform apply
Terraform will perform the following actions:

  # aws_network_interface_sg_attachment.sg_attachment["eni-04fa961abdea856cc"] will be created
  + resource "aws_network_interface_sg_attachment" "sg_attachment" {
      + id                   = (known after apply)
      + network_interface_id = "eni-04fa961abdea856cc"
      + security_group_id    = "sg-00dfa5b3b913d7418"
    }

  # aws_network_interface_sg_attachment.sg_attachment["eni-08c88cf16277e295d"] will be created
  + resource "aws_network_interface_sg_attachment" "sg_attachment" {
      + id                   = (known after apply)
      + network_interface_id = "eni-08c88cf16277e295d"
      + security_group_id    = "sg-00dfa5b3b913d7418"
    }

  # aws_network_interface_sg_attachment.sg_attachment["eni-0abedc09c1e159842"] will be created
  + resource "aws_network_interface_sg_attachment" "sg_attachment" {
      + id                   = (known after apply)
      + network_interface_id = "eni-0abedc09c1e159842"
      + security_group_id    = "sg-00dfa5b3b913d7418"
    }

Plan: 3 to add, 0 to change, 0 to destroy.

aws_network_interface_sg_attachment.sg_attachment["eni-08c88cf16277e295d"]: Creating...
aws_network_interface_sg_attachment.sg_attachment["eni-0abedc09c1e159842"]: Creating...
aws_network_interface_sg_attachment.sg_attachment["eni-04fa961abdea856cc"]: Creating...
aws_network_interface_sg_attachment.sg_attachment["eni-0abedc09c1e159842"]: Creation complete after 0s [id=sg-00dfa5b3b913d7418_eni-0abedc09c1e159842]
aws_network_interface_sg_attachment.sg_attachment["eni-04fa961abdea856cc"]: Creation complete after 0s [id=sg-00dfa5b3b913d7418_eni-04fa961abdea856cc]
aws_network_interface_sg_attachment.sg_attachment["eni-08c88cf16277e295d"]: Creation complete after 0s [id=sg-00dfa5b3b913d7418_eni-08c88cf16277e295d]

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

$ terraform apply
Terraform will perform the following actions:

  # aws_network_interface_sg_attachment.sg_attachment["eni-04fa961abdea856cc"] will be created
  + resource "aws_network_interface_sg_attachment" "sg_attachment" {
      + id                   = (known after apply)
      + network_interface_id = "eni-04fa961abdea856cc"
      + security_group_id    = "sg-00dfa5b3b913d7418"
    }

  # aws_network_interface_sg_attachment.sg_attachment["eni-08c88cf16277e295d"] will be created
  + resource "aws_network_interface_sg_attachment" "sg_attachment" {
      + id                   = (known after apply)
      + network_interface_id = "eni-08c88cf16277e295d"
      + security_group_id    = "sg-00dfa5b3b913d7418"
    }

  # aws_network_interface_sg_attachment.sg_attachment["eni-0abedc09c1e159842"] will be created
  + resource "aws_network_interface_sg_attachment" "sg_attachment" {
      + id                   = (known after apply)
      + network_interface_id = "eni-0abedc09c1e159842"
      + security_group_id    = "sg-00dfa5b3b913d7418"
    }

Plan: 3 to add, 0 to change, 0 to destroy.

aws_network_interface_sg_attachment.sg_attachment["eni-08c88cf16277e295d"]: Creating...
aws_network_interface_sg_attachment.sg_attachment["eni-0abedc09c1e159842"]: Creating...
aws_network_interface_sg_attachment.sg_attachment["eni-04fa961abdea856cc"]: Creating...
aws_network_interface_sg_attachment.sg_attachment["eni-0abedc09c1e159842"]: Creation complete after 0s [id=sg-00dfa5b3b913d7418_eni-0abedc09c1e159842]
aws_network_interface_sg_attachment.sg_attachment["eni-04fa961abdea856cc"]: Creation complete after 0s [id=sg-00dfa5b3b913d7418_eni-04fa961abdea856cc]
aws_network_interface_sg_attachment.sg_attachment["eni-08c88cf16277e295d"]: Creation complete after 0s [id=sg-00dfa5b3b913d7418_eni-08c88cf16277e295d]

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

»Deploy HashiCups

The hashicups.hcl job spec file configures the HashiCups application.

Submit the job file to Nomad. Some services may report as unhealthy at first but will become healthy soon after.

Note: If you are running a Nomad cluster that is not on AWS, you will need to use a different attribute for each of the services' addresses because the attr.unique.platform.aws.public-ipv4 attribute is specific to AWS. For local dev clusters, you can use attr.unique.network.ip-address. Check the Client Detail page in the Nomad UI to see a full list of the available attributes.

$ nomad job run hashicups.hcl
==> 2022-08-10T14:57:06-04:00: Monitoring evaluation "972d3a90"
    2022-08-10T14:57:06-04:00: Evaluation triggered by job "hashicups"
    2022-08-10T14:57:06-04:00: Allocation "2b71c82b" created: node "bacfe7b3", group "product-api"
    2022-08-10T14:57:06-04:00: Allocation "39b4797d" created: node "bacfe7b3", group "db"
    2022-08-10T14:57:06-04:00: Allocation "8c7efb80" created: node "d51c20d9", group "nginx"
    2022-08-10T14:57:06-04:00: Allocation "9c6ce57a" created: node "d51c20d9", group "public-api"
    2022-08-10T14:57:06-04:00: Allocation "ddc0691d" created: node "d51c20d9", group "frontend"
    2022-08-10T14:57:06-04:00: Allocation "e75d82a7" created: node "bacfe7b3", group "payments-api"
    2022-08-10T14:57:07-04:00: Evaluation within deployment: "852e99c4"
    2022-08-10T14:57:07-04:00: Evaluation status changed: "pending" -> "complete"
==> 2022-08-10T14:57:07-04:00: Evaluation "972d3a90" finished with status "complete"
==> 2022-08-10T14:57:07-04:00: Monitoring deployment "852e99c4"
  ✓ Deployment "852e99c4" successful
    
    2022-08-10T14:58:42-04:00
    ID          = 852e99c4
    Job ID      = hashicups
    Job Version = 0
    Status      = successful
    Description = Deployment completed successfully
    
    Deployed
    Task Group    Desired  Placed  Healthy  Unhealthy  Progress Deadline
    db            1        1       1        0          2022-08-10T19:07:41Z
    frontend      1        1       1        0          2022-08-10T19:07:38Z
    nginx         1        2       1        1          2022-08-10T19:08:40Z
    payments-api  1        1       1        0          2022-08-10T19:07:41Z
    product-api   1        2       1        1          2022-08-10T19:08:14Z
    public-api    1        1       1        0          2022-08-10T19:07:28Z

$ nomad job run hashicups.hcl
==> 2022-08-10T14:57:06-04:00: Monitoring evaluation "972d3a90"
    2022-08-10T14:57:06-04:00: Evaluation triggered by job "hashicups"
    2022-08-10T14:57:06-04:00: Allocation "2b71c82b" created: node "bacfe7b3", group "product-api"
    2022-08-10T14:57:06-04:00: Allocation "39b4797d" created: node "bacfe7b3", group "db"
    2022-08-10T14:57:06-04:00: Allocation "8c7efb80" created: node "d51c20d9", group "nginx"
    2022-08-10T14:57:06-04:00: Allocation "9c6ce57a" created: node "d51c20d9", group "public-api"
    2022-08-10T14:57:06-04:00: Allocation "ddc0691d" created: node "d51c20d9", group "frontend"
    2022-08-10T14:57:06-04:00: Allocation "e75d82a7" created: node "bacfe7b3", group "payments-api"
    2022-08-10T14:57:07-04:00: Evaluation within deployment: "852e99c4"
    2022-08-10T14:57:07-04:00: Evaluation status changed: "pending" -> "complete"
==> 2022-08-10T14:57:07-04:00: Evaluation "972d3a90" finished with status "complete"
==> 2022-08-10T14:57:07-04:00: Monitoring deployment "852e99c4"
  ✓ Deployment "852e99c4" successful
    
    2022-08-10T14:58:42-04:00
    ID          = 852e99c4
    Job ID      = hashicups
    Job Version = 0
    Status      = successful
    Description = Deployment completed successfully
    
    Deployed
    Task Group    Desired  Placed  Healthy  Unhealthy  Progress Deadline
    db            1        1       1        0          2022-08-10T19:07:41Z
    frontend      1        1       1        0          2022-08-10T19:07:38Z
    nginx         1        2       1        1          2022-08-10T19:08:40Z
    payments-api  1        1       1        0          2022-08-10T19:07:41Z
    product-api   1        2       1        1          2022-08-10T19:08:14Z
    public-api    1        1       1        0          2022-08-10T19:07:28Z

When the deployment is complete, list the HashiCups services.

$ nomad service list
Service Name  Tags
database      []
frontend      []
nginx         []
payments-api  []
product-api   []
public-api    []

$ nomad service list
Service Name  Tags
database      []
frontend      []
nginx         []
payments-api  []
product-api   []
public-api    []

Inspect the nginx service, copy the public address listed, and open it in your browser to see the HashiCups application.

$ nomad service info nginx
Job ID     Address                                                Tags  Node ID   Alloc ID
hashicups  ec2-3-96-66-175.ca-central-1.compute.amazonaws.com:80  []    bacfe7b3  382cdc86

$ nomad service info nginx
Job ID     Address                                                Tags  Node ID   Alloc ID
hashicups  ec2-3-96-66-175.ca-central-1.compute.amazonaws.com:80  []    bacfe7b3  382cdc86

»Enable service load balancing

To see the service load balancing available in Nomad, you will modify the HashiCups job spec to create an additional instance of the frontend and nginx services. Each of the nginx services will communicate with a different instance of frontend.

»Modify the HashiCups job

Open hashicups.hcl. Add the count attribute to the frontend service group.

group "frontend" {
  count = 2
  network {
    port "frontend" {
      static = var.frontend_port
    }
  }
  # ...
}

hashicups.hcl

group "frontend" {
  count = 2
  network {
    port "frontend" {
      static = var.frontend_port
    }
  }
  # ...
}

Then, add the same attribute to the nginx group.

group "nginx" {
  count = 2
  network {
    port "nginx" {
      static = var.nginx_port
    }
  }
  # ...
}

hashicups.hcl

group "nginx" {
  count = 2
  network {
    port "nginx" {
      static = var.nginx_port
    }
  }
  # ...
}

Finally, in the template block of the nginx group, add a line for the $allocID variable and modify the nomadService to match the highlighted example below. Save the file.

group "nginx" {
  # ...
  task "nginx" {
    # ...
    template {
        data =  <<EOF
proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=STATIC:10m inactive=7d use_temp_path=off;
upstream frontend_upstream {
  {{ $allocID := env "NOMAD_ALLOC_ID" }}
  {{ range nomadService 1 $allocID "frontend" }}
    server {{ .Address }}:{{ .Port }};
  {{ end }}
}
    }
  }
  # ...
}

hashicups.hcl

group "nginx" {
  # ...
  task "nginx" {
    # ...
    template {
        data =  <<EOF
proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=STATIC:10m inactive=7d use_temp_path=off;
upstream frontend_upstream {
  {{ $allocID := env "NOMAD_ALLOC_ID" }}
  {{ range nomadService 1 $allocID "frontend" }}
    server {{ .Address }}:{{ .Port }};
  {{ end }}
}
    }
  }
  # ...
}

Submit the job file to Nomad.

$ nomad job run hashicups.hcl
==> 2022-08-10T15:01:24-04:00: Monitoring evaluation "601e50cd"
    2022-08-10T15:01:24-04:00: Evaluation triggered by job "hashicups"
    2022-08-10T15:01:24-04:00: Allocation "e75d82a7" modified: node "bacfe7b3", group "payments-api"
    2022-08-10T15:01:24-04:00: Allocation "2eef23d1" created: node "0c40d12b", group "nginx"
    2022-08-10T15:01:24-04:00: Allocation "db819fd3" created: node "0c40d12b", group "frontend"
    2022-08-10T15:01:24-04:00: Allocation "a5bbf109" modified: node "d51c20d9", group "product-api"
    2022-08-10T15:01:24-04:00: Allocation "39b4797d" modified: node "bacfe7b3", group "db"
    2022-08-10T15:01:24-04:00: Allocation "9c6ce57a" modified: node "d51c20d9", group "public-api"
    2022-08-10T15:01:24-04:00: Allocation "ddc0691d" modified: node "d51c20d9", group "frontend"
    2022-08-10T15:01:25-04:00: Evaluation within deployment: "ea8bfb0f"
    2022-08-10T15:01:25-04:00: Evaluation status changed: "pending" -> "complete"
==> 2022-08-10T15:01:25-04:00: Evaluation "601e50cd" finished with status "complete"
==> 2022-08-10T15:01:25-04:00: Monitoring deployment "ea8bfb0f"
  ✓ Deployment "ea8bfb0f" successful
    
    2022-08-10T15:01:54-04:00
    ID          = ea8bfb0f
    Job ID      = hashicups
    Job Version = 1
    Status      = successful
    Description = Deployment completed successfully
    
    Deployed
    Task Group    Desired  Placed  Healthy  Unhealthy  Progress Deadline
    db            1        1       1        0          2022-08-10T19:11:34Z
    frontend      2        2       2        0          2022-08-10T19:11:48Z
    nginx         2        2       2        0          2022-08-10T19:11:53Z
    payments-api  1        1       1        0          2022-08-10T19:11:34Z
    product-api   1        1       1        0          2022-08-10T19:11:34Z
    public-api    1        1       1        0          2022-08-10T19:11:34Z

$ nomad job run hashicups.hcl
==> 2022-08-10T15:01:24-04:00: Monitoring evaluation "601e50cd"
    2022-08-10T15:01:24-04:00: Evaluation triggered by job "hashicups"
    2022-08-10T15:01:24-04:00: Allocation "e75d82a7" modified: node "bacfe7b3", group "payments-api"
    2022-08-10T15:01:24-04:00: Allocation "2eef23d1" created: node "0c40d12b", group "nginx"
    2022-08-10T15:01:24-04:00: Allocation "db819fd3" created: node "0c40d12b", group "frontend"
    2022-08-10T15:01:24-04:00: Allocation "a5bbf109" modified: node "d51c20d9", group "product-api"
    2022-08-10T15:01:24-04:00: Allocation "39b4797d" modified: node "bacfe7b3", group "db"
    2022-08-10T15:01:24-04:00: Allocation "9c6ce57a" modified: node "d51c20d9", group "public-api"
    2022-08-10T15:01:24-04:00: Allocation "ddc0691d" modified: node "d51c20d9", group "frontend"
    2022-08-10T15:01:25-04:00: Evaluation within deployment: "ea8bfb0f"
    2022-08-10T15:01:25-04:00: Evaluation status changed: "pending" -> "complete"
==> 2022-08-10T15:01:25-04:00: Evaluation "601e50cd" finished with status "complete"
==> 2022-08-10T15:01:25-04:00: Monitoring deployment "ea8bfb0f"
  ✓ Deployment "ea8bfb0f" successful
    
    2022-08-10T15:01:54-04:00
    ID          = ea8bfb0f
    Job ID      = hashicups
    Job Version = 1
    Status      = successful
    Description = Deployment completed successfully
    
    Deployed
    Task Group    Desired  Placed  Healthy  Unhealthy  Progress Deadline
    db            1        1       1        0          2022-08-10T19:11:34Z
    frontend      2        2       2        0          2022-08-10T19:11:48Z
    nginx         2        2       2        0          2022-08-10T19:11:53Z
    payments-api  1        1       1        0          2022-08-10T19:11:34Z
    product-api   1        1       1        0          2022-08-10T19:11:34Z
    public-api    1        1       1        0          2022-08-10T19:11:34Z

When the deployment is complete, inspect the nginx service. Note that there are two instances. Copy each address and open them in a different browser tab.

$ nomad service info nginx
Job ID     Address                                                Tags  Node ID   Alloc ID
hashicups  ec2-35-183-78-6.ca-central-1.compute.amazonaws.com:80  []    0c40d12b  2eef23d1
hashicups  ec2-3-96-66-175.ca-central-1.compute.amazonaws.com:80  []    bacfe7b3  61934d47

$ nomad service info nginx
Job ID     Address                                                Tags  Node ID   Alloc ID
hashicups  ec2-35-183-78-6.ca-central-1.compute.amazonaws.com:80  []    0c40d12b  2eef23d1
hashicups  ec2-3-96-66-175.ca-central-1.compute.amazonaws.com:80  []    bacfe7b3  61934d47

Scroll down to the bottom of the page. The text in the footer shows which instance of the frontend service the Nginx instance is using. Note that they are different as Nomad attaches each one to a specific nginx instance during the job creation process.

Note: If you notice that both pages are displaying the same frontend instance name, your browser may be caching the result. In this case, try a hard refresh of the page or open it in a private browser window. If that still doesn't work, you can try stopping and deleting the job with nomad job stop -purge hashicups and resubmitting with nomad job run hashicups.hcl.

Screenshot of the HashiCups application highlighting the footer text that shows frontend instance zero

»Next Steps

In this tutorial, you learned about the native service discovery present in Nomad and deployed an application that uses it.

Continue onto the next tutorial in the collection to learn how to convert from Nomad to Consul service discovery.

Back to Collection