Nomad node pool specification

Node pools group nodes, letting you restrict which jobs are able to place allocations in a given set of nodes. Specify node pools as HCL files and submit them with the nomad node pool apply CLI command. Unlike job specifications, node pool specifications do not support HCL2 features like functions.

This example is a node pool specification generated with the nomad node pool init command.

node_pool "example" {

  description = "Example node pool"

  # meta is optional metadata on the node pool, defined as key-value pairs.
  # The scheduler does not use node pool metadata as part of scheduling.
  meta {
    environment = "prod"
    owner       = "sre"
  }

  # node_identity_ttl is an optional parameter that sets the TTL for all node
  # identities issued to nodes in this node pool. The value must be a valid
  # duration string (e.g. "30m", "1h", "24h"). If not set, the default value is
  # "24h".
  # node_identity_ttl = "24h"

  # The scheduler configuration options specific to this node pool. This block
  # supports a subset of the fields supported in the global scheduler
  # configuration as described at:
  # https://developer.hashicorp.com/nomad/commands/operator/scheduler/set-config
  #
  # * scheduler_algorithm is the scheduling algorithm to use for the pool.
  #   If not defined, the global cluster scheduling algorithm is used.
  #
  # Available only in Nomad Enterprise.

  # scheduler_config {
  #   scheduler_algorithm = "spread"
  # }
}

Example usage if the above is in a file named pool.nomad.hcl:

$ nomad node pool apply ./pool.nomad.hcl
Successfully applied node pool "example"!

Parameters

• description (string: <optional>) - Sets a human readable description for the node pool.

• meta (map[string]string: <optional>) - Sets optional metadata on the node pool, defined as key-value pairs. The scheduler does not use node pool metadata as part of scheduling.

• node_identity_ttl (string: <optional>) - Sets the TTL for node identities issued to nodes in this node pool. The value must be a valid duration string (e.g. "30m", "1h", "24h"). If not set, the default value is "24h".

• scheduler_config (SchedulerConfig: nil) - Sets scheduler configuration options specific to the node pool. If not defined, the global scheduler configurations are used.

  Enterprise

• batch_queue (BatchQueue: nil) - Configures a batch job queue for the node pool. The queue holds the batch jobs registered in the node pool and sends them to the scheduler one at a time. If not defined, the node pool does not queue batch jobs. You cannot configure a batch job queue on the built-in default and all node pools.

scheduler_config parameters

• scheduler_algorithm (string: <optional>) - The scheduler algorithm used for this node pool. Must be one of binpack or spread.

• memory_oversubscription_enabled (bool: <optional>) - The memory oversubscription setting to use for this node pool.

batch_queue parameters

The batch_queue block must contain one of the following blocks, which sets the queue type:

• fifo (Fifo: nil) - Configures a FIFO queue, which sends jobs to the scheduler in the order that you register them.

• dynamic_priority (DynamicPriority: nil) - Configures a dynamic priority queue, which sends jobs to the scheduler in order of their adjusted priority. Refer to Adjusted priority for how the queue calculates it.

The batch_queue block has the following optional parameter:

• constraint_timeout (string: <optional>) - Specifies how long the queue waits for a job that Nomad cannot place because of its constraints, as a duration string such as "5m". The timeout starts when the scheduler creates the job's blocked evaluation. When the timeout expires, the queue sends the next job to the scheduler, and the blocked job waits for placement outside the queue. If not set, the queue waits until Nomad places the job or you stop it. The timeout does not apply to jobs that wait for resources.

This example configures a dynamic priority queue that treats each namespace as a tenant.

node_pool "batch" {
  description = "Clients that run batch workloads"

  batch_queue {
    dynamic_priority {
      calc_interval = "10s"

      tenant_fairshare {
        tenant_type   = "namespace"
        cpu_weight    = 25
        memory_weight = 25
      }

      age {
        weight = 10
        max    = "1h"
      }

      job_size {
        cpu_weight    = 10
        cpu_max       = 1000
        memory_weight = 10
        memory_max    = 1024
      }
    }
  }
}

fifo parameters

The fifo block has no parameters. Specify an empty block.

batch_queue {
  fifo {}
}

dynamic_priority parameters

• calc_interval (string: <required>) - Specifies how often the queue recalculates each tenant's fairshare and the adjusted priority of every queued job, as a duration string such as "30s". Must be greater than zero.

• tenant_fairshare (TenantFairshare: <required>) - Configures how the queue groups jobs into tenants and how much each tenant's share of allocated resources affects the priority of its jobs.

• age (Age: <required>) - Configures how much the time a job waits in the queue affects its priority. Specify an empty block, age {}, to leave job age out of the calculation.

• job_size (JobSize: <required>) - Configures how much the resources that a job requests affect its priority. Specify an empty block, job_size {}, to leave job size out of the calculation.

tenant_fairshare parameters

• tenant_type (string: <required>) - Specifies how the queue groups jobs into tenants. Must be one of namespace or metadata. With namespace, each namespace is a tenant. With metadata, each distinct value of the job meta key named in metadata_key is a tenant, and jobs that do not set that key skip the queue.

• metadata_key (string: "") - Specifies the job meta key whose value identifies a job's tenant. Required when tenant_type is metadata.

• cpu_weight (int: <required>) - Specifies the maximum amount that a tenant's share of allocated CPU adds to the priority of its jobs. Tenants whose jobs hold a smaller share of the CPU allocated to batch jobs in the node pool receive more of this weight. If 0, allocated CPU does not affect priority.

• memory_weight (int: <required>) - Specifies the maximum amount that a tenant's share of allocated memory adds to the priority of its jobs. Tenants whose jobs hold a smaller share of the memory allocated to batch jobs in the node pool receive more of this weight. If 0, allocated memory does not affect priority.

• exclude_alloc_statuses (array<string>: []) - Specifies the allocation client statuses that the queue excludes when it adds up the resources allocated to each tenant. For example, ["failed", "lost"] excludes failed and lost allocations. If empty, the queue counts every allocation of the tenant's batch jobs. Refer to Tenant fairshare for details.

age parameters

• weight (int: 0) - Specifies the maximum amount that the time a job waits in the queue adds to its priority. If 0, job age does not affect priority. Set max when you set this parameter. If max is not set, every job receives the full weight as soon as it enters the queue.

• max (string: "") - Specifies the wait time, as a duration string such as "12h", at which a job receives the full weight.

job_size parameters

• cpu_weight (int: 0) - Specifies the maximum amount that a job's requested CPU adds to its priority. Jobs that request less CPU receive more of this weight. If 0, requested CPU does not affect priority. Set cpu_max when you set this parameter.

• cpu_max (int: 0) - Specifies the requested CPU, in MHz, at or above which a job receives none of the cpu_weight. A job's requested CPU is the sum of each task's cpu resource multiplied by its group's count. A task that sets cores instead of cpu adds 0 MHz to the sum. Must be set if cpu_weight is greater than 0.

• memory_weight (int: 0) - Specifies the maximum amount that a job's requested memory adds to its priority. Jobs that request less memory receive more of this weight. If 0, requested memory does not affect priority. Set memory_max when you set this parameter.

• memory_max (int: 0) - Specifies the requested memory, in MB, at or above which a job receives none of the memory_weight. A job's requested memory is the sum of each task's memory resource multiplied by its group's count. Must be set if memory_weight is greater than 0.