Nomad 1.11.x release notes

We are pleased to announce the following Nomad updates.

1.11.1 release highlights

Client disk_free_mb deprecated

We deprecated the client disk_free_mb parameter. Use the client.reserved.disk parameter instead.

Nomad now calculates the storage available for scheduling using only the client.reserved.disk value. The previous strategy using free disk space could lead to incorrect values when clients with running allocations restarted. The unique.storage.bytesfree attribute has also been removed. We recommend that you reserve at least the amount of disk that is used by the host OS.

New QEMU task driver parameters

We added emulator and machine_type to the task configuration. These default to the previously used values of qemu-system-x86_64, and pc. Refer to Use the QEMU task driver in a job for details.

We also added the driver.qemu.emulators parameter to the QEMU task driver configuration. This parameter defaults to allowing any emulator to run.

Autoscaling permissions to scale policy EnterpriseEnterprise

We added the read-job and submit-recommendation permissions to the default ACL scale policy. Both are needed for the Nomad Autoscaler to run using the Nomad targets. Refer to the Nomad ACL policy specification for details.

Identity added to job specification sidecar_task block

Expose an extra workload identity to your job specification task. Nomad automatically configures Consul Service Mesh sidecars with the required identities to communicate with Consul. Specifying this block lets you add extra identities that can be consumed by third-party services. Refer to the sidecar_task block for details.

Breaking changes

Docker task driver: We removed the deprecated email auth config parameter.

Changelog

Review improvements, security fixes, and breaking changes in the changelog.

• v1.11.1

1.11.0 release highlights

Client node introduction and identity

Nomad's client node identity feature uniquely identifies each Nomad client node and provides an authentication mechanism for nodes to make RPC calls to the Nomad servers.

Introduce Nomad clients to the cluster with JWT tokens. Configure Nomad servers with introduction enforcement levels that dictate how clients join the cluster. This approach results in logs and metrics to detail introduction violations. Once registered, Nomad clients are now provided with an identity token, used for RPC communication which is periodically renewed.

The client node introduction and identity feature functions as multi-factor authentication for your Nomad clusters. It does not replace mTLS. Instead, it adds a second layer of security to prevent an unauthorized client from joining a Nomad cluster.

Using a client introduction token gives you additional control over misconfigured clients trying to join the Nomad cluster because you can specify node names, node pools, and TTLs for the tokens you generate.

Relevant documentation

• Client node identity concepts
• Client identity node pool TTL configuration option
• Use client node introduction tokens to connect clients to your Nomad server
• Client introduction server configuration options
• Client introduction monitoring detail
• Client introduction agent CLI token flag

Commands:

• nomad node intro create: Generates a node introduction token which is a signed JWT.
• nomad node identity get: View the identity claims for a node.
• nomad node identity renew: Instruct a node to renew its identity of its next heartbeat.
• nomad node pool info: View the identity TTL for nodes in the provided pool.
• nomad operator client-state: View the client persistent state in JSON, which includes the client identity.
• nomad agent command's -client-intro-token parameter: Set the JWT token used to authenticate with servers during the client's initial registration.

API:

• /v1/acl/identity/client-introduction-token: Generates a node introduction token which is a signed JWT.
• /v1/client/identity: View the identity claims for a node.
• /v1/client/identity/renew: Instruct a node to renew its identity of its next heartbeat.
• /v1/node/pool/:node_pool: View the identity TTL for nodes in the provided pool.

Artifact secrets

Use the new job specification secrets block to fetch and interpolate secrets from a specified location. Nomad automatically retrieves the contents of the secret. In your task specification, reference the variable with ${secret.secret_name.key} format.

Nomad provides built-in support to fetch secrets from Nomad and Vault. Alternately, you may build your own custom secret provider.

Relevant documentation

• Job specification secret block
• Secret provider plugin authoring guide

System job deployments

Jobs of type “system” now support deployment functionality, so you may roll out job updates in a more controlled manner. Additionally, you may now view system job deployment status in the web UI.

Relevant documentation

• Job specification update block
• Blue/Green deployment detail
• nomad deployment commands for interacting with Nomad job deployments

Scheduler improvements

We improved Nomad's scheduler observability for job authors.

Relevant documentation

• nomad eval status command: View more details on an evaluation including related evaluations, successful placements, scheduler annotations, failed placements, and preemptions.

  • The Plan Annotations table represents the intended plan before Nomad checks nodes for feasibility. This can help you narrow down where in the scheduling pipeline unexpected results are coming from.
  • The Nomad repository scheduler README has architectural details.

• nomad alloc status command: When you run the command with the -verbose flag, the output includes information about the scheduler's placement decision, including the number of nodes evaluated and rejected, and the scoring of each node considered.

Additional census usage metrics EnterpriseEnterprise

We added several Enterprise reporting metrics.

Relevant documentation

• Automated license utilization reporting
• Agent reporting configuration disable_product_usage_reporting parameter: Disable detailed product usage metrics.
• nomad operator utilization command: If you have disabled automated reporting, use this command to manually generate the report and send it to HashiCorp.

Upgrade details

Refer to the upgrade details page for more information.

Changelog

Review improvements, security fixes, and breaking changes in the changelog.

• v1.11.0