Sentinel Validation Policies

The goal of this tutorial is to illustrate how to write Sentinel Endpoint Governing Policies (EGP) that can be used in Vault Enterprise to validate that specific keys of secrets adhere to certain formats. It includes some example policies that validate ZIP codes, state codes, and more.

There is also an example EGP that requires the delete_version_after metadata property of KV version 2 secrets to be less than 30 days (720 hours). This effectively restricts the duration of versions of secrets; after this amount of time, each version is deleted.

Challenge

When a client sends a secret to Vault, there is no way to validate that the provided value complies with the desired format. For example, the expected input value is a 5-digit numeric value, and you wish to reject values that are not.

Solution

Sentinel allows Vault Enterprise customers to solve this challenge by checking that keys with specified names adhere to the desired formats. Additionally, the EGPs can be restricted to only apply when a secret includes multiple related keys or occurs on specific Vault paths.

Personas

In an enterprise setting, developers, admins, or other members of the security team can author Sentinel EGPs like the examples in this tutorial to ensure that secrets written to Vault adhere to desired formats.

All secrets written by users who do not use a root token will be subject to the format validation rules enforced by EGPs.

Prerequisites

This tutorial is aimed primarily at developers or other personas who will author policies; you do not have to already have knowledge of the Sentinel language, but it is helpful.

You also need the following:

• A clone of the vault-guides repository to examine the example Sentinel EGPs used in this tutorial
• Download Sentinel Simulator and add it to your PATH environment variable (this tutorial uses version 0.15.6)

Prepare your environment

Before you can follow the tutorial, take some time to prepare your environment.

• Ensure that you can open a terminal session and run the sentinel binary.
• Create a project directory, change into it, and clone the vault-guides Git repository

Open a new terminal session and verify that you have a working sentinel binary.

$ sentinel version
Sentinel v0.15.6

If you encounter an error with either of these two steps, ensure that your sentinel binary matches your platform and operating system types, and that it resides in a directory named in the list output by echo $PATH.

Next, make a project directory to contain all the content used in this tutorial and change into that directory.

$ mkdir $HOME/learn-vault-sentinel-validation && cd $_

Next, clone the vault-guides repository.

$ git clone https://github.com/hashicorp/vault-guides

You are now ready to examine the example policy files and learn more about authoring Sentinel EGPs.

Validate ZIP code

Let's review the first example EGP (validate-zip-codes.sentinel). It uses a regular expression to test request data for specifically formatted fields like zipcode, zip_code, or zip-code, and ensures that their contents are correctly formatted for U.S. ZIP codes.

The policy file is located at vault-guides/governance/validation-policies/validate-zip-codes.sentinel.

Here are the contents of the policy; a detailed explanation of the code follows.

1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475# Policy that requires keys of secrets with name "zipcode", "zip-code", or
# "zip_code" to be a valid 5-digit U.S. zipcode
# For use with Vault KV v1 and v2 secrets engines

# Function that validates zip codes
validate_zip_codes = func() {

  # Print some information about the request
  # Note that these messages will only be printed when the policy is violated
  print("Namespace path:", namespace.path)
  print("Request path:", request.path)
  print("Request data:", request.data)

  # Test for request.data for kv v1

  # Test for "zipcode" key
  if "zipcode" in keys(request.data) {
    if request.data.zipcode not matches "^[0-9]{5}$" {
      print("Invalid zip code")
      return false
    }
  }

  # Test for "zip_code" key
  if "zip_code" in keys(request.data) {
    if request.data.zip_code not matches "^[0-9]{5}$" {
      print("Invalid zip code")
      return false
    }
  }

  # Test for "zip-code" key
  if "zip-code" in keys(request.data) {
    # Using request.data.zip-code gives error
    if request.data["zip-code"] not matches "^[0-9]{5}$" {
      print("Invalid zip code")
      return false
    }
  }

  # Test for request.data.data for kv v2

  # Test for "zipcode" key
  if "data" in keys(request.data) and "zipcode" in keys(request.data.data) {
    if request.data.data.zipcode not matches "^[0-9]{5}$" {
      print("Invalid zip code")
      return false
    }
  }

  # Test for "zip_code" key
  if "data" in keys(request.data) and "zip_code" in keys(request.data.data) {
    if request.data.data.zip_code not matches "^[0-9]{5}$" {
      print("Invalid zip code")
      return false
    }
  }

  # Test for "zip-code" key
  if "data" in keys(request.data) and "zip-code" in keys(request.data.data) {
    if request.data.datazip-code not matches "^[0-9]{5}$" {
      print("Invalid zip code")
      return false
    }
  }

  return true

}

# Main Rule
zip_codes_validated = validate_zip_codes()
main = rule {
  zip_codes_validated
}

At line 6, the validate_zip_codes function is defined; this function does all of the heavy lifting for this policy, and contains 6 individual conditionals defined as if statements. It returns either true or false depending on the outcomes of the 6 conditional tests within the function.

Lines 10 through 12 add some print statements for debugging purposes. They print any Enterprise Namespace value contained in the request along with the request path, and request data. It is important to understand how print statements in Sentinel policies for Vault work. They are never invoked on successful policy evaluation; instead, they are invoked only when policy evaluation fails for debugging purposes.

The first of 3 conditionals that relate to the KV version 1 secrets engine begins on line 17. It uses an if statement to check the request data zipcode field. Using a regular expression (^[0-9]{5}$), the field is tested against the U.S. ZIP code format to determine whether it contains 5 sequential numbers 0-9.

The next two conditionals beginning at lines 24 and 32 are variations of the first one, and perform essentially the same regular expression comparison, but against fields named zip_code and zip-code respectively.

The next 3 conditionals on lines 44 through 65 define the same checks as the first 3 except that they are for the KV version 2 secrets engine as evidenced by the request.data.data field.

Line 73 defines the main rule. It is the only rule defined in this policy; the output of validate_zip_codes determines this rule evaluation, with true evaluating to a passing evaluation and false evaluating to a failing evaluation.

zip_codes_validated = validate_zip_codes()
main = rule {
  zip_codes_validated
}

Write tests for passing scenarios

If you want to familiarize yourself with the tests specific to the example policies used in this tutorial, they are available in the vault-guides repository.

As a refresher on tests or to those curious about policy testing, the tests for validate-zip-codes are included here for your reference along with instructions on how to run them.

Here are the passing tests. They mock data under global and requires that the main rule evaluate to true. First is the passing test for KV version 1 secrets engine.

{
  "global": {
    "namespace": {
      "path": ""
    },
    "request": {
      "path": "kv1/address",
      "data": {
        "zip-code": "12345"
      }
    }
  },
  "test": {
    "main": true
  }
}

This is the passing test for KV version 2 secrets engine.

{
  "global": {
    "namespace": {
      "path": ""
    },
    "request": {
      "path": "kv2/address",
      "data": {
        "data": {
          "zip_code": "77777"
        }
      }
    }
  },
  "test": {
    "main": true
  }
}

Write tests for failure scenarios

Here are the failing test examples. They mock incorrect data in the same way as the passing tests.

Here is the failing test for KV version 1 secrets engine.

{
  "global": {
    "namespace": {
      "path": ""
    },
    "request": {
      "path": "kv1/address",
      "data": {
        "zip_code": "1234e"
      }
    }
  },
  "test": {
    "main": false
  }
}

Here is the failing test for KV version 2 secrets engine.

{
  "global": {
    "namespace": {
      "path": ""
    },
    "request": {
      "path": "kv2/address",
      "data": {
        "data": {
          "zipcode": "123456"
        }
      }
    }
  },
  "test": {
    "main": false
  }
}

You can test the policies with the Sentinel Simulator. To test the included test cases against example policies, first change into the directory.

$ cd vault-guides/governance/validation-policies

Execute a test on the validate-zip-codes policy.

$ sentinel test -run=zip

Successful output example:

PASS - validate-zip-codes.sentinel
  PASS - test/validate-zip-codes/fail-kv-v1.json
  PASS - test/validate-zip-codes/fail-kv-v2.json
  PASS - test/validate-zip-codes/pass-kv-v1.json
  PASS - test/validate-zip-codes/pass-kv-v2.json

For verbose output including things the policies print, add the -verbose flag to these commands.

Set version deletion policy

The policy file is located at vault-guides/governance/validation-policies/validate-delete-version-after-setting.sentinel.

Here are the contents of the policy; a detailed explanation of the code follows.

1  2  3  4  5  6  7  8  9  10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100101102103104105106107108109110111112113# Policy that validates that delete_version_after less than 30 days (720 hours)

# This is intended to be used against paths of form <path>/metadata/<secret>
# for the KV v2 secrets engine

import "strings"

parse_duration = func(duration) {

  # Process hours
  hours_segment = strings.split(duration, "h")
  if hours_segment[0] matches "^[0-9]+$" {
    # Hours were specified
    hours = int(hours_segment[0])
    if length(hours_segment) == 1 {
      # Minutes and seconds were not specified
      minutes_segment = ""
    } else {
      # Minutes and/or seconds were specified
      minutes_segment = hours_segment[1]
    }
  } else {
    # Hours were not specified
    # hours_segment will then be list with single element
    # equal to original duration
    hours = 0
    minutes_segment = hours_segment[0]
  }

  # Process minutes
  minutes_segment = strings.split(minutes_segment, "m")
  if minutes_segment[0] matches "^[0-9]+$" {
    # Minutes were specified
    minutes = int(minutes_segment[0])
    if length(minutes_segment) == 1 {
      # Seconds were not specified
      seconds_segment = ""
    } else {
      # Seconds were specified
      seconds_segment = minutes_segment[1]
    }
  } else {
    # Minutes were not specified
    # minutes_segment will then be list with single element
    # equal to hours_segment
    minutes = 0
    seconds_segment = minutes_segment[0]
  }

  # Process seconds
  seconds_segment = strings.split(seconds_segment, "s")
  if seconds_segment[0] matches "^[0-9]+$" {
    # Seconds were specified
    seconds = int(seconds_segment[0])
  } else {
    # Seconds were not specified
    seconds = 0
  }

  # Roll seconds into minutes
  if seconds >= 60 {
    minutes_from_seconds = seconds / 60
    minutes = minutes + minutes_from_seconds
    seconds = seconds % 60
  }

  # Roll minutes into hours
  if minutes >= 60 {
    hours_from_minutes = minutes / 60
    hours = hours + hours_from_minutes
    minutes = minutes % 60
  }

  print("hours:", hours)
  print("minutes:", minutes)
  print("seconds:", seconds)


  return {"hours": hours, "minutes": minutes, "seconds": seconds}

}

# Function that validates delete_version_after less than 30 days
validate_delete_version_after_setting = func() {

  # Print some information about the request
  # Note that these messages will only be printed when the policy is violated
  print("Namespace path:", namespace.path)
  print("Request path:", request.path)
  print("Request data:", request.data)

  # Only validate against paths of form <segment>/metadata/<segments>
  if request.path matches "^(.*)\\/metadata\\/(.*)$" and "delete_version_after" in keys(request.data) {
    parsed_duration = parse_duration(request.data.delete_version_after)
    hours = parsed_duration["hours"]
    #hours = int(strings.split(request.data.delete_version_after, "h")[0])
    print("hours in validation:", hours)
    if hours >= 720 {
      print("Invalid value of delete_version_after")
      print("It must be under 720 hours")
      return false
    }
  }

  return true

}

# Main Rule
delete_version_after_setting_validated = validate_delete_version_after_setting()
main = rule {
  delete_version_after_setting_validated
}

This policy code is designed to enforce policy on KV version 2 secrets engine secret versions, which are associated with paths like <path>/metadata/<secret>. It is considerably more complex than the validate-zip-codes example in that t uses an import, and two fairly complex functions.

Line 6 imports the strings import to gain additional string handling functions.

The first of the two functions, parse_duration is defined beginning on line 8. This is a utility function that accepts a string, duration that holds the value of delete_version_after for the secret version path requested.

Lines 11 through 28 use a regular expression (^[0-9]+$) to test the segments of the duration to process the hours portion.

Lines 31 through 48 use the same approach to process the minutes.

The seconds are processed on lines 51 through 58.

Seconds at or over a value of 60 are rolled to minutes between lines 61 and 65.

Likewise, minutes at or over a value of 60 are rolled to hours between lines 68 and 72.

Some debugging print statements appear on lines 74 through 76 to print the hours, minutes, and seconds values.

print("hours:", hours)
print("minutes:", minutes)
print("seconds:", seconds)

Finally on line 79 the hours, minutes, and seconds values are returned by the function.

return {"hours": hours, "minutes": minutes, "seconds": seconds}

The second function, validate_delete_version_after_setting begins on line 84.

validate_delete_version_after_setting = func() {
  # Print some information about the request
  # Note that these messages will only be printed when the policy is violated
  print("Namespace path:", namespace.path)
  print("Request path:", request.path)
  print("Request data:", request.data)
  # Only validate against paths of form <segment>/metadata/<segments>
  if request.path matches "^(.*)\\/metadata\\/(.*)$" and "delete_version_after" in keys(request.data) {
    parsed_duration = parse_duration(request.data.delete_version_after)
    hours = parsed_duration["hours"]
    #hours = int(strings.split(request.data.delete_version_after, "h")[0])
    print("hours in validation:", hours)
    if hours >= 720 {
      print("Invalid value of delete_version_after")
      print("It must be under 720 hours")
      return false
    }
  }
    return true
}

It begins by using print statements on lines 88 through 90 to print debug information on policy evaluation failure about the Enterprise Namespace path, the request path, and request data.

At line 93, it tests the request data to ensure that the path contains /metadata/ and that there is also a delete_version_after key present in the request data.

On line 94, the parse_duration function is called and the request data delete_version_after key value is passed. The result is stored in the variable parsed_duration.

The hours calculated by the parse_duration function are stored in the hours variable on line 95, and at line 98, the hours variable is tested as to whether it is greater or equal to 720. If it is found to be greater than 720, two error lines are printed at lines 99 and 100, and the function returns false at line 101.

Otherwise, the function returns true at line 105.

A global variable, delete_version_after_setting_validated is set to the return value of the validate_delete_version_after_setting function at line 110.

Finally, the main rule is defined on line 111, and contains either the value true or false based on the outcome of the validate_delete_version_after_setting function on 112.

Summary

In this tutorial, you learned how to develop advanced Sentinel EGPs that can be used in Vault Enterprise to validate the content of secret fields. In particular, you have explored the development of policies that validate zip codes, and versions of secrets the Vault KV version 2 secrets engine.

You were also provided information on how to test one of the policies with the Sentinel Simulator using test cases that were provided in the vault-guides repository.

The vault-guides/governance/validation-policies folder contains additional Sentinel examples and their test files. Explore those policies to help you guide writing more complex policies to have a finer control over incoming data.

Next Steps

You can learn more about Sentinel with Vault by exploring the Sentinel HTTP Import tutorial, which covers using the response from an HTTP request to drive policy evaluation.

Help and Reference

• Sentinel Vault overview.
• Sentinel Documentation
• Sentinel testing
• Sentinel imports