• HashiCorp Developer

  • HashiCorp Cloud Platform
  • Terraform
  • Packer
  • Consul
  • Vault
  • Boundary
  • Nomad
  • Waypoint
  • Vagrant
Consul
  • Install
  • Tutorials
  • Documentation
  • API
  • CLI
  • Try Cloud(opens in new tab)
  • Sign up
Consul Home

Documentation

Skip to main content
  • Documentation
  • What is Consul?



    • Overview
    • Invoke Lambda Functions from Services
    • Invoke Services from Lambda Functions
      BETABETA

  • HCP Consul


  • Resources

  • Tutorial Library
  • Certifications
  • Community Forum
    (opens in new tab)
  • Support
    (opens in new tab)
  • GitHub
    (opens in new tab)
  1. Developer
  2. Consul
  3. Documentation
  4. AWS Lambda
  5. Invoke Services from Lambda Functions
  • Consul
  • v1.13.x
  • v1.12.x
  • v1.11.x
  • v1.10.x
  • v1.9.x
  • v1.8.x

ยปInvoke Services from Lambda Functions

This topic describes how to invoke services in the mesh from Lambda functions registered with Consul.

Lambda-to-mesh functionality is currently in beta: Functionality associated with beta features are subject to change. You should never use the beta release in secure environments or production scenarios. Features in beta may have performance issues, scaling issues, and limited support.

Introduction

The following steps describe the process:

  1. Deploy the destination service and mesh gateway.
  2. Deploy the Lambda extension layer.
  3. Deploy the Lambda registrator.
  4. Write the Lambda function code.
  5. Deploy the Lambda function.
  6. Invoke the Lambda function.

You must add the consul-lambda-extension extension as a Lambda layer to enable Lambda functions to send requests to mesh services. Refer to the AWS Lambda documentation for instructions on how to add layers to your Lambda functions.

The layer runs an external Lambda extension that starts a sidecar proxy. The proxy listens on one port for each upstream service and upgrades the outgoing connections to mTLS. It then proxies the requests through to mesh gateways.

Prerequisites

You must deploy the destination services and mesh gateway prior to deploying your Lambda service with the consul-lambda-extension layer.

Deploy the destination service

There are several methods for deploying services to Consul service mesh. The following example configuration deploys a service named static-server with Consul on Kubernetes.

kind: Service
apiVersion: v1
metadata:
  # Specifies the service name in Consul.
  name: static-server
spec:
  selector:
    app: static-server
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: static-server
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: static-server
spec:
  replicas: 1
  selector:
    matchLabels:
      app: static-server
  template:
    metadata:
      name: static-server
      labels:
        app: static-server
      annotations:
        'consul.hashicorp.com/connect-inject': 'true'
    spec:
      containers:
        - name: static-server
          image: hashicorp/http-echo:latest
          args:
            - -text="hello world"
            - -listen=:8080
          ports:
            - containerPort: 8080
              name: http
      serviceAccountName: static-server

Deploy the mesh gateway

The mesh gateway must be running and registered to the Lambda functionโ€™s Consul datacenter. Refer to the following documentation and tutorials for instructions:

  • Mesh Gateways between WAN-Federated Datacenters
  • Mesh Gateways between Admin Partitions
  • Mesh Gateways between Peered Clusters
  • Connect Services Across Datacenters with Mesh Gateways

Deploy the Lambda extension layer

The consul-lambda-extension extension runs during the Init phase of the Lambda function execution. The extension retrieves the data that the Lambda registrator has been configured to store from AWS Parameter Store and creates a lightweight TCP proxy. The proxy creates a local listener for each upstream defined in the CONSUL_SERVICE_UPSTREAMS environment variable.

The extension periodically retrieves the data from the AWS Parameter Store so that the function can process requests. When the Lambda function receives a shutdown event, the extension also stops.

  1. Download the consul-lambda-extension extension from releases.hashicorp.com:

    curl -o consul-lambda-extension_<version>_linux_amd64.zip https://releases.hashicorp.com/consul-lambda/<version>/consul-lambda-extension_<version>_linux_amd64.zip
    
  2. Create the AWS Lambda layer in the same AWS region as the Lambda function. You can create the layer manually using the AWS CLI or AWS Console, but we recommend using Terraform:

    consul-lambda-extension.tf
    resource "aws_lambda_layer_version" "consul_lambda_extension" {
        layer_name       = "consul-lambda-extension"
        filename         = "consul-lambda-extension_<version>_linux_amd64.zip"
        source_code_hash = filebase64sha256("consul-lambda-extension_<version>_linux_amd64.zip")
        description      = "Consul service mesh extension for AWS Lambda"
    }
    

Deploy the Lambda registrator

Configure and deploy the Lambda registrator. Refer to the registrator configuration documentation and the registrator deployment documentation for instructions.

Write the Lambda function code

Refer to the AWS Lambda documentation for instructions on how to write a Lambda function. In the following example, the function calls an upstream service on port 2345:

package main

import (
    "context"
    "io"
    "fmt"
    "net/http"
    "github.com/aws/aws-lambda-go/lambda"
)    

type Response struct {
    StatusCode int  `json:"statusCode"`
    Body    string `json:"body"`
}

func HandleRequest(ctx context.Context, _ interface{}) (Response, error) {
    resp, err := http.Get("http://localhost:2345")
    fmt.Println("Got response", resp)
    if err != nil {
     return Response{StatusCode: 500, Body: "Something bad happened"}, err
    }

    if resp.StatusCode != 200 {
     return Response{StatusCode: resp.StatusCode, Body: resp.Status}, err
    }

    defer resp.Body.Close()

    b, err := io.ReadAll(resp.Body)
    if err != nil {
     return Response{StatusCode: 500, Body: "Error decoding body"}, err
    }

    return Response{StatusCode: 200, Body: string(b)}, nil
}

func main() {
    lambda.Start(HandleRequest)
}

Deploy the Lambda function

  1. Create and apply an IAM policy that allows the Lambda functionโ€™s role to fetch the Lambda extensionโ€™s data from the AWS Parameter Store. The following example, creates an IAM role for the Lambda function, creates an IAM policy with the necessary permissions and attaches the policy to the role:

    lambda-iam-policy.tf
    resource "aws_iam_role" "lambda" {
    name = "lambda-role"
    
    assume_role_policy = <<EOF
    {
    "Version": "2012-10-17",
    "Statement": [
        {
        "Action": "sts:AssumeRole",
        "Principal": {
            "Service": "lambda.amazonaws.com"
        },
        "Effect": "Allow",
        "Sid": ""
        }
    ]
    }
    EOF
    }
    
    resource "aws_iam_policy" "lambda" {
    name        = "lambda-policy"
    path        = "/"
    description = "IAM policy lambda"
    
    policy = <<EOF
    {
    "Version": "2012-10-17",
    "Statement": [
        {
        "Effect": "Allow",
        "Action": [
            "ssm:GetParameter"
        ],
        "Resource": "arn:aws:ssm:*:*:parameter${local.this_lambdas_extension_data_path}"
        }
    ]
    }
    EOF
    }
    
    resource "aws_iam_role_policy_attachment" "lambda" {
    role    = aws_iam_role.lambda.name
    policy_arn = aws_iam_policy.lambda.arn
    }
    
  2. Configure and deploy the Lambda function. Refer to the Lambda extension configuration reference for information about all available options. There are several methods for deploying Lambda functions. The following example uses Terraform to deploy a function that can invoke the static-server upstream service using mTLS data stored under the /lambda_extension_data prefix:

    lambda-function.tf
    resource "aws_lambda_function" "example" {
    โ€ฆ
    function_name = "lambda"
    role = aws_iam_role.lambda.arn
    tags = {
        "serverless.consul.hashicorp.com/v1alpha1/lambda/enabled" = "true"
    }
    variables = {
        environment = {
          CONSUL_MESH_GATEWAY_URI = var.mesh_gateway_http_addr
          CONSUL_SERVICE_UPSTREAMS = "static-server:2345:dc1"
          CONSUL_EXTENSION_DATA_PREFIX = "/lambda_extension_data"
        }
    }
    layers = [aws_lambda_layer_version.consul_lambda_extension.arn]  
    
  3. Run the terraform apply command and Consul automatically configures a service for the Lambda function.

Lambda extension configuration

Define the following environment variables in your Lambda functions to configure the Lambda extension. The variables apply to each Lambda function in your environment:

VariableDescriptionDefault
CONSUL_MESH_GATEWAY_URISpecifies the URI where the mesh gateways that the plugin makes requests are running. The mesh gateway should be registered in the same Consul datacenter and partition that the service is running in. For optimal performance, this mesh gateway should run in the same AWS region.none
CONSUL_EXTENSION_DATA_PREFIXSpecifies the prefix that the plugin pulls configuration data from. The data must be located in the following directory:
โ€œ${CONSUL_EXTENSION_DATA_PREFIX}/${CONSUL_SERVICE_PARTITION}/${CONSUL_SERVICE_NAMESPACE}/<lambda-function-name>โ€
none
CONSUL_SERVICE_NAMESPACESpecifies the Consul namespace the service is registered into.default
CONSUL_SERVICE_PARTITIONSpecifies the Consul partition the service is registered into.default
CONSUL_REFRESH_FREQUENCYSpecifies the amount of time the extension waits before re-pulling data from the Parameter Store. Use Go time.Duration string values, for example, โ€30sโ€.
The time is added to the duration configured in the Lambda registrator sync_frequency_in_minutes configuration. Refer to Lambda registrator configuration options. The combined configurations determine how stale the data may become. Lambda functions can run for up to 14 hours, so we recommend configuring a value that results in acceptable staleness for certificates.
โ€œ5mโ€
CONSUL_SERVICE_UPSTREAMSSpecifies a comma-separated list of upstream services that the Lambda function can call. Specify the value as an unlabelled annotation according to the consul.hashicorp.com/connect-service-upstreams annotation format in Consul on Kubernetes. For example, "[service-name]:[port]:[optional-datacenter]"none

Invoke the Lambda function

If intentions are enabled in the Consul service mesh, you must create an intention that allows the Lambda function's Consul service to invoke all upstream services prior to invoking the Lambda function. Refer to Service Mesh Intentions for additional information.

There are several ways to invoke Lambda functions. In the following example, the aws lambda invoke CLI command invokes the function:

$ aws lambda invoke --function-name lambda /dev/stdout | cat
Edit this page on GitHub

On this page

  1. Invoke Services from Lambda Functions
  2. Introduction
  3. Prerequisites
  4. Deploy the Lambda extension layer
  5. Deploy the Lambda registrator
  6. Write the Lambda function code
  7. Deploy the Lambda function
  8. Invoke the Lambda function
Give Feedback(opens in new tab)
  • Certifications
  • System Status
  • Terms of Use
  • Security
  • Privacy
  • Trademark Policy
  • Trade Controls
  • Give Feedback(opens in new tab)