Using AWS Lambda Layers to implement sub-millisecond static caches

Paul Heath
3 min readFeb 24, 2020

--

Leveraging Lambda Layers for Lightning Lookups !!

In an AWS serverless environment, the default solution for very low latency data access is usually DynamoDB.

Read response times depend on a variety of things, but for round numbers lets call it in the 10–20 milliseconds range (what I have experienced), to perform a keyed lookup from Lambda. What’s not to like.

This article discusses an approach to achieve sub-millisecond performance on datasets still in the multi-1000 records. This opens the door to more use cases, offers more control of data and access, and of course there’s raw performance.

TL;DR — Example

  • 7000+ Keys
  • 10MB total Javascript file size (in a Lambda Layer)
  • Lambda Memory Settings: 1536 MB
  • Lookup performance: Less than 1/10 millisecond

Environment

AWS Lambda, Lambda Layers, Node.js 12.x

Sample Use Case

Determine entities in a chatbot scenario.

Entities may include: airports, cities, attractions, zip codes, profanity, stop words etc. Each dataset contains many 100’s or 1000’s of records and 10'sMB+ of raw data.

For an incoming (tokenized) user input, perform many lookups to determine if a word, or sequence of words, is a valid entity. This is a lookup-intensive task, so typical cache implementations could get slow. Shaving a few 100ms from such a task is what we’re going for.

Data Preparation

To prepare the terrain for creating a lambda layer. In your own filesystem:

  • Create and cd to directory: nodejs
  • npm init -y
  • Create and cd to directory: node_modules
  • Create and cd to directory: cacheData (named whatever you like)
  • Create a Javascript file, let’s call it: airportData.js

Sample airportData.js

module.exports = {
airports: {
"DEN": {
"airportName": "Denver International Airport",
"latitude": "39.86169815",
"longitude": "-104.6729965,
"countryCode": "US"
},
"DFW": {
"airportName": "Dallas Fort Worth Intl Airport",
"latitude": "32.896801",
"longitude": "-97.038002",
"countryCode": "US"
}
}
}

Create the Lambda Layer

Zip the above nodejs directory, name the zip file whatever you like. Let’s go with layer1.zip for our purposes

  • In AWS Lambda dashboard: Create Layer
  • Enter the layer name, e.g. layer1 (whatever you like)
  • Upload the (layer1.zip) file
  • Specify the runtime, e.g. node.js 12.x
  • Click Create button

Accessing the data from Lambda

In your calling function…

// Lambda knows how to find this
const airportData = require('cacheData/airportData');
// Reference the JS object containing the keys/data
const airports = airportData.airports;
let airport = airports['DFW']; // Keyed lookupconsole.log('DFW Airport: ' + JSON.stringify(airport, null, 2));

Grow the Cache

Add multiple files and/or directories into this nodejs/node_modules directory structure to expand the number of cached data objects, re-zip & upload into the already-created layer by creating a new version.

TIP: Don’t forget to point your lambda function to this newly created version to access the new/updated files.

Timing Utility

To verify your (sub millisecond) performance:

const startTimer = () => {
return process.hrtime();
}
const reportTimer = (start) => {
const endTime = process.hrtime(start);
const seconds = endTime[0];
// convert nanoseconds to milliseconds
const ms = endTime[1] / 1000000;
return `${seconds}s , ${ms}ms`;
}
const start1 = startTimer();let airport = airports['DFW']console.log(reportTimer(start1) +
' for DFW Airport: ' + JSON.stringify(airport, null, 2));

NOTES

The size of the data objects may impact cold-start performance, although should be minimally invasive. Will leave that for you to verify your scenario.

In my app, I have 14 JS objects (files) spread across 6 directories.

  • Zip file size: 5MB
  • Underlying data size: 30MB
  • Cold start typically < 2s.
  • Lookup performance: sub millisecond

Counterbalance

In case you think I’m obsessed with speed, here is a loaf of bread I baked recently. End to end, prep-to-completion time — approx 16 hours. Some things are worth waiting for. Response times on the Internet are not like bread!

Overnight White Artisan Bread

--

--

Paul Heath

2x CTO Co-Founder from the travel world. Retired but can't help still coding & researching the cool tech.