DEV Community

Cover image for How to Use Source Maps in TypeScript Lambda Functions (with Benchmarks)
AWS Community Builders profile image Matt Morgan
Matt Morgan for AWS Community Builders

Posted on • Edited on

How to Use Source Maps in TypeScript Lambda Functions (with Benchmarks)

#lambda #aws #typescript #serverless

TypeScript is a popular language for developers of all kinds and it's made its mark on the serverless space. Most of the major Lambda frameworks now have solid TypeScript support. The days of struggling with webpack configurations are mostly behind us.

Table of Contents

Stack Traces

If we're already transpiling our code to convert the TypeScript source to Lambda-friendly JavaScript, we might as well go ahead and minify and tree-shake the code as well. Smaller bundles can make deployments faster and may even help with cold start and execution time. However, this can make debugging difficult when we start seeing stack traces that look like this:

{
    "errorType": "SyntaxError",
    "errorMessage": "Unexpected end of JSON input",
    "stack": [
        "SyntaxError: Unexpected end of JSON input",
        "    at JSON.parse (<anonymous>)",
        "    at VA (/var/task/index.js:25:69708)",
        "    at Runtime.R8 [as handler] (/var/task/index.js:25:69808)",
        "    at Runtime.handleOnce (/var/runtime/Runtime.js:66:25)"
    ]
}
Enter fullscreen mode Exit fullscreen mode

The error message here lets us know that we're failing to parse a JSON string, but the stack trace itself is useless for finding the line where the code is failing. We'll have no choice but to look through our code and hope to find the error. We might be able to do a text search for JSON.parse but if that is happening in one of our dependencies, searching won't work. What's next? Add a bunch of log statements to the code? No! If we use Source Maps, we can get more useful stack traces:

{
    "errorType": "SyntaxError",
    "errorMessage": "Unexpected end of JSON input",
    "stack": [
        "SyntaxError: Unexpected end of JSON input",
        "    at JSON.parse (<anonymous>)",
        "    at VA (/fns/db.ts:39:8)",
        "    at Runtime.R8 (/fns/list.ts:6:24)",
        "    at Runtime.handleOnce (/var/runtime/Runtime.js:66:25)"
    ]
}
Enter fullscreen mode Exit fullscreen mode

Now we can see the stack includes a call from line 6 of list.ts which finally fails on line 39 of db.ts.

To enable Source Maps in our application, we need to tell our build tool to emit Source Maps and we need to enable Source Map support in our runtime.

Emitting Source Maps

Emitting Source Maps is very easy with esbuild. We simply set the boolean property in our configuration. Now when we run the build, we'll get an index.js.map file as well as our index.js. This file must be uploaded to the Lambda service. We'll see how to do that with AWS CDK, AWS SAM and the Serverless Framework a bit later in this article.

Source Map Support

Having the index.js.map file in our Lambda runtime isn't sufficient to enable Source Maps. We also have to make sure the runtime knows to make use of them. Fortunately this is very easy ever since Node.js version 12.12.0. We just have to set the --enable-source-maps command line option. Command line options can be set in AWS Lambda by setting the NODE_OPTIONS environment variable. At the time of this writing, AWS Lambda supports Node.js versions 12 and 14. AWS does not publish the minor versions in use in Lambda, but we can discover it by logging out process.version in a function. As of late January, 2022, the Node.js versions in use in Lambda in us-east-1 are v12.22.7 and v14.18.1 so we'll have no trouble using Source Maps.

If we needed to enable Source Maps in a runtime that doesn't support the native version, we could always use Source Map Support.

CDK Example

All example code is available on GitHub.

AWS CDK is my preferred tool for writing and deploying serverless applications, in part because of the aws-lambda-nodejs construct. This construct makes it very easy to work with TypeScript. It wraps esbuild and exposes options. It also supports setting environment variables.

When I'm working with multiple Lambda functions, I often find it helpful to create a single props object that then gets shared among multiple functions.

const lambdaProps = {
  architecture: Architecture.ARM_64,
  bundling: { minify: true, sourceMap: true },
  environment: {
    NODE_OPTIONS: '--enable-source-maps',
  },
  logRetention: RetentionDays.ONE_DAY,
  runtime: Runtime.NODEJS_14_X,
  memorySize: 512,
  timeout: Duration.minutes(1),
};

new NodejsFunction(this, 'FuncOne', {
  ...lambdaProps,
  entry: `${__dirname}/../fns/one.ts`,
});

new NodejsFunction(this, 'FuncTwo', {
  ...lambdaProps,
  entry: `${__dirname}/../fns/two.ts`,
});
Enter fullscreen mode Exit fullscreen mode

As we can see, it's very simple to enable Source Maps when already using AWS CDK and NodejsFunction.

Serverless Stack

Serverless Stack is a very cool value add that builds on top of AWS CDK delivering an awesome developer experience and dashboard. SST's own version of NodejsFunction, Function automatically emits source maps. Their use can be enabled simply by setting NODE_OPTIONS as described.

SAM Example

SAM support for TypeScript has been lagging for some time, but a pull request was just merged that should change that. It looks like we'll be able to add an aws_sam key in the package.json file to enable building within the SAM engine as part of sam build. Although this PR has been merged to aws-lambda-builders, the engine behind sam build, it will still need to be added to aws-sam-cli and released (to much fanfare, one expects) before it can be used with SAM.

Meanwhile - or if we're considering other options for deploying our functions - we can add an extra build step. We'll create an esbuild.ts file that transpiles the functions and then point our SAM template.yaml file at the output of that step.

import { build, BuildOptions } from 'esbuild';

const buildOptions: BuildOptions = {
  bundle: true,
  entryPoints: {
    ['create/index']: `${__dirname}/../fns/create.ts`,
    ['delete/index']: `${__dirname}/../fns/delete.ts`,
    ['list/index']: `${__dirname}/../fns/list.ts`,
  },
  minify: true,
  outbase: 'fns',
  outdir: 'sam/build',
  platform: 'node',
  sourcemap: true,
};

build(buildOptions);
Enter fullscreen mode Exit fullscreen mode

SAM templates will take everything in the CodeUri so the above code will transpile a TypeScript file at fns/list.ts and output sam/build/list/index.js and sam/build/list/index.js.map. By setting CodeUri: sam/build/list, SAM will package the two files and upload them to Lambda.

Now we just need to set the environment variable. This is easy enough to do in a SAM template. We can set it as a global so it only needs to be in the template once

Globals:
  Function:
    Environment:
      Variables:
        NODE_OPTIONS: '--enable-source-maps'
Enter fullscreen mode Exit fullscreen mode

In order to make sure we always build before deploying, we can add some npm scripts.

"scripts": {
  "build:lambda": "npm run clean && ts-node --files sam/esbuild.ts",
  "clean": "rimraf cdk.out sam/build",
  "deploy:sam": "npm run build:lambda && sam deploy --template template.yaml",
  "destroy:sam": "sam delete"
}
Enter fullscreen mode Exit fullscreen mode

This works well enough, but does take some extra effort. SAM users are no doubt eagerly awaiting better TypeScript support.

Architect

Alternately, use Architect. Architect is a 3rd party developer experience that builds on top of AWS SAM. Architect includes a TypeScript plugin.

Serverless Framework Example

The Serverless Framework is known for its plugin system. serverless-esbuild brings bundling and all the options we need to support Source Maps in TypeScript into the sls package and sls deploy commands.

The plugin is configured in our serverless.yml file.

custom:
  esbuild:
    bundle: true
    minify: true
    sourcemap: true
Enter fullscreen mode Exit fullscreen mode

And then we just point our functions at our TypeScript handlers.

functions:
  create:
    handler: fns/create.handler
Enter fullscreen mode Exit fullscreen mode

Much like SAM, Serverless lets us set global environment variables for our functions.

provider:
  name: aws
  lambdaHashingVersion: 20201221
  runtime: nodejs14.x
  environment:
    NODE_OPTIONS: '--enable-source-maps'
Enter fullscreen mode Exit fullscreen mode

Much like AWS CDK, this is a good experience for TypeScript developers. Those already using Serverless Framework should have an easy time adding Source Maps to their applications.

Benchmarks

There's a lot of guidance against using Source Maps in production because of a supposed negative performance impact. Let's measure the admittedly-simple list function and see if minification or the use of source maps has any noticeable effect.

I used autocannon to test the function at 100 concurrent executions for 30 seconds. I also used Lambda Power Tuning to find the ideal memory configuration, which proved to be 512MB. All the results are available.

Not Minified without Source Maps

The unminified function is 1.2MB in size. This is mostly from @aws-sdk/client-dynamodb and @aws-sdk/lib-dynamodb. Whether sticking with SDK v2 might be better performance would be a good topic for another post. This function is over one MB, despite a small amount of custom code.

Running the test, the function has an average execution of 46.99ms and a max of 914ms. 99% of my requests are at or below 90ms.

Minified without Source Maps

Minifying the function drops the size to 534.8kb, less than half the size. My test showed an average response of 47.83ms, a max of 1010ms and 90ms at the 99th percentile. This is slightly worse, but not statistically significant. I expect if I ran these tests over and over again, I would see that minifying the code has no real effect on performance. This is not a surprise. 1.2MB is still fairly small and I don't expect to see much in the way of added latency at that size.

Minified with Source Maps

The minified function is still 534.8kb, of course, but the Source Map is 1.5MB so this will be the largest upload, not that ~2MB is a lot or will significantly slow down our deployments.

The average response time for this test is 46.52ms, max is 968mb and the 99th percentile is 82ms. This is the best result so far, but not statistically significant. I would say this is truly a three-way tie which tells us that adding Source Maps to this function did not increase latency.

That's because of the native support in Node.js! Since no stack traces were emitted, the Source Maps were never referenced. It's possible if we had to rely on a library for this capability, we wouldn't have the same outcome.

Error Minified without Source Maps

Will triggering error conditions make a difference? Let's see. I ran the same test but this time the function has that JSON.parse error in it. This happens before the call to DynamoDB, so we can expect it to be a little faster than the successful function. We get 37.86ms as the average, 1004ms as the max and 58ms at 99%. It's impressive the call to DynamoDB only seems to add about 10ms of latency!

Error Minified with Source Maps

Enabling Source Maps for the errors does impact performance. Our average has dropped to 97.54ms, max at 1129ms and 99% is 243. This is a significant increase. Source Maps do impact latency when an error occurs. This makes sense and confirms the idea that the Source Maps are only referenced when an error occurs - but now that Source Map must be parsed and that takes time.

Conclusion

Use Source Maps in production! In my view, if you are getting so many errors that the performance hit from Source Maps is impacting your bottom line, you should probably go and fix those errors. It's very easy to implement Source Maps in a variety of popular Lambda frameworks and they don't impact successful execution. When functions do fail, the useful stack trace is going to be worth the added latency. Developer hours are always going to be more expensive than a few milliseconds of Lambda execution.

COVER

Top comments (4)

Collapse
 
mxro profile image
Max Rohde •

Thanks for this! Any thoughts on externalizing source maps from the bundle? For instance by storing them on a web server and providing a source map reference as https:// link? That way we can deploy a much smaller lambda function. However, I guess that would expose all our server source code, unless there would be a way to put the source map somewhere only the Lambda can access it?

Collapse
 
elthrasher profile image
Matt Morgan •

Hi Max, thanks for reading! I think keeping the source map in Lambda storage is what you want to do. I doubt deploy speeds are going to be impacted too much unless you have really huge Lambda functions. The main reason I would NOT enable SourceMaps is if you are using try/catch for purposes other than rare error conditions (and ideally ones you intend to fix). Some devs use try/catch for flow control, such as:

try {
  // something that fails often
} catch {
  // perfectly acceptable alternate workflow
}
Enter fullscreen mode Exit fullscreen mode

If your code, or a library you consume does this, then SourceMaps might add significant latency to your app. Likewise if you have a high error rate, SourceMaps might compound your woes by making the customer experience even worse and making your app more expensive to operate.

Hope this helps!

Collapse
 
mxro profile image
Max Rohde •

Yes that sounds reasonable - it is a bit unfortunate that source maps easily double the deployment size for Lambdas - but as per your measurements the performance impact from that increased size seems to be quite negligible. I guess it should also be possible to have CloudWatch log out the stack trace without source mapping and then reverse-engineer that locally, although I didn't seem to find too much information on how to do that!

Thread Thread
 
elthrasher profile image
Matt Morgan •

I don't know of a way to reverse-engineer the stack trace into something readable using a SourceMap. It might not be possible because your logged stack trace is probably missing some details.

IMO the cost of a larger bundle isn't a very high price since it doesn't impact latency unless you throw an error. I suspect a lot of people will compromise by enabling sourcemaps early in projects or when there's a tricky bug to fix.

Read next

taishi profile image

Supabase Just Got More Powerful: Queue, Cron, and Background Tasks in Edge Functions

Taishi -

tanushree_aggarwal profile image

Dr. Swami Sivasubramanian's Keynote - under 5 minutes

Tanushree Aggarwal -

michael_flanagan_red profile image

Building the Backbone: Entities Part 2, Agent

Michael Flanagan -

dayjournal profile image

Create Map Images Using the Static Map Function of Amazon Location Service v2

Yasunori Kirimoto -