Great article! Whenever I try to use a "build" or "dist" folder I get the following errors:
Build Failed
Error: NodejsNpmBuilder:NpmPack - NPM Failed: npm ERR! code ENOLOCAL
npm ERR! Could not install from "/path/to/dist/folder" as it does not contain a package.json file.
I would love to learn about how you got around that!
Hey Nick, thanks for reading! If you're getting that error, I assume you're doing a sam build? Using the technique in this article, you won't use sam build and the reasons for that are outlined in the article. Instead you npm run build and then sam local start-api or sam deploy. If that's what you're doing and you're still getting the error above, let me know and I'll try to figure it out. It's been a bit since I've run this.
I haven't actually done that myself with webpack - since the point of webpack is to bundle and tree-shake all the dependencies - but if you are solid on creating a Lambda layer with your modules, you'd just need to set whatever dependencies are in the layer as an external. See webpack.js.org/configuration/exter...
A lot of people do this with aws-sdk since it's already available in Lambda, but I've seen benchmarks show that you can actually get slightly faster cold starts if you bundle it so that's why I didn't do that in this example (though I have done it and TBH haven't noticed any difference either way).
If already using webpack with SAM, I'd probably only worry about Lambda layers if A) I had some huge dependency or B) I had some layer I wanted to share across many projects.
Do you have an example of what that structure looks like? Right now, I package everything separately with Webpack and then build the application with sam build. Here's the basic repo structure I use: github.com/Borduhh/serverless-SAM-...
I'm curious to know if this might be a more efficient way to do it though.
The main thing I'm going for is a single package.json at the root of my repo, not multiple throughout. I introduced lerna into a UI project last year and my team hated having to manage dependencies (i.e. update them!) at multiple layers throughout the project. We ended up pulling that out for a single webpack build that could produce multiple bundles from the entrypoints. It's much cleaner and builds much faster!
So my article is about applying those same principles to building multiple Lambda functions from the same code base without multiple package.json files.
Like I said, I haven't worked with Lambda layers because I just webpack my entire project. What's the use case for using layers in your app? Are you just trying to make the bundle smaller? Share something across several code repos?
Yeah, that makes a lot of sense. I am using layers for precisely that reason, to avoid having to go into each package and update shared dependencies (i.e. a lot of our functions will use the aws-sdk package so I have an AwsSdkLayer which loads that subset of tools). That way I can pop into the layer, and update the dependency once and I am done.
It sounds like this would be similar though in a more efficient manner. I am just am having trouble wrapping my head around what a larger project structure would look like with multiple functions that share the same code.
The project structure ends up looking a lot like your typical Express or NestJS app. That's the great thing about this approach. So we might do something like
/aws
/handlers
/repositories
/utils
The handlers are like controllers or routes in your average NodeJS app and everything else is shared or at least potentially shared. If you are somewhat careful in your imports, you should be able tree-shake away the things you don't need when you build. I did have a developer point out a little leakage with some extra unnecessary code showing up in a function due to the way things were being imported, but it hardly added any size and our functions are around 5MB, well below the 50MB limit.
I don't really have any other insights around lambda layers except I haven't felt the need to use them except for adding binaries to a lambda runtime.
We also follow the practice of putting a unit test right next to each of the code modules. The tree-shaking approach helps there too since obviously none of the production code has imports from the tests.
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
Great article! Whenever I try to use a "build" or "dist" folder I get the following errors:
I would love to learn about how you got around that!
Hey Nick, thanks for reading! If you're getting that error, I assume you're doing a
sam build? Using the technique in this article, you won't usesam buildand the reasons for that are outlined in the article. Instead younpm run buildand thensam local start-apiorsam deploy. If that's what you're doing and you're still getting the error above, let me know and I'll try to figure it out. It's been a bit since I've run this.How would this work if I were to try to add dependency layers?
I haven't actually done that myself with webpack - since the point of webpack is to bundle and tree-shake all the dependencies - but if you are solid on creating a Lambda layer with your modules, you'd just need to set whatever dependencies are in the layer as an external. See webpack.js.org/configuration/exter...
A lot of people do this with aws-sdk since it's already available in Lambda, but I've seen benchmarks show that you can actually get slightly faster cold starts if you bundle it so that's why I didn't do that in this example (though I have done it and TBH haven't noticed any difference either way).
If already using webpack with SAM, I'd probably only worry about Lambda layers if A) I had some huge dependency or B) I had some layer I wanted to share across many projects.
Do you have an example of what that structure looks like? Right now, I package everything separately with Webpack and then build the application with
sam build. Here's the basic repo structure I use: github.com/Borduhh/serverless-SAM-...I'm curious to know if this might be a more efficient way to do it though.
The main thing I'm going for is a single package.json at the root of my repo, not multiple throughout. I introduced lerna into a UI project last year and my team hated having to manage dependencies (i.e. update them!) at multiple layers throughout the project. We ended up pulling that out for a single webpack build that could produce multiple bundles from the entrypoints. It's much cleaner and builds much faster!
So my article is about applying those same principles to building multiple Lambda functions from the same code base without multiple package.json files.
Like I said, I haven't worked with Lambda layers because I just webpack my entire project. What's the use case for using layers in your app? Are you just trying to make the bundle smaller? Share something across several code repos?
Yeah, that makes a lot of sense. I am using layers for precisely that reason, to avoid having to go into each package and update shared dependencies (i.e. a lot of our functions will use the
aws-sdkpackage so I have anAwsSdkLayerwhich loads that subset of tools). That way I can pop into the layer, and update the dependency once and I am done.It sounds like this would be similar though in a more efficient manner. I am just am having trouble wrapping my head around what a larger project structure would look like with multiple functions that share the same code.
The project structure ends up looking a lot like your typical Express or NestJS app. That's the great thing about this approach. So we might do something like
/aws
/handlers
/repositories
/utils
The handlers are like controllers or routes in your average NodeJS app and everything else is shared or at least potentially shared. If you are somewhat careful in your imports, you should be able tree-shake away the things you don't need when you build. I did have a developer point out a little leakage with some extra unnecessary code showing up in a function due to the way things were being imported, but it hardly added any size and our functions are around 5MB, well below the 50MB limit.
I don't really have any other insights around lambda layers except I haven't felt the need to use them except for adding binaries to a lambda runtime.
We also follow the practice of putting a unit test right next to each of the code modules. The tree-shaking approach helps there too since obviously none of the production code has imports from the tests.