I write code, front-end and back-end, and like deploying it on AWS. Software Developer for 20 years, and still love it. Amateur Powerlifter & Parkourist.
Yeah, AWS Lambda can go up to 10 gigs of memory and 6 vCPU's but most of my code is I/O bound (meaning waiting on HTTP stuff), so I don't really need that much power. However, I'm not sure I'm smart enough to use 'em yet hah. Maybe someday I'll need to parse lots of stuff or something.
Ah, the workflow issue makes sense. It would be more of a hassle, indeed.
That's really weird that Google Cloud does that. AWS doesn't have that Lambda concurrency constraint issue.
It seems that AWS Lambdas work the same way:
When your function is invoked, Lambda allocates an instance of it to process the event. When the function code finishes running, it can handle another request. If the function is invoked again while a request is still being processed, another instance is allocated,
So, theoretically, you can't use the idle time, when the Node.js instance is waiting for I/O for one request, to process new incoming requests from the same Lambda function. Most Lambda function runtime would thus be idle time (waiting for I/O). Just like with Google Cloud Functions. Resulting in massive under-utilization of resources (Lambda runtime). But you want the Lambda to do what Node is good at: process massive amounts of requests without ever idling waiting for the I/O.
Note that this is different from AWS retaining an instance of a Lambda for later reuse (i.e. will AWS Lambda reuse function instaces?), because that happens after the original request has finished. What we want is to reuse an instance while it is being used by one request (simply switching between requests when one request is waiting for I/O).
If the above described is indeed the case, it looks like the dirty little secret of serverless deployments... AWS and Google don't really want you to reuse instances, because that reduces their total runtime which is how they bill you.
On how to circumvent it, see for instance: How We Massively Reduced Our AWS Lambda Bill With Go (Update: on second reading it seems like this is talking about doing multiple outgoing I/O requests from within the same Lambda function invocation, like you mentioned Promise.all could have handled in Node.js. But what I am talking about is reusing Lambda invocations to serve multiple new incoming requests while it would otherwise be waiting for the DB response for the first request). In Cloud Run (with containers), this reuse of idle I/O timeshould be possible with Node too, but without the immediate benefit of multi-core concurrency (unless resorting to a Node clustering setup).
I write code, front-end and back-end, and like deploying it on AWS. Software Developer for 20 years, and still love it. Amateur Powerlifter & Parkourist.
I think we're talking about different things or I'm just misunderstanding. You can, and do, make multiple HTTP outgoing REST calls from a single Lambda execution context. Even if you set your Lambda reserved concurrency to 1, ensuring you only ever get 1 Lambda execution context, and only 1 actual instance being used, you can do a Promise.all with 5 HTTP calls, and they all work at the same time. Now "by same time" I don't mean parallelism vs. concurrency, or Node.js's fake concurrency vs. Go, I just mean that you can see it take 30ms with multiple calls going out and responding vs. doing a regular Promise chain of 1 call after another. Now, yes, you can lower the Lambda's memory to like less than 512k which I think gives you a super small Raspberry PI vCPU and so your network calls and code in general goes much slower, but my point is you CAN do concurrency; I've seen it work.
What that article is talking about regarding Lambda is that "running Node.js code" doesn't act like Express.js where you have a single EC2/server it's hosted on having some ALB throwing 50 calls at it, and Express.js keeps accepting those and handling them in turn using Node's non-blocking I/O concept. So while you're writing code that feels synchronous, you may have 50 routes all executing "at the same time", but because you wrote it in an immutable way and are using Express.js, you don't really know this. You also don't even know that you're 1 of 3 in a Container on an ECS cluster. Whereas, with Lambda, you typically have 1 ALB or API Gateway URL firing up "1 Lambda per request". So 50 requests == 50 Lambda instances. AWS's Firecracker has a way to re-use those instances, though, so you may actually get 50 requests only spawning 30 Lambdas; 20 of those are a single Lambda Execution Context handling 2 calls each. Meaning your handler function is called twice, but the code up top that goes const fetch = require('fetch') is only run once... because you're in the same instance.
When you're building Serverless Lambdas, your Lambda is invoked by that event; it's not like Express.js where you "just have Express listening on some port and it just fires the wired up function for that route every time it gets an incoming HTTP request". So Lambdas don't care who invoked them, you just have a handler that runs, and you return a value, and that's it. There's no "listening for other events while your Lambda is running"; it's litterly just const handler = async event => console.log("got an event:", event) ... and that's it. So with 1 Execution Context per call, that'd be 50 of those functions running independently. If it were 30, and 20 of them shared the same Execution Context, JavaScript/Node doesn't care because it's just running the handler function twice, there's no shared global variables/state between them, and that's fine.
AWS has done an amazing job of making SQS, API Gateway, ALB, SNS, etc. all "handle their own calling and retry and some batching", so your code doesn't care about any of that as opposed to the stupid process.on('SIGTERM', process.exit) crap you see in some Express/Restify code bases.
So again, that article is talking about "Your Lambda runs and stops, you can't have it sit there waiting for new API Gateway, or S3 or whatever requests triggered it". That's a fundamental of how AWS Lambda works. I have zero idea if Google/Azure works the same way.
We're on the same page with the I/O, though, again, most of my Lambdas run in 10 seconds or less; if they need more time, too bad; we just make 'em timeout. That said, I have this one AppSync Lambda function that makes 14 concurrent requests that's super close to 9.8 seconds, but I blame the Go developers I'm calling and their lack of caching vs. being constructive and helping them on said cache 😜.
Yes, ok, read your other paragraph, yeah, we're on the same page there too on re-used instances.
Yeah, you can do with Node.js what they're doing with Go. Slower, sure, but like SQS or AppSync or Dynamo; all 3 support batching. Rather than "1 SQS message is parsed by 1 Lambda", instead, you can say "no no, 200 messages are handled by this Lambda", and you either map each to a Promise to do stuff like copy data to S3, OR do Promise.all if it can be concurrently. And it works, we've done it. Now is that Lambda somehow flattening similiar to the browser those "200 HTTP requests into 1 TCP/IP call stack request, then unpacking on the receiving end to 200 requests to optimize network traffic"? Maybe, I have no idea, but I know Promise.all for our 200 DynamoDB messages in Node using Promise.all takes about 3 seconds to write back to Dynamo concurrently, but if you do one at a time, it's 20 seconds. So the end result is even if it's fake concurrency, it's working like we expect.
I write code, front-end and back-end, and like deploying it on AWS. Software Developer for 20 years, and still love it. Amateur Powerlifter & Parkourist.
Yeah, AWS Lambda can go up to 10 gigs of memory and 6 vCPU's but most of my code is I/O bound (meaning waiting on HTTP stuff), so I don't really need that much power. However, I'm not sure I'm smart enough to use 'em yet hah. Maybe someday I'll need to parse lots of stuff or something.
Ah, the workflow issue makes sense. It would be more of a hassle, indeed.
It seems that AWS Lambdas work the same way:
docs.aws.amazon.com/lambda/latest/...
So, theoretically, you can't use the idle time, when the Node.js instance is waiting for I/O for one request, to process new incoming requests from the same Lambda function. Most Lambda function runtime would thus be idle time (waiting for I/O). Just like with Google Cloud Functions. Resulting in massive under-utilization of resources (Lambda runtime). But you want the Lambda to do what Node is good at: process massive amounts of requests without ever idling waiting for the I/O.
Note that this is different from AWS retaining an instance of a Lambda for later reuse (i.e. will AWS Lambda reuse function instaces?), because that happens after the original request has finished. What we want is to reuse an instance while it is being used by one request (simply switching between requests when one request is waiting for I/O).
If the above described is indeed the case, it looks like the dirty little secret of serverless deployments... AWS and Google don't really want you to reuse instances, because that reduces their total runtime which is how they bill you.
On how to circumvent it, see for instance: How We Massively Reduced Our AWS Lambda Bill With Go (Update: on second reading it seems like this is talking about doing multiple outgoing I/O requests from within the same Lambda function invocation, like you mentioned Promise.all could have handled in Node.js. But what I am talking about is reusing Lambda invocations to serve multiple new incoming requests while it would otherwise be waiting for the DB response for the first request). In Cloud Run (with containers), this reuse of idle I/O timeshould be possible with Node too, but without the immediate benefit of multi-core concurrency (unless resorting to a Node clustering setup).
I think we're talking about different things or I'm just misunderstanding. You can, and do, make multiple HTTP outgoing REST calls from a single Lambda execution context. Even if you set your Lambda reserved concurrency to 1, ensuring you only ever get 1 Lambda execution context, and only 1 actual instance being used, you can do a
Promise.allwith 5 HTTP calls, and they all work at the same time. Now "by same time" I don't mean parallelism vs. concurrency, or Node.js's fake concurrency vs. Go, I just mean that you can see it take 30ms with multiple calls going out and responding vs. doing a regular Promise chain of 1 call after another. Now, yes, you can lower the Lambda's memory to like less than 512k which I think gives you a super small Raspberry PI vCPU and so your network calls and code in general goes much slower, but my point is you CAN do concurrency; I've seen it work.What that article is talking about regarding Lambda is that "running Node.js code" doesn't act like Express.js where you have a single EC2/server it's hosted on having some ALB throwing 50 calls at it, and Express.js keeps accepting those and handling them in turn using Node's non-blocking I/O concept. So while you're writing code that feels synchronous, you may have 50 routes all executing "at the same time", but because you wrote it in an immutable way and are using Express.js, you don't really know this. You also don't even know that you're 1 of 3 in a Container on an ECS cluster. Whereas, with Lambda, you typically have 1 ALB or API Gateway URL firing up "1 Lambda per request". So 50 requests == 50 Lambda instances. AWS's Firecracker has a way to re-use those instances, though, so you may actually get 50 requests only spawning 30 Lambdas; 20 of those are a single Lambda Execution Context handling 2 calls each. Meaning your
handlerfunction is called twice, but the code up top that goesconst fetch = require('fetch')is only run once... because you're in the same instance.When you're building Serverless Lambdas, your Lambda is invoked by that event; it's not like Express.js where you "just have Express listening on some port and it just fires the wired up function for that route every time it gets an incoming HTTP request". So Lambdas don't care who invoked them, you just have a handler that runs, and you return a value, and that's it. There's no "listening for other events while your Lambda is running"; it's litterly just
const handler = async event => console.log("got an event:", event)... and that's it. So with 1 Execution Context per call, that'd be 50 of those functions running independently. If it were 30, and 20 of them shared the same Execution Context, JavaScript/Node doesn't care because it's just running the handler function twice, there's no shared global variables/state between them, and that's fine.AWS has done an amazing job of making SQS, API Gateway, ALB, SNS, etc. all "handle their own calling and retry and some batching", so your code doesn't care about any of that as opposed to the stupid
process.on('SIGTERM', process.exit)crap you see in some Express/Restify code bases.So again, that article is talking about "Your Lambda runs and stops, you can't have it sit there waiting for new API Gateway, or S3 or whatever requests triggered it". That's a fundamental of how AWS Lambda works. I have zero idea if Google/Azure works the same way.
We're on the same page with the I/O, though, again, most of my Lambdas run in 10 seconds or less; if they need more time, too bad; we just make 'em timeout. That said, I have this one AppSync Lambda function that makes 14 concurrent requests that's super close to 9.8 seconds, but I blame the Go developers I'm calling and their lack of caching vs. being constructive and helping them on said cache 😜.
Yes, ok, read your other paragraph, yeah, we're on the same page there too on re-used instances.
Yeah, you can do with Node.js what they're doing with Go. Slower, sure, but like SQS or AppSync or Dynamo; all 3 support batching. Rather than "1 SQS message is parsed by 1 Lambda", instead, you can say "no no, 200 messages are handled by this Lambda", and you either map each to a Promise to do stuff like copy data to S3, OR do Promise.all if it can be concurrently. And it works, we've done it. Now is that Lambda somehow flattening similiar to the browser those "200 HTTP requests into 1 TCP/IP call stack request, then unpacking on the receiving end to 200 requests to optimize network traffic"? Maybe, I have no idea, but I know Promise.all for our 200 DynamoDB messages in Node using Promise.all takes about 3 seconds to write back to Dynamo concurrently, but if you do one at a time, it's 20 seconds. So the end result is even if it's fake concurrency, it's working like we expect.
... ok I should play with Google and Azure more... so many toys...