Hi Erik, that's really an issue we need to cover. Essentially, the last "get_warm" request should go out before the first one is terminated by the target Lambda.
What we can to do is benchmark the interval taken between "get_warm" requests, then multiply by the number of requests being sent concurrently. We can then pass a parameter to the target Lambdas, asking them to sleep for a few milliseconds to avoid container reuse.
What do you think about this solution?
That's more or less what my team does, although we don't do anything as sophisticated as automatically tuning the warmer with a feedback loop.
I think it would make more sense for AWS to let customers pay a fee to keep some idle capacity running, or to at least expose metrics on it. That would at least be more simple and direct.
Nice. Agreed about AWS as well. It could be something like Dynamo reserved capacity. I read from Jeremy Daly sometime ago that the Lambda team has no plans to release something like this, but they're looking into ways to tackle cold starts in the future.
We're a place where coders share, stay up-to-date and grow their careers.
We strive for transparency and don't collect excess data.