When you start with serverless you will very soon learn/hear about functions cold start (I believe serverless = cloud functions + APIs). A cold start happens when a cloud function is invoked for the first time or after a long time of no invocations. Basically, it takes the server (yes – there are servers!) a little bit longer to get the function ready the first time, so it’s ready to accept and process the request. If a function is invoked a second time, it will execute faster. There is a time period during which a function stays warm. If a function is invoked again during that time period – it will be executed fast.
If a function is not invoked within some period, it becomes cold again and the next time it’s invoked, it will be a little bit slower again (cold start).
This makes sense. When you launch an app on your phone or computer for the first time – it takes a little bit longer the first time. When you launch it again very soon, it usually starts faster.
For many applications cold starts are not a problem. It’s very important we consider the type of application we are building. If we are building a business application or an internal backend application – then cold starts are not a problem. It’s not going to make a difference if an application starts a fraction of a second slower or responds to a request a fraction of a second slower. Type of an application is important when talking about cold starts. It’s only a problem for some applications and probably in those cases serverless is not the best fit.
I believe most applications won’t be impacted by a cold start. Even for most consumer applications a cold start shouldn’t be a problem. Same goes for most mobile applications. Again, it’s unlikely most mobile applications will be impacted by a fraction of second slower backend (function) execution.
Now, for example, if you are building an application that sends a rocket into space where every fraction of a second might count – then yes, a cold start is a problem and serverless is probably not the best fit. Medical applications might also fall into this category. I’m sure there are other examples but hopefully you get the point.
Modern applications based on serverless approach are built with cloud functions and various managed services and APIs. For example, if you need to send an SMS, you would use Twilio or Nexmo. If you need to add payments, you would use Stripe or PayPal. If you need mapping/location service, you would use Google Maps API or HERE. There are thousands of other APIs that you can use to build your application. A modern application based on serverless architecture will consist of cloud functions + APIs.
External APIs you use in an application need to be of high quality and from reputable vendors. But, it’s not often that you worry if the service behind the API has a cold or warm start. The reason is again, it’s not going to make a difference if an API executes a fraction of a second slower. The same way we usually don’t worry about external APIs, there is little reason to worry about this in the context of cloud functions.
Lastly, it’s very likely that cloud functions vendors will make a cold start a non-issue at all as they improve/upgrade their services.
To summarize, a cold start in a serverless-based application is not a problem for most applications. For applications where every fraction of a second counts, serverless approach is probably not the best fit.
Top comments (6)
Cold starts are not necessarily "fractions of a second"
.Net on a VPC is roughly 15 seconds.
Node on a VPC is still up to 10 seconds.
Pretty much a problem across the board with AWS Lambda behind VPC.
VPC adds about 5-10 seconds as the virtual network adaptor gets provisioned
Agree, 5-10 seconds delay is not good (although probably OK for background processing that runs periodically). Without VPC, Node is much faster.
Cant apply for Slack App(Bot) where every responses need to be in 3s.
What are your thoughts on Cloudflare workers?
Hi Ben - I think the general concepts I cover in this article will apply to any serverless environment, including Cloudfare Workers. I read a little bit about Cloudfare Workers and they say that Cloudfare Workers is faster than a serverless platform like Amazon Lambda because they use Google V8 JavaScript engine instead of NodeJS. Here is the article: blog.cloudflare.com/serverless-per...