DEV Community

Cover image for Cloud Run vs App Engine: a head-to-head comparison using facts and science

Cloud Run vs App Engine: a head-to-head comparison using facts and science

Paul Craig on November 12, 2020

For low-traffic applications, Cloud Run is dramatically cheaper than App Engine. Abstract I was hosting a small web app as a side-proje...
Collapse
 
ashishlogmaster profile image
Ashish “Logmaster” Boston

Your app-engine was setup to "autoscale" hence the instance would stay up constantly costing you $. If you changed it to "basic" auto-scaling, GAE would have auto scale down and stop the instance and costs should be similar to cloud run. Could you pls re-test with this setting so its a more fair comparison. Thanks,
cloud.google.com/appengine/docs/st...

Collapse
 
msl00 profile image
msl00 • Edited

The linked documentation could be more clear, but it is not correct to say that "autoscaled" instances are "up constantly costing you $". You linked to the "Instance State" section, and it is saying that "autoscaled" instances will only ever show as being in the "running" state (vs the "stopped" state possible for "manual" or "basic" scaling). This is because "autoscaled" instances are shut down after some time (if no requests come in), not that the instances are running 24/7.

Collapse
 
confiq profile image
Igor Konforti

I have to agree with @msl00 here! It's unclear and AFAIK AppEngine can NOT auto-scale to 0!

Thread Thread
 
vajahath7 profile image
Vajahath

App engine standard environment can scale down to zero. I'm paying zero for for my hobby project. cloud.google.com/appengine/docs/th...

Collapse
 
vugar005 profile image
Vugar

The problem with autoscaling to 0 is that it causes cold start. Assume your app is idle state(usually after 20 minutes when no new requests coming) so the total number of instance would be 0. So it take about 10 seconds to start the server in AppEngine for a first request.

Collapse
 
tenthyoung profile image
Izzy Young

I laughed out a loud a couple of times reading this article. You have a great sense of humor and a fantastic writing style :)

Collapse
 
mdovn profile image
mdovn

"sometimes tripling or quadrupling depending on Who The F*ck Knows."
=))

Collapse
 
bharatsawnani profile image
bharatsawnani

Nice tests and post, but you should specify which environment you're using for GAE.

I primarily code in Java (Haven't deployed with Node.js on GAE so far) and from my experience the Standard environment works similarly to Cloud Run, as it spins up a new instance when a request is made (if there wasn't one already idle). The instance stays idle for 15 mins after that it's shutted down. Google gives you a daily free usage quota of 28 hours for instances. Hence if you tests were runnning once every 47 mins (and the requests didn't require much processing power)... then your daily cost would be 0.00$ as you wouldn't be surpassing the daily free quota.

If your tests were on the Flexible Environment then that's a whole different story as an instance has to be idle all the time and I am not so sure what machine type they start of from there. In Standard the lowest instance is an F1, whicho would have 256mb RAM, which is not much but enough for a simple app.

Collapse
 
samuelbfavarin profile image
Samuel Favarin

Nice post! In the future would be cool to do a benchmark to compare with a similar AWS service.

Collapse
 
pcraig3 profile image
Paul Craig

It's a good question. At work we use Fargate a lot, which I find a lot more complex than Cloud Run to set up, but it has a similar "serverless container" platform concept as CR does.

Collapse
 
lm2a profile image
Mario La Menza Perello • Edited

I was unsuccessful trying to find out something about your app's datasource. Because IMO there is the big cost, when using Cloud Run. I agree with you, Cloud Run is cheap, but you have to use Cloud SQL as a datasource and in my experience it is far expensive compared with a SQL instance running in GCE.

Collapse
 
maximtan profile image
Maxim Tan • Edited

Just to chime in for anyone confused by the huge price difference.

In my experience, App Engine Standard Environment with automatic scaling will effectively scale down to "0 instances": After 15 minutes with no request, with automatic scaling, you are billed NOTHING.

(dev-to-uploads.s3.amazonaws.com/up...)

Google states on their pricing page:

Accrual of instance hours begins when an instance starts and ends as described below, depending on the type of scaling you specify for the instance:

  • Basic or automatic scaling: accrual ends fifteen minutes after an instance finishes processing its last request.
  • Manual scaling: accrual ends fifteen minutes after an instance shuts down.
Collapse
 
lostinthefield profile image
lostinthefield

Paul, thanks for this hilarious and informative comparison!

I was wondering if you had also considered deploying this site as a static site on something like Firebase Hosting/Vercel/Netlify/Github Pages, etc. (straight to a CDN, instead of worrying about hosting)? Next.js takes cares of a lot of pain points (data fetching, caching etc.). We're considering something like that for our own site.

You do lose the benefit of having a proper node.js backend, but so far our needs can be met by Next.js mixed with maybe some serverless functions.

Overall, that could maybe bring costs down even further than an auto-scaling container, as long as you don't need access to a real server...?

Just food for thought.

Collapse
 
spock123 profile image
Lars Rye Jeppesen

NextJs is great but locks you in to using old hat React, not everybody's cup of tea, mind you.

Collapse
 
mikeneilens profile image
Mike Neilens

I’ve been running a couple of applications on App Engine for several years and never been billed more than $0.50 per month. I think you may have set up App Engine incorrectly for the workload you are using.

Collapse
 
dtobias profile image
Dom • Edited

Misleading post as the premise is that you pay for App Engine 24/7 which isn't true on the standard instances (predefined languages versions e.g. Go 1.12) only if you choose flex (custom versions). Otherwise you have a point but it's not clear and standard App Engine covers most use cases.

Collapse
 
jenil777007 profile image
Jenil Calcuttawala

Great post! App Engine standard or flex?

Collapse
 
deama profile image
dmytro lysak

What about the 1 million requests are free per month on cloud run?

Collapse
 
lguinn profile image
Lisa Guinn

This is the monthly free tier on Cloud Run:

  • 180,000 vCPU-seconds
  • 360,000 GiB-seconds (memory)
  • 2 million requests
  • 1 GiB free data egress within North America So the app must have exceeded one of these parameters to incur a monthly charge cf. cloud.google.com/run/pricing
Collapse
 
tranquan profile image
Tran Quan

I am building the same kind of hobby app, and Google lead me here ヽ(^O^)ノ
Thank you very much.

I think I will also use Cloud Run, it's flexible and we can move the whole docker to another cloud later

Collapse
 
spock123 profile image
Lars Rye Jeppesen

I've built a startup company using Google Cloud - not sure why you would move to somewhere else that has matching products. Once you have Cloud Run containers, you'll want to use pub/sub for event driven architecture. Combined with realtime Firebase Firestore database you're pretty much in a situation that is not matched anywhere else. Especially for the price.

Collapse
 
pcraig3 profile image
Paul Craig

Yeah, it's a great place to start! Heroku is easier to use but it costs like 100x more money.

Collapse
 
_gdelgado profile image
Gio

Really appreciate the commentary on costs 👏👏

Collapse
 
bhagyash007 profile image
Bhagyash

It's a naive article. Highlighting saving 10$. Would appreciate if you could do more research and provide insightful facts about use cases, trade offs and so on.

Collapse
 
ekinsey profile image
ekinsey

What service did you use for the database? Cloud SQL?

Collapse
 
joenas profile image
Jon Neverland

I opted for the immersion method where I would assume the role of a developer trying to host an app on Google Cloud.

You don't shy away from anything do you? 😂

Collapse
 
artu_hnrq profile image
Arthur Henrique

Nice share

Collapse
 
bobywno profile image
Bob

Technicality aside, this post is beautiful. I was feeling unwell and then when I read your Canadian jokes, it makes me giggle a lot. Thanks for the post Sir!

Collapse
 
emwadde profile image
emwadde

What about Google Cloud Functions?