Trialing AWS Lambda performance? Are you being fair?

Robin Smith — Fri, 01 Mar 2019 13:39:18 +0000

I was helping out a colleague this week who had created a simple serverless style setup using AWS: API Gateway -> Lambda -> DynamoDB

There was nothing unusual about the setup, and the function worked as expected first time. Their problem came when they decided to benchmark performance against the traditional 24/7 deployed server that the function was designed to replace. Their results left them massively disappointed, verging on concerned.

Why?

The easiest thing to do in this situation is to think about the current version first. Your existing service is effectively fully scaled out at all times; it sits quietly in your data centre burning cash waiting for requests, and whether it receives a single request or 50 requests per second (RPS) it's ready to respond with the full provisioned capacity from the first millisecond.

But this isn't how lambda works: If there are no requests then there is no capacity.

When that single request arrives, the architecture rapidly provisions resource to deal with the request and also leaves that resource available for a short time in case additional requests come through. This means that if you suddenly load your function with 50RPS from it being idle, then you will be forced to wait for
a short time while the architecture reacts, which it does by spinning up a fleet of functions to process the sudden influx of requests.

Lambda doesn't scale linearly to requests

Receiving 50 requests at once doesn't mean you will have 50 concurrent executions. There are complex calculations performed behind-the-scenes, trading off things like:

time it takes to start a new function
the expected amount of remaining time on the current execution
historical data about how the function is being utilised
individual account limits

All of these things play a factor in how your function is auto-scaled, and as a result in some circumstances Lambda may choose to simply queue some of your invocations up behind each other, instead of opting for concurrent executions.

What this means is that if you decide to perform an unrealistic load test then you are very likely to see much poorer scalability and performance than you would expect to see under real production load.

The point here is that Lambda can easily handle load, but you have to understand how it works in order to evaluate it fairly. If your function is expected to handle 50RPS constantly as a baseline then it's unrealistic to benchmark Lambda from 0 to 50RPS within a 60 second window: you need to give it time to scale up if you want a realistic view of how it will perform at that level.

What you'll find is that if you run your function at 50RPS for a longer period, the architecture will work out exactly how much resource is required in order to best service that load, and your function will perform fantastically without any need to scale up further. If you do suddenly get a huge spike and have 1000RPS lambda will still deal with that, but it won't be at the same performance level of your 50rps baseline immediately; it needs to react to the additional load with extra resource which will impact the performance at the beginning of the spike.

Lambda is fantastic, but it's not magic: it takes time to deal with unexpected traffic spikes.

If you want to understand how your service will behave within Lambda under normal load then you need to run that test over a sensible timeframe. This will allow Lambda to understand that this is your normal load, and this will enable you to see the reliable performance you expect.

Of course, running tests like this over a short period also allows you to see the worst-case scenario for how your function will behave should it get bombarded with unexpected spikes.

Both are important metrics but they shouldn't be confused as equal. Don't write Lambda off because of your unrealistic expectations; if you do you'll be missing out.

Add value, not complexity: Change the way you think about "The Cloud"

Robin Smith — Tue, 26 Feb 2019 15:35:37 +0000

I dislike using the phrase “the cloud” and I’m not alone but here we are; it’s become the standard term for anything at all related to “things not on your hardware”. Be that personal storage from your phone to the deployment of multi-region redundant database clusters, they all get lumped under the same umbrella question of “why not move it to the cloud?”

Then there is the over-trivialisation of “the cloud”. One of the biggest challenges for a tech department when trying to persuade their business to migrate is that it’s hard to describe the benefits. Teams spend ages planning a pitch to outline the benefits, only to be shot down with a single response: “but isn’t it just the same computers in someone else’s data centre?”

How it normally gets sold

So the tech team have been talking, and they really want to move the system to the cloud. They have gone away to work out what will be involved and what it will cost in order to convince the decision-maker. A couple of weeks later (if all goes well) they come back with the numbers and a migration plan. The conversation normally goes something like this:

Decision-Maker: “So how much will this cost?”
Team: “It is going to cost about $X thousand to provision the architecture to replicate what we have now”
Decision-Maker: “Ok but what about all this expensive stuff that you convinced me to buy last year that would make our system future proof for years to come?”
Team: “We don’t need that anymore: it will be on the cloud!”
Decision-Maker: “..................”
Team: “......erm we could probably sell it to recoup something but it prob won’t be a lot, but moving really is the right thing to do”
Decision-Maker: “Ok let’s ignore the cost for a second. How long will this take to implement?”
Team: “Well it will take about 6 months to build up the architecture to replicate the current system. Once that is done we will turn off the current system migrate all of our data and then start up the systems again”
Decision-Maker: “turn off……”
Team: “Well there will have to be some point in which we switch from one system to another. That could cause some downtime but it will be kept to a minimum.”
Decision-Maker: “ ….. Ok, but then we will be better off and more resilient than we are now?”
Team: “Yes ….. well actually this just replicates the current system we have now, but with a dedicated resource to keep the servers running etc. if there is a fault”
Decision-Maker: “Don’t we have a team for that already?”
Team: “We don’t need that anymore: it will be on the cloud!”
Decision-Maker: “..... ok well thanks for your time I’ll get back to you.”

I’ve been a bit harsh there - and people will point out that there are a multitude of ways to have seamless migrations that don’t involve a downtime etc. - but the reality is that this is often how these conversations go.

Decision-makers often don’t understand the benefits, and those trying to influence don't know how to quantify them.

When starting out anew, it makes perfect sense to start out on the cloud. Quite simply the cloud providers - both public and private - have one advantage over us: Purchasing power. They are buying enough hardware that they are able to offer us a tiny slice of it for a fraction of what we could purchase it for directly. We will be able to buy exactly what we need for the exact amount of time we need it - literally by the minute - versus years of investment in our own hardware. In fact, we will most likely to be able to provision hardware much higher specification that our budget would normally allow.

More established businesses, however, tend to have thousands of pounds worth of equipment that has built up over the years. It’s maintained by their team, and represents a massive investment, why would they trash it all just to move to the cloud?

Because what they have isn’t good enough!

Nobody has enough capacity for the unexpected, but we all want our business to be the next Slack or JustEat. We constantly need to consider how to increase capacity so that when we win that big client or become the next overnight success we aren’t left waiting for delivery of new kit. With growth comes the need for resilience - it’s no good having provisioned tons of hardware if it’s all sat next to each other in a single location waiting for the power to come back on - so we deploy across multiple sites and invest in high availability failover equipment. This is just how it has always worked; we take a gamble based on the best data we have and hope to have what we need when we need it. But this isn’t how it has to work anymore.

When just starting out we can deploy on a small scale for very little cost and really trial what you need without any long-term commitment. As we grow, being on the cloud makes sense as we can quickly respond to that growth by provisioning or de-provisioning hardware within minutes, not weeks. When we’re huge, then being on the cloud makes sense because you are able to deploy those services resiliently across multiple locations and even across multiple countries.

We could do all the above without “the cloud” but really what this comes down to is: Why would you want to?

What’s important to your business? Is it protecting your hardware investment and traditional processes, or is it giving the best quality service to your users? Your focus shouldn’t be on recruiting staff and expertise to manage infrastructure; it should be on recruiting staff and expertise to solve customers’ problems and advance the product. The reality is that cloud providers are better at managing these things than you are! So why on Earth would you want to waste time and effort trying to replicate something that someone else is doing better and cheaper? Accept your constraints and concentrate your efforts on what makes you great.

If you get to this point and you’re starting to come around to the idea of using the cloud then great! But to be honest you are probably still thinking about it in the wrong way. Whilst all the benefits I’ve alluded to so far should be clear, this is only scratching the surface of what can be achieved after you move to the cloud.

It’s so much more than just “someone else’s computer”!

All of the above is focusing on the very obvious benefits: cost, scale, and maintenance. But you may have noticed I’ve gone to great length to use the phrase “on the cloud” and not “in the cloud” and that’s because there is a very big difference between those two ideas.

Once you get past the initial steps then a whole new world of possibilities will open up. With engineers totally focused on solving actual business problems - and not infrastructure problems - you can utilise a huge array of services beyond virtual servers.

The difference between the concepts is subtle, but here are a couple of examples:

On the cloud - You provision a virtual instance and deploy your application. It services requests over the internet and as you need to scale you provision more identical instances in order to spread the load and improve performance etc.
In the cloud - You extract just your application logic and deploy it to a Functions as a Service (FaaS) provider within your cloud provider account. Scaling, execution, and resilience of the application are managed by your cloud provider and you only pay for the requests your customers actually make.
On the cloud - Your application is well-architected, and messages between different parts of your system are handled by passing events to keep the system decoupled. You deploy more virtual instances to act as brokers and queues to support this decoupled system.
In the cloud - In order to manage your decoupled system you simply make use of the cloud provider’s messaging and queue systems. Able to cope with millions of messages, your events are now handled by a native solution designed to scale with your use across multiple availability zones.

The end result is the same, but the solution is the key. This comes back to the same point when discussing cost and scalability: Anyone can migrate their systems to the cloud.

Anyone can gain the cost benefits: but that’s not the real value. The real value isn’t the cost; it isn’t even the resilience. It’s about what can be done once you get there. There are new services being offered up to cloud providers on an almost daily basis; from new databases, to machine learning, to satellite comms, all designed to make your life easier.

Stop focusing on the cost associated with it; just accept it needs to happen. Stop trying to compare a cloud provider with what you can do yourself; just accept they can do it better. Stop thinking about your service in its current form; just accept that it needs to evolve to meet modern demands.

Concentrate on making the best product you can, and utilise everything you can that makes that easier. Nobody has ever used a product because they like the way the database has been configured or the particular data centre it’s hosted in.

Add value: not complexity.

Robin Smith
Chief Product Engineer
Click Travel

DEV Community: Robin Smith