DEV Community: Andreas Casen

Vercel’s Fluid Compute and what it means for AWS Lambda

Andreas Casen — Wed, 19 Feb 2025 08:00:00 +0000

Vercel recently announced Fluid Compute, presenting it as “The power of servers, in serverless form”. I won’t be digging into the details of what goes on behind the scenes with this new feature, there’s a lot of more capable and knowledgeable folks out there who have great pieces of content explaining what it is exactly and how it works (Theo Browne’s video is a great place to start).

Servers vs Serverless

Serverless was put in place to deal with some of the challenges and shortcomings of traditional servers. Unless you have a very predictable traffic pattern from your users, figuring out how much compute power your backend needs at any given time is far from an exact science.

For example, let’s say a single server has the capacity to handle 1000 users at most. You’re under this threshold 80% of the time, and you can spin up a second server to handle that heavier 20%. The problem is, you can’t exactly predict when these spikes happen, and spinning up a server is slow. You’re now stuck with a dilemma:

Spin up (and pay for) the extra server just in case so that you’re ready for when that spike comes, or start spinning up the second server when you detect the traffic increase and risk a window of time where some users might run into an unavailable service.

Serverless functions take this dilemma off your hands by spinning up a function for each request that comes in, which is what allows it to scale so effectively. 1 request, 1 function. 1 million requests the next minute, 1 million functions spun up.

This works great in terms of scaling, but also means each function is an isolated, ephemeral server that needs to remain stateless, and in-function concurrency doesn’t exist.

The problem with idle compute time

If user 1 makes a request that spins up Lambda A, a request from user 2 will spin up a Lambda B. This is all fine and dandy, but Lambda A had just made a 3rd party API call, and was awaiting a response when the request from user 2 came in. Lambda B is now in the same state, and we’re stuck paying for compute time across two lambdas when in reality, they’re both just waiting.

At small scales (meaning when that waiting time isn’t huge) the money you spend on idle compute might be negligible. As an industry, we’ve been optimising for this model for a long time, with Serverless compute like Lambda and Vercel’s functions bringing a ton of value in terms of DX, performance and enabling some pretty neat architecture patterns.

As idle time increases however, things start becoming a problem. Serverless functions are billed for duration, meaning you’re paying for both the active and idle parts of your function execution.

Take a use cases where your downstream service is a slow responder, either by nature (LLMs anyone?) or because of performance issues, and you’re suddenly spending a ton of money on compute resources that are sitting around, doing nothing.

In-function concurrency

Vercel came up with a way to enable their serverless functions (aka the Vercel Lambda) to handle concurrent requests by taking advantage of idle time during IO operations.

In-function concurrency is at the core of this update. Again, there are smarter folks out there explaining this in detail but in short, it allows a single function to handle multiple requests by taking advantage of that idle time. You make a request to your function, which in turn calls that 3rd party API and waits for a response. The window of time that used to be spent idle (ie waiting) is now effectively looked at as an available function to which another request can be sent.

The 2 obvious benefits here are simple:

You’re no longer paying for idle compute time.
You no longer need to spin up a function for each incoming request, greatly reducing the likelihood of hitting a cold start.

Functions stay alive

Another benefit with Fluid Compute is that functions no longer shut down connections immediately after responding, which enables streaming for purposes like dynamic rendering of components or those “thinking out loud” features you see in ChatGPT, where the responses to your prompt come to you in what seems to be one character at the time.

Serverless Servers but, you know… better

Vercel presents Fluid Compute as the thing that brings together the best from both worlds, with a paradigm that aims to introduce real concurrency, minimise cold starts and maximise CPU efficiency the way a traditional server would while also providing the main advantages of serverless:

Auto-scaling
Automatic provisioning and optimisation
Pay-per-use pricing
The developer experience so many of us know and love

Tailor made for Gen AI

It’s hard to overlook the fact that this improvement couldn’t have come at a more relevant time. In a time where everybody is jumping on the AI bandwagon and exploring new ways to build on top of the mainstream GenAI models, it isn’t uncommon for these requests to take many seconds or even minutes to respond. Seconds and minutes that you ultimately pay for while your traditional serverless functions remain idle.

AWS Lambda introduced the world to the concept of true serverless computing in 2014 and while many great options have popped up since, I don’t think I’m alone in the camp that still considers Lambda the king of serverless.

Vercel’s Fluid Compute poses some very interesting questions, however.

I’m in the middle of writing an AI based application myself and was just in the process of figuring out what the best hosting approach would be. I’m a serverless-first (almost serverless-only) dev these days and was a bit torn between a few options that all had their trade-offs.

My DX with AWS has always been amazing, especially since CDK came out with v2. But for a project that relies so heavily on GenAI and the slow nature of the APIs that come with it, Vercel just made themselves the obvious choice.

How will AWS answer?

Whether AWS Lambda has yet to figure out in-function concurrency or already has and simply chose to not “pass the savings on”, Fluid Compute provides a competitive edge in terms of cost efficiency, and it’s hard for me to imagine Lambda won’t keep up.

The ball is now in AWS’s court, will they respond? My hope and expectation is that they will.

As someone who relies heavily on Bun during development, I’ll tell you It’s no coincidence that NodeJS started churning out features like Typescript support, built in ability to read environment variables, etc. Bun was putting on pressure.

Huge props to the folks over at Vercel, for being to AWS Lambda what Bun is to Node. The game is on, and I’m excited to see what AWS comes up with in the coming months.

Diving Into AWS: A Smarter Start

Andreas Casen — Mon, 16 Dec 2024 15:26:14 +0000

Many of us sign into AWS for the first time to learn or build something—whether it’s hosting a web app, running AI models, or automating processes. The urge to dive into a service and figure it out is natural and reflects the curiosity that drives great developers.

But AWS’s complexity can lead to pitfalls we might not see coming, and while mistakes are part of learning, some lessons are best avoided the hard way. To help you make the most of your early AWS experience, here are a few essential tips to stay safe and maximize your time.

Disclaimer

This post is for individuals starting with a personal AWS account. Companies with bigger goals should bring in experts early to build a solid foundation (more on this in a future post).

Here are my top recommendations for Day 1 of your AWS journey:

Safety First

The power and complexity that AWS and other cloud providers are capable of are astonishing, but also present the potential for an innocent mistake or lack of basic knowledge to cause massive damage in the form of data loss or extremely high costs, not to mention the risks of an account getting hijacked because of exposed credentials or anything along those lines.

Make sure you’ve checked these boxes before you move on to the fun stuff:

Secure access

Root Account MFA

Relying on username and password as your lone security mechanism paints a target on your back. Set up MFA for your account or user before you do anything else. Read more on that here.

Do not use your root account

Once that is out the way, you want to use your root account user as little as possible and instead do the following:

Look into creating a IAM user, it’s what you should be using in your day to day work in AWS. If you’re a company or dev team aiming to do serious work, look into setting up federated access through IAM Identity Center.

Regardless of your use case, make it your highest priority to understand the basics of how to access AWS correctly and securely, both through the CLI and the Management Console. You might want to look into this article about securing your AWS account.

Cost alerts

Most AWS services offer a free tier for 12 months or indefinitely, but mistakes can still lead to surprise bills that break the bank. Unfortunately, AWS doesn’t allow you to set a hard spending cap (yes, AWS, this is a complaint), but you can use AWS Budgets to set alerts when your costs exceed a chosen threshold. Learn more here and set it up!

Privacy by default

Managing access and permissions between AWS resources can be challenging, especially early on. Many “solve” this by toggling “allow public access” or using overly broad IAM policies to get things working.

This approach creates serious risks, from resource abuse to data leaks. The urge to “just get things working” often leads to bad habits that aren’t addressed until disaster strikes.

Take the time to understand IAM, VPC, and the services you use.

An ocean of tech

Anytime someone says the know AWS, I reply with a question: which parts of it?

AWS offers over 200 services, spanning areas like container orchestration, compute, CDN, machine learning, and CI/CD. Even AWS employees admit no one fully understands the entire ecosystem. On top of that, the AWS ecosystem is under constant change with floods of updates and new releases.

For us, that means two key takeaways:

Eternal Impostor Syndrome

No matter how good you get at any given area of AWS, there will always be a lot of things you still don’t know, many details you don’t fully grasp. Just like with tech in general, you’ll probably never know it all.

I personally think that is what makes tech so exciting and motivating, but the fact is that there are inevitable slumps where all the stuff we know some times gets overshadowed by the stuff we don’t. It’s a part of the cycle, so you better expect it.

Focus is key

AWS offers virtually infinite services, from general concepts like security and scalability to specific use cases like web hosting, automation, or machine learning. The best way to navigate AWS’s vast ecosystem is to focus on what you’re trying to build or solve. Whether you’re hosting an app or centralising data, focus on the AWS services that best fit your goals.

When I started with AWS, my focus was web development, though I’ve since learned adjacent areas like monitoring and security. But my core toolbox in AWS is constituted of services that, at least in my case, are highly related to software and web development.

The clearer your purpose for using AWS, the easier it is to focus on the services that matter most.

The community

The AWS and tech community is my favorite part of being in this field. Attending meetups and hackathons since 2021 has transformed my motivation and personal development. I’ve met countless inspiring people, made close friends, and explored paths I couldn’t have on my own.

I also love witnessing others’ journeys and how tech culture evolves in different companies. Over time, I’ve become more active in user groups and forums, and I’m constantly amazed at how much there is to learn and experience once you start connecting with people.

Find your community early

Yet, I can’t help but think that I might have been much further along in my own journey and development if I had gotten involved in the community earlier, so my advice is simple:

Find your local tech community early. If your town has an AWS User Group that’s awesome but if not, just find a community around your niche. There’s nothing like being surrounded by inspiring people to propel you forward and thankfully for us in tech, there are plenty of groups to chose from.

The power of mentors

As freeing as a blank page can be, it can get overwhelming to enter a complex domain without a clear direction. Focusing on a specific area helps narrow your scope—like building a house one brick at a time.

Mentors are invaluable for smoothing your learning curve. Meetups and gatherings are great places to find them, but they’re not the only option. The AWS community is vast, with amazing YouTubers, writers, and podcasters.

Here is a list of my personal favorites:

Ari Palo
Rossana Suarez (for spanish speakers)
Yan Cui
Eric Johnson
Luc van Donkersgoed (and his site, The Unofficial AWS News Feed)

Explore early to find the voices and formats that resonate most with you.

To summarise

If you’re early on in your AWS journey or maybe just thinking about getting started, the best advice I can give you is:

Be safe

Make sure you’re not setting yourself up for unpleasant surprises and secure your AWS account. Understand how to experiment in a safe and secure way.

Have a focus area

You’ll be tempted to learn it all but the truth is you can’t, especially early on. Many services apply to most of us, but many don’t. The more you can go in with an idea of what it is you want to learn to build, the easier it will be to figure out what to learn and in what order.

Engage in the community, early

Find local user groups, meetups, hackatons and online forums. Explore them early and start finding the groups and environments that suit you. It’s impossible to quantify the value a good community will provide in terms of knowledge, inspiration and bonds, and AWS happens to have one of the most vibrant and engaged communities out there. Use it!

Thanks a ton for reading this far! If you’re early on in your AWS journey I sincerely hope you found some value, and if you’re still on the fence about starting out, listen to Nike: Just Do It

Have anything you’d add to the list? Put it in the comments or reach out to me on LinkedIn or BlueSky, I’ll be happy to chat! 😊

Re:Invent 2024 - A New Hope

Andreas Casen — Thu, 28 Nov 2024 07:23:46 +0000

AWS has historically been highly centered around its developer community, consistently keeping their ears to the ground and staying tuned to what "the people" wanted. For the past year however, many of us in the AWS community have expressed concern over how much of the new services, announcements and presentations have been focused on or related to AI.

And look, we get it: The AI trend is a thing, and a business can only sell what customers are looking for, but it really started to feel like web developers, devops engineers and architects were being forgotten about and our favourite services becoming afterthoughts.

Nothing paints a clearer picture than this article, recapping re:Invent 2023 by highlighting:

New capabilities for Amazon Bedrock
Expanded choice of models in Amazon Bedrock
New SageMaker capabilities
Amazon Q
Other serverless innovations

Note how innovations to Aurora, Redshift, S3 and ElastiCache where all bundled together and mentioned as a footnote. An accurate representation of the times.

The Turnaround

Heading into AWS re:Invent 2024, however, AWS seems to have gotten the message loud and clear. We're a week away from Re:Invent, but a huge list of amazing announcements and releases covering all kinds of services and areas have already been dropping over the last month.

Among these announcements, some of my personal highlights include:

DynamoDB announcing huge price reductions to on-demand throughput (50%) and global tables (up to 67%)
Cloudwatch adding context to observability data
Powerful improvements to OpenSearch Ingestion
A huge makeover to the coding experience in the Lambda Console
Amazon Aurora Serverless actually becoming serverless
Simple but impactful workflow improvements in areas like the management console and Step Functions
Amazon Q Developer now provides natural language cost analysis (I never said AI was bad 😁).

And of course, like Ari Palo points out in this post, the Lambda team deserves major props for their recent commitment to keeping up more closely with runtimes, adding support for Node.js 22 a mere month after it's LTS release.

Conclusion

I might be biased by hope but it really feels like AWS got right back on track with this recent storm of announcements, and it validates the fact that we can continue to bet and rely on AWS as a community. Who knows, maybe this was the planned roadmap all along with last year being a premeditated temporary focus on AI, maybe not.

All I know is I'm relieved AWS is back to focusing its heavy guns on what they do best, and I know I'm not alone in this feeling.

If you've made it so far, thanks a ton for reading my first post, I hope you found some value!

If you want to stay tuned on a day to day basis on new AWS related posts and announcements, make sure to keep an eye on the AWS News Feed, an amazing site built by Luc van Donkersgoed that tracks all kinds of AWS related news.

Want to chat?

Let's connect on LinkedIn, BlueSky, or find me on my website.