The backend was for Sheetrocket, a tool that turns any Google Sheet into a REST API or a set of no-code
embeddable widgets.
The idea is simple: developers connect
a Google Sheet and get a REST endpoint that returns their
data as JSON, while non-technical users display that same data
on any website as cards, catalogues, or tables without writing a single line of code. Update the sheet, and everything stays in sync automatically.
The problem is that Google Sheets enforces rate limits on API requests. If every request to a Sheetrocket endpoint triggered a fresh call to the Google Sheets API, I would hit those limits fast, especially under any real traffic. The obvious solution was in-memory caching: fetch the sheet data once, store it in a Map, return the cached version on subsequent requests, and only refresh from Google when the cache expired.
I had built exactly this pattern before, on long-running servers, without any issues. The cache sat at the module level, warmed up on the first request, and served every request after that from memory. Fast, simple, and it kept Google Sheets API calls to a minimum.
Vercel felt like the obvious deployment choice. One-click deploy from GitHub, a generous free tier, and I was already using Next.js for other work. It barely felt like a decision.
const cache = new Map();
export default async function handler(req, res) {
if (cache.has(req.query.id)) {
return res.json(cache.get(req.query.id));
}
const data = await fetchFromDatabase(req.query.id);
cache.set(req.query.id, data);
res.json(data);
}
Then in production, the cache never worked. Every request was hitting the Google Sheets API directly, consuming rate limit quota on every single call. The cache that worked perfectly in local development was silently empty in production every time.
I spent time assuming it was a bug in my caching logic before I realized the problem had nothing to do with my code at all.
The problem was not the code. It was where the code was running.
And the cost of that was not just slower responses; it was correctness. A cache failure here meant real Google Sheets API quota burning on every request instead of most requests getting served from memory. Under any real traffic, that means hitting Google's rate limit, getting throttled, and returning errors instead of sheet data. Visitors don't see a slow widget at that point; they see a broken one.
Why this happens
Vercel runs Next.js API routes as serverless functions, not as a traditional server that starts once and stays alive. A serverless function spins up when a request arrives and shuts down when it goes idle.
My in-memory cache lived inside that function instance. When the instance shut down, the cache went with it. When a new instance started, the cache was empty again. The code correctly populated the cache on the first request, but by the time the next request came in, Vercel may well have spun up a fresh instance with no memory of it at all.
This is why the same code behaves differently locally versus on Vercel. Locally, the Next.js development server runs as a long-running process. It starts once and stays alive, so the cache persists across requests because the same process handles them all. Everything works perfectly in development. On Vercel, the same code runs as serverless functions. The cache works sometimes, when the same warm instance happens to handle consecutive requests. It fails other times, when a cold instance with an empty cache picks up the request instead. The behavior is inconsistent, and it depends entirely on Vercel's internal instance management, which the developer has no visibility into or control over.
One sentence covers all of it: Vercel runs API routes as serverless functions, and serverless functions have no guaranteed persistent memory between requests. An in-memory cache assumes the same process handles every request. On Vercel, that assumption is simply wrong.
Before I go further: Vercel has genuinely improved here. Fluid compute now keeps functions warm between requests for longer and reduces cold starts meaningfully compared to a couple of years ago. This isn't a case against Vercel as a platform. It's a case about matching an execution model to a problem, and a pure JSON API with in-memory state and no rendering is not the problem serverless was built to solve.
Everything else that breaks, for the same reason
Every problem in this post traces back to one root cause: serverless functions are stateless, and in-memory state requires a stateful process to hold it.
Rate limiting
A common pattern tracks how many requests an IP address has made in a rolling window, using an in-memory counter.
const requestCounts = new Map();
function isRateLimited(ip) {
const count = requestCounts.get(ip) || 0;
if (count >= 10) return true;
requestCounts.set(ip, count + 1);
return false;
}
On a long-running server, this works exactly as written. On serverless, each function instance keeps its own counter. Ten concurrent requests might land on ten different instances, each with a counter showing one request. A limit meant to be ten requests per minute is effectively multiplied by however many instances Vercel has spun up. The rate limiter isn't broken loudly; it's broken silently, which is worse.
Cold start latency
When a function has been idle, Vercel has to spin up a new instance: load the Node.js runtime, import every module, establish any connections, and only then handle the request. For a simple always-on API expected to respond in milliseconds, that adds latency that is not just slower; it's unpredictable, shifting based on traffic patterns rather than staying consistent.
For an internal tool used sporadically through the day, this means the first person to use it after a quiet stretch always waits longer than everyone after them. For a customer-facing API, it's the opposite of what you'd want: the slowest responses land on whoever arrives after the longest gap, often the exact users you'd least want to leave with a bad first impression.
Database connection overhead
A long-running server maintains a connection pool to the database once, at startup. A pool of ten connections can handle thousands of requests a second by reusing those same ten connections over and over.
const pool = new Pool({
connectionString: process.env.DATABASE_URL,
max: 10,
});
app.get('/users', async (req, res) => {
const result = await pool.query('SELECT * FROM users');
res.json(result.rows);
});
A serverless function doesn't get that luxury. Each cold start either opens a fresh connection or competes with other invocations over a shared pool. If a traffic spike causes Vercel to spin up 50 instances and each one opens its own connection, you've consumed 50 of your available database connections almost instantly. PostgreSQL's free and small paid tiers often cap connections somewhere between 25 and 100. The 51st instance simply can't connect, and that request fails, not because of load on the database itself, but because of how many separate processes are all trying to reach it at once.
Background jobs
If you want to kick off a background task when a request arrives and let it keep running after the response is sent, Vercel will kill that task the moment the function shuts down. Fire-and-forget patterns that work cleanly on a long-running server don't work on serverless without extra infrastructure to keep the job alive independently.
Formgrid relies on exactly this pattern for four background operations after every form submission: Google Sheets sync, Notion sync, AI analysis, and the email notification itself. None of them block the HTTP response the visitor sees, and all of them keep running after that response is sent. That only works because Formgrid runs on a long-running Express server on Render, where the process stays alive long enough to actually finish the background work instead of being torn down mid-task.
When Vercel and serverless are genuinely the right call
This isn't an argument that serverless is bad; it's specific to what it's optimized for. Serverless is a strong choice for:
- Static sites and marketing pages, where there's no state to maintain
- Server-side rendered pages with Next.js, where rendering happens fresh per request anyway
- Edge functions serving geographically distributed, low-latency responses
- Webhooks and event-driven functions that run occasionally, not continuously, with no state requirements
- Bursty, unpredictable workloads where paying per request beats paying for idle time
- Rapid prototyping, where deployment simplicity matters more than performance tuning
A decision framework
A short set of questions, each pointing toward an answer:
Does your API maintain state between requests, caches, counters, session data? Yes: long-running server. No: either works.
Does it hold open connections to a database or external service that are expensive to establish? Yes: long-running server with connection pooling. No: either works.
Do you need consistent sub-100ms response times regardless of traffic patterns? Yes: long-running server. No: serverless is acceptable.
Do you need background processing that continues after the response is sent? Yes: long-running server. No: either works.
Is your traffic bursty and unpredictable, and is paying per request more important than consistent latency? Yes: serverless is the better economic choice. No: a long-running server is usually more cost-effective and predictable.
Is this a Next.js app with both a frontend and API routes, where the frontend genuinely benefits from Vercel's CDN and deploy experience? Yes: Vercel makes sense; the frontend benefits usually outweigh the API route limitations for most apps. No, pure API only: a long-running server on Render or Railway is almost always the better choice.
Platform comparison, with real costs
Render: the free tier spins down after inactivity, which reintroduces the same cold start problem you're trying to escape. The paid tier, $7 a month, keeps the server always on with consistent performance. Formgrid runs on a $7 a month Render instance serving around 1,000 daily visitors.
Railway: a free tier with a $5 monthly credit, then usage-based, roughly $5 to $10 a month for a small always-on API. Good developer experience, flexible for more complex setups.
Fly.io: a free tier is available, runs containers globally, a solid option for geographic distribution without full serverless complexity.
Hetzner: around 4 euros a month for a small VPS with 2GB of RAM. The cheapest option if you're comfortable managing your own server, full control, no abstraction layer.
DigitalOcean App Platform: a free tier available, managed containers, an experience fairly similar to Render.
The hybrid approach
Plenty of real applications use both, and it's worth naming because it's the actual nuance most take on this miss. A Next.js frontend on Vercel handles the marketing site, while a separate, long-running Express API on Render handles the backend that needs connection pooling and background processing. Vercel does what it's built for. Render does what it's built for.
Formgrid is built this way. The API is a long-running Express server on Render. The dashboard is a React SPA. They're separate services, deployed separately, because each one has genuinely different infrastructure requirements.
Closing
The infrastructure decision you make at the start of a project is one of the hardest to undo later. Not because the migration itself is usually technically complex; it typically isn't. But because you have to do it without anyone noticing.
Understanding what serverless is actually optimized for, before reaching for it because it feels familiar, is one of the most valuable habits you can build as an engineer. Not because serverless is wrong. Because the right tool for the right job beats the familiar tool for every job, every time.
If you've hit this same wall, or made the same call and lived to tell about it, I'd genuinely like to hear what you learned. Reach me at allen@formgrid.dev.
I'm Allen, a full-stack TypeScript engineer and the founder of SheetRocket, a tool that turns Google Sheets into REST APIs and embeddable widgets, and Formgrid, an open-source form backend and lead pipeline. Both run on long-running Express servers on Render. I write from what actually happens in production, not from theory. More at jonesstack.com.
Top comments (0)