Muhammad Abdullah

Posted on Jul 11

How I Built a Production Multi-Tenant SaaS on AWS for $94/month - And Almost Gave Up

#saas #aws #architecture #devops

I Almost Quit This Project at 2am. I'm Glad I Didn't.

After OrderFlow I thought I understood AWS.

I understood enough to be dangerous.

StatusNest was supposed to be the next step up - a real multi-tenant SaaS platform. Not a learning project. Not a portfolio checkbox. Something that actually demonstrated architecture-level thinking. Production grade. Built from scratch.

One week later I was staring at a screen at 2am watching my third consecutive deployment fail, wondering if I should just stop.

This is that story.

Why StatusNest

OrderFlow taught me event-driven architecture. Three microservices, EventBridge, SQS, SNS, clean Terraform modules. I was proud of it.

But it had no users. No tenants. No authentication. No real-world complexity beyond the messaging patterns.

I wanted to build something where the hard problems weren't just infrastructure - where multi-tenancy, security, cost optimization, and real application logic all had to work together simultaneously.

A status page platform made sense. Every SaaS company needs one. It has natural multi-tenancy - each customer gets their own page, their own monitors, their own incident history. The monitoring pipeline is genuinely event-driven. The public status pages need CDN. The auth layer needs to be real.

It touched every AWS service I wanted to prove I could use properly.

I gave myself one week.

The Architecture I Was Aiming For

Three FastAPI microservices on ECS Fargate:

Auth Service (port 8000) - registration, login, JWT
Monitor Service (port 8001) - monitor CRUD, check scheduling
Status Service (port 8002) - public status page data

Behind an ALB with path-based routing. RDS PostgreSQL for persistent storage. ElastiCache Redis for caching and rate limiting. An EventBridge → Lambda → SQS monitoring pipeline. CloudFront with WAF in front of everything. GitHub Actions OIDC for CI/CD. Full Terraform IaC. CloudWatch and X-Ray for observability.

On paper it was clear. In practice it was the hardest thing I had built.

Day One: The Port Problem That Wasn't Simple

The first thing I hit was the ECS services showing Running=0.

Tasks were registering. Services were active. Nothing was actually running. ALB health checks failing across all three services.

I spent hours checking the Terraform - security groups, task definitions, ALB target groups. Everything looked right on paper.

The actual problem: port mismatch in the Dockerfiles. The containers were exposing different ports than what the ECS task definitions were mapping. The fix sounds simple written out like that. Finding it was not simple at all. CloudWatch logs showed the containers starting and immediately stopping. No useful error message. Just an exit code.

That was day one.

The Lambda Deployment Timeout

The monitoring pipeline uses Lambda to execute health checks. When I first deployed the Lambda functions the deployments kept timing out.

The function zip was too large - I was packaging all dependencies directly into the deployment package. AWS has a 50MB compressed zip limit for direct uploads and I was hitting it.

The fix was Lambda Layers - extract the heavy dependencies into a separate layer, keep the function code itself small. But figuring that out, restructuring the packaging, updating the Terraform Lambda resources, and redeploying took most of a day.

Every time I thought I had it the deployment would run for 16 minutes and then fail. Then I'd fix something and try again. 12 minutes. Fail. 12 minutes. Fail.

2am on July 2nd

Look at this GitHub Actions history:

Jul 2, 01:42 AM  — Deploy Auth Service #4      ❌  27m 51s
Jul 2, 02:16 AM  — fix: remove duplicate redis_url line in config
                 — Deploy Auth Service #5      ✅  17m 50s

That 27-minute failed deployment at 1:42am. I was trying to get the auth service running with Redis. The config had a duplicate redis_url line that was causing a startup crash. The container would start, hit the config parsing error, and die. The ALB would keep retrying health checks. The deployment would wait the full timeout period before marking as failed.

27 minutes of watching a progress bar, knowing something was wrong, not knowing what.

Fixed it at 2:16am with a one-line change.

July 3rd: The CI/CD Failures

Jul 3, 01:59 AM  — chore: remove scratch files     ✅
Jul 3, 14:07 PM  — Add CI/CD workflows             ❌  16m 20s  (Monitor Service)
Jul 3, 14:07 PM  — Add CI/CD workflows             ❌  12m 01s  (Auth Service)
Jul 3, 14:07 PM  — Add CI/CD workflows             ❌  12m 57s  (Status Service)
Jul 3, 14:08 PM  — Remove sensitive files           ✅  11m 52s

Three services. Three simultaneous CI/CD workflow failures. Same commit. 14:07pm.

I had pushed the GitHub Actions workflows for all three services at once. All three failed. The OIDC configuration, the ECR push permissions, the ECS deploy step - all of it needed to be debugged across three separate workflows simultaneously.

And before that - 1:59am again the night before. Still going past midnight.

The Free Tier Problem

I had planned to use Aurora Serverless v2 for the database. Scales to zero when idle. Perfect for a dev environment.

Not on free tier. The minimum Aurora Serverless v2 capacity costs more than a standard RDS instance running 24/7 at t3.micro.

So I switched to standard RDS PostgreSQL. Changed the Terraform, updated connection strings across three services, redeployed everything. Half a day gone.

This is the kind of decision that doesn't show up in architecture diagrams but defines what you actually ship. Real cloud engineering is full of these tradeoffs.

The Moment I Almost Stopped

It was somewhere in the middle of that week.

Deployments were failing. The Lambda timeouts weren't resolved yet. The API was returning errors I didn't fully understand. I had been debugging for hours and the list of open issues wasn't getting shorter.

I remember thinking: this is too much. I could just leave it at OrderFlow. That's already a solid portfolio project. Nobody is forcing me to finish this.

I sat there for a while.

Then I thought about why I started. I wanted to prove I could build something at architecture level - not just follow patterns but make real decisions under real constraints. Quitting with a half-built system proved nothing except that I quit when it got hard.

So I made chai, opened CloudWatch logs, and kept going.

When It Finally Worked

The moment the StatusNest dashboard loaded for the first time - auth working, monitors creating, status page serving data through CloudFront - was one of the best feelings I've had building anything.

Not because it was perfect. It wasn't. There were still bugs. There are always still bugs.

But the architecture was running. Every component I had designed on paper was live on AWS. The EventBridge rules were firing. Lambda was executing health checks. SQS was buffering. Redis was caching. CloudFront was serving. X-Ray was tracing the full request path.

$94/month. Built from scratch. Zero manual console interaction after the initial setup.

What StatusNest Actually Is

A production-grade multi-tenant SaaS status page platform:

3 FastAPI microservices on ECS Fargate
RDS PostgreSQL - primary data store
ElastiCache Redis - caching and rate limiting
EventBridge → Lambda → SQS - monitoring pipeline
CloudFront + WAF - CDN and DDoS protection
GitHub Actions OIDC - zero stored credentials
Terraform - every resource as code
CloudWatch + X-Ray - full observability

Full architecture documented in an 8-ADR Solution Architecture Document in the repo.

What This Taught Me

Errors are information. Every failed deployment told me something specific if I read the logs carefully enough. The ones that cost me the most time were the ones where I guessed the fix instead of reading the actual error.

2am debugging is a skill. Not a fun one. But learning to stay methodical when you're tired and frustrated and nothing is working - that's what separates someone who ships from someone who has half-finished projects.

Architecture decisions have real costs. Aurora vs RDS. Lambda Layers vs direct packaging. Every choice has a tradeoff that tutorial projects never show you because tutorial projects never have real constraints.

Finishing matters more than perfect. The bugs I didn't fix before shipping taught me more than the ones I did, because I had to understand the system well enough to know which ones were safe to leave for later.

GitHub: github.com/aboodi679

The architecture diagram, full Terraform modules, and 8-ADR Solution Architecture Document are all in the repo.