DEV Community: Jeremiah Anku Coblah

Building a Production-Ready URL Shortener with NestJS, Supabase, and Redis: A Complete Journey

Jeremiah Anku Coblah — Tue, 14 Oct 2025 12:28:37 +0000

TL;DR: I built a complete URL shortener using NestJS, Supabase, and Redis — packed with analytics, caching, auto-expiring links, and encrypted storage. In this post, I walk through the entire process, from the initial idea to the final build, sharing the challenges I ran into and what I learned along the way.

🎯 Why Build Another URL Shortener?

URL shorteners may look like a solved problem — with tools like Bitly and TinyURL already around — but building one from scratch is an amazing way to learn and explore concepts like:

Backend architecture (API design, database modeling)
Performance optimization (caching strategies, query optimization)
Security (encryption, authentication, authorization)
DevOps (deployment, monitoring, scaling)
Data engineering (analytics, time-series data)

Plus, you gain complete control over your data and can customize features exactly how you want them.

📋 What We're Building

Our URL shortener includes:

✅ Core Features:

URL shortening with custom aliases
Fast redirects (< 50ms with caching)
Automatic URL expiration
Password-protected URLs (ready for implementation)
Collision-free short code generation

✅ Analytics:

Click tracking with geographic data (country, city)
Device type detection (mobile, desktop, tablet)
Browser and OS breakdown
Referrer tracking
Time-series data (daily, weekly, monthly trends)

✅ Performance:

Redis caching for lightning-fast redirects
Non-blocking click recording
Intelligent cache invalidation
Connection pooling

✅ Security & Reliability:

AES-256 URL encryption
Supabase Row Level Security (RLS)
JWT authentication
Automated cleanup cron jobs

🛠 The Tech Stack

After researching various options, I settled on this stack:

Component	Choice	Why?
Backend	NestJS	Modular architecture, TypeScript support, excellent documentation
Database	Supabase (PostgreSQL)	Managed database + auth + RLS out of the box
Cache	Redis	Industry standard for fast in-memory caching
Language	TypeScript	Type safety prevents bugs, better developer experience
Analytics	geoip-lite + ua-parser-js	Lightweight, no external API dependencies

Why Supabase Over Traditional PostgreSQL?

Supabase was a game-changer. Instead of setting up:

PostgreSQL server
Authentication system
Authorization logic
API layer
Real-time subscriptions

I got all of this built-in with Supabase. The Row Level Security (RLS) feature alone saved me from writing hundreds of lines of permission-checking code.

🏗 System Architecture

Here's the high-level architecture:

┌─────────────┐
│   Client    │ (Makes request to short URL)
└──────┬──────┘
       │
       ▼
┌─────────────┐
│  NestJS API │ (Checks cache, queries DB)
└───┬─────┬───┘
    │     │
    ▼     ▼
┌────────┐ ┌──────────┐
│ Redis  │ │ Supabase │
│ Cache  │ │  (Auth   │
│        │ │   + DB)  │
└────────┘ └──────────┘

Request Flow for a redirect:

User clicks https://short.url/abc123
NestJS checks Redis cache
Cache hit? → Redirect immediately (15-50ms)
Cache miss? → Query Supabase → Cache result → Redirect
Asynchronously record click with analytics data

💾 Database Design

Schema

I designed three main tables:

1. profiles - User accounts

CREATE TABLE profiles (
  id UUID PRIMARY KEY REFERENCES auth.users(id),
  email TEXT NOT NULL,
  created_at TIMESTAMPTZ DEFAULT NOW()
);

This links to Supabase's built-in auth.users table.

2. urls - The shortened URLs

CREATE TABLE urls (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  short_code TEXT UNIQUE NOT NULL,          -- "abc123"
  long_url TEXT NOT NULL,                   -- Encrypted!
  user_id UUID REFERENCES profiles(id),
  custom_alias BOOLEAN DEFAULT FALSE,
  password_hash TEXT,                       -- For password-protected URLs
  expires_at TIMESTAMPTZ,                   -- Auto-expiration
  is_active BOOLEAN DEFAULT TRUE,
  created_at TIMESTAMPTZ DEFAULT NOW(),
  updated_at TIMESTAMPTZ DEFAULT NOW()
);

Key Decision: I encrypt long_url using AES-256. Even if someone gains database access, they can't see the original URLs without the encryption key.

3. clicks - Analytics data

CREATE TABLE clicks (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  short_code TEXT NOT NULL REFERENCES urls(short_code),
  clicked_at TIMESTAMPTZ DEFAULT NOW(),
  ip_address TEXT,
  country TEXT,
  city TEXT,
  referrer TEXT,
  device_type TEXT,                         -- mobile/desktop/tablet
  browser TEXT,
  os TEXT
);

Indexing Strategy

Performance optimization through strategic indexing:

-- Critical for fast lookups
CREATE INDEX idx_urls_short_code ON urls(short_code);

-- Fast user URL queries
CREATE INDEX idx_urls_user_id ON urls(user_id);

-- Analytics queries
CREATE INDEX idx_clicks_short_code ON clicks(short_code);
CREATE INDEX idx_clicks_clicked_at ON clicks(clicked_at);

Result: Queries that took 200ms now take 5-10ms.

🔐 Security: Row Level Security (RLS)

One of Supabase's killer features is Row Level Security. Instead of writing permission checks in my code, I define policies at the database level:

-- Users can only see their own URLs
CREATE POLICY "Users can view own URLs"
  ON urls FOR SELECT
  USING (auth.uid() = user_id);

-- Anyone can read URLs for redirects (but not see user_id)
CREATE POLICY "Anyone can read URLs for redirects"
  ON urls FOR SELECT
  USING (TRUE);

-- Users can only view clicks for their URLs
CREATE POLICY "Users can view clicks for own URLs"
  ON clicks FOR SELECT
  USING (
    short_code IN (
      SELECT short_code FROM urls WHERE user_id = auth.uid()
    )
  );

Why This Is Powerful:

Security enforced at database level
Impossible to bypass with API manipulation
Policies are centralized (not scattered across code)
Performance: PostgreSQL optimizes RLS queries

⚡ Performance: The Caching Strategy

The Problem

Without caching, every redirect requires:

Database query (~50-100ms)
Decryption (~1-2ms)
HTTP redirect

For a popular short URL with 1000 clicks/second, that's 1000 database queries per second. Not scalable.

The Solution: Redis

I implemented a two-tier caching strategy:

Tier 1: URL Data Cache

// Cache structure
{
  "abc123": {
    "long_url": "https://example.com/actual-url",
    "expires_at": "2025-10-20T10:00:00Z"
  }
}

TTL: 24 hours

Invalidation: On URL update/delete

Tier 2: Analytics Cache

// Cache key: stats:abc123
{
  "total_clicks": 1567,
  "clicks_last_7_days": 234,
  "top_countries": [...],
  "device_breakdown": [...]
}

TTL: 5 minutes

Why shorter?: Analytics can be slightly stale, but not too much

The Results

Scenario	Without Cache	With Cache
Redirect time	120-200ms	15-50ms
Database queries	Every request	Only cache misses
Scalability	~100 req/s	~10,000 req/s

🎯 Implementing Core Features

1. Short Code Generation

I use Base62 encoding (a-z, A-Z, 0-9) for short codes:

export const CodeGenerator = (): string => {
  const chars = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789';
  let code = '';
  for (let i = 0; i < 6; i++) {
    code += chars.charAt(Math.floor(Math.random() * chars.length));
  }
  return code;
};

Why 6 characters?

62^6 = 56.8 billion possible combinations
Even at 1 million URLs, collision probability is ~0.002%

Collision Handling

What if we generate a code that already exists? Retry logic:

let attempts = 0;
while (attempts < 5) {
  const { data: existingCode } = await supabase
    .from('urls')
    .select('short_code')
    .eq('short_code', shortCode)
    .maybeSingle();

  if (!existingCode) break; // Code is unique!

  shortCode = CodeGenerator(); // Try again
  attempts++;
}

Trade-off: Slight performance hit on collisions vs. guaranteed uniqueness.

2. URL Encryption

I encrypt all URLs before storing them:

import * as crypto from 'crypto';

export const encrypt = (text: string): string => {
  const algorithm = 'aes-256-cbc';
  const key = Buffer.from(process.env.ENCRYPTION_KEY!, 'hex');
  const iv = crypto.randomBytes(16);

  const cipher = crypto.createCipheriv(algorithm, key, iv);
  let encrypted = cipher.update(text, 'utf8', 'hex');
  encrypted += cipher.final('hex');

  return iv.toString('hex') + ':' + encrypted;
};

export const decrypt = (encrypted: string): string => {
  const [ivHex, encryptedData] = encrypted.split(':');
  const iv = Buffer.from(ivHex, 'hex');
  const key = Buffer.from(process.env.ENCRYPTION_KEY!, 'hex');

  const decipher = crypto.createDecipheriv('aes-256-cbc', key, iv);
  let decrypted = decipher.update(encryptedData, 'hex', 'utf8');
  decrypted += decipher.final('utf8');

  return decrypted;
};

Security Benefit: Even if someone dumps the database, they can't see original URLs without the encryption key (stored as environment variable, never committed to Git).

3. The Redirect Endpoint (Most Critical!)

This endpoint must be blazingly fast:

async shortCode(shortCode: string, req: Request): Promise<{ long_url: string }> {
  // ⚡ Step 1: Check cache first (fast path)
  const cache = await this.cacheManager.get<{ long_url: string }>(shortCode);
  if (cache) {
    this.recordClick(shortCode, req); // Non-blocking
    return { long_url: cache.long_url };
  }

  // 🔍 Step 2: Cache miss - query database
  const { data: url, error } = await supabase
    .from('urls')
    .select('*')
    .eq('short_code', shortCode)
    .maybeSingle();

  if (!url) {
    throw new NotFoundException(`URL not found`);
  }

  // ⏰ Step 3: Check expiration
  if (new Date(url.expires_at) < new Date()) {
    throw new NotFoundException('This URL has expired');
  }

  // 🔓 Step 4: Decrypt URL
  const decryptedUrl = decrypt(url.long_url);

  // 💾 Step 5: Cache result
  await this.cacheManager.set(shortCode, { long_url: decryptedUrl }, 86400);

  // 📊 Step 6: Record click (asynchronous - doesn't block redirect)
  this.recordClick(shortCode, req);

  return { long_url: decryptedUrl };
}

Key Optimization: Click recording happens asynchronously. The user gets their redirect immediately, and analytics are processed in the background.

4. Analytics: The recordClick Method

This method extracts everything from the request:

private async recordClick(shortCode: string, req?: Request): Promise<void> {
  try {
    // 🌍 Extract IP address
    const ip =
      (req?.headers['x-forwarded-for'] as string)?.split(',')[0]?.trim() ||
      req?.socket?.remoteAddress?.replace('::ffff:', '') ||
      null;

    // 📱 Parse user agent
    const userAgent = req?.headers['user-agent'] || '';
    const parser = new UAParser(userAgent);
    const result = parser.getResult();

    const deviceType = result.device.type || 'desktop';
    const browser = result.browser.name || 'Unknown';
    const os = result.os.name || 'Unknown';

    // 🗺️ Geo lookup
    const geo = geoip.lookup(ip);
    const country = geo?.country || null;
    const city = geo?.city || null;

    // 🔗 Referrer
    const referrer = req?.headers['referer'] || null;

    // 💾 Save to database
    await supabaseAdmin.from('clicks').insert({
      short_code: shortCode,
      clicked_at: new Date().toISOString(),
      ip_address: ip,
      referrer,
      device_type: deviceType,
      browser,
      os,
      country,
      city,
    });
  } catch (err) {
    console.error('Failed to record click:', err.message);
    // Don't throw - we don't want analytics failures to break redirects
  }
}

Why geoip-lite?

No external API calls (faster)
No rate limits
Works offline
Trade-off: Less accurate than paid services, but good enough

5. Analytics Endpoint

Aggregating click data into useful insights:

async getUrlStats(shortCode: string, userId: string): Promise<any> {
  // 🔐 Verify ownership
  const { data: url } = await supabase
    .from('urls')
    .select('*')
    .eq('short_code', shortCode)
    .single();

  if (url.user_id !== userId) {
    throw new ForbiddenException('Not your URL');
  }

  // 💾 Check cache
  const cacheKey = `stats:${shortCode}`;
  const cached = await this.cacheManager.get(cacheKey);
  if (cached) return cached;

  // 📊 Query all clicks
  const { data: clicks } = await supabase
    .from('clicks')
    .select('*')
    .eq('short_code', shortCode)
    .order('clicked_at', { ascending: false });

  // 📈 Calculate statistics
  const now = new Date();
  const sevenDaysAgo = new Date(now.getTime() - 7 * 24 * 60 * 60 * 1000);
  const thirtyDaysAgo = new Date(now.getTime() - 30 * 24 * 60 * 60 * 1000);

  const stats = {
    total_clicks: clicks.length,
    clicks_last_7_days: clicks.filter(c => new Date(c.clicked_at) >= sevenDaysAgo).length,
    clicks_last_30_days: clicks.filter(c => new Date(c.clicked_at) >= thirtyDaysAgo).length,

    // Group by date
    clicks_by_date: this.groupByDate(clicks, thirtyDaysAgo),

    // Top countries
    top_countries: this.aggregateField(clicks, 'country'),

    // Device breakdown
    device_breakdown: this.aggregateField(clicks, 'device_type'),

    // Browser breakdown
    browser_breakdown: this.aggregateField(clicks, 'browser'),

    // OS breakdown
    os_breakdown: this.aggregateField(clicks, 'os'),

    // Top referrers
    top_referrers: this.aggregateField(clicks, 'referrer'),
  };

  // 💾 Cache for 5 minutes
  await this.cacheManager.set(cacheKey, stats, 300);

  return stats;
}

Optimization Opportunity: For URLs with millions of clicks, querying all clicks every time is slow. Future enhancement: Daily aggregation table (pre-computed stats).

6. Automated Expiration with Cron Jobs

URLs can have expiration times. I built a cron job to automatically delete expired URLs:

@Cron(CronExpression.EVERY_MINUTE)
async handleExpiredUrls() {
  console.log('⏰ Checking for expired URLs...');

  const now = new Date().toISOString();

  // Use service role to bypass RLS
  const supabaseAdmin = createClient(
    this.configService.get<string>('SUPABASE_URL')!,
    this.configService.get<string>('SUPABASE_SERVICE_ROLE_KEY')!
  );

  // Find expired URLs
  const { data: expiredUrls } = await supabaseAdmin
    .from('urls')
    .select('id, short_code')
    .lt('expires_at', now)
    .limit(100);

  if (!expiredUrls || expiredUrls.length === 0) {
    console.log('✅ No expired URLs found.');
    return;
  }

  console.log(`🗑 Found ${expiredUrls.length} expired URL(s). Deleting...`);

  // Delete them
  const ids = expiredUrls.map(u => u.id);
  const shortCodes = expiredUrls.map(u => u.short_code);

  await supabaseAdmin.from('urls').delete().in('id', ids);

  // Clear from cache
  for (const code of shortCodes) {
    await this.cacheManager.del(code);
  }

  console.log(`🧹 Successfully deleted ${expiredUrls.length} expired URLs`);
}

Why every minute? Balance between:

Immediate cleanup (every second = too many queries)
Delayed cleanup (every hour = expired URLs still work for too long)

📊 API Documentation with Swagger

NestJS makes API documentation effortless with decorators:

@ApiTags('URLs')
@ApiBearerAuth()
@Controller('urls')
export class UrlsController {

  @Post('shorten')
  @UseGuards(SupabaseAuthGuard)
  @ApiOperation({
    summary: 'Shorten a long URL',
    description: 'Creates a new shortened URL for the authenticated user.'
  })
  @ApiBody({ type: UrlsDto })
  @ApiCreatedResponse({
    description: 'Successfully created',
    type: ShortUrlResponseDto
  })
  async createShortUrl(@Body() body: UrlsDto, @Req() req: any) {
    return this.urlsService.createShortUrl(
      body.long_url,
      req.user.id,
      req.accessToken,
      body.password,
      body.customAlias
    );
  }
}

Result: Beautiful, interactive documentation at /api endpoint.

🚧 The AWS Deployment Challenge

The Plan

I initially planned to deploy on AWS with this architecture:

┌─────────────┐
│ CloudFront  │ (CDN for edge caching)
└──────┬──────┘
       │
       ▼
┌─────────────┐
│     EC2     │ (NestJS app on t2.micro)
└───┬─────┬───┘
    │     │
    ▼     ▼
┌────────┐ ┌──────────┐
│ElastiCache││ Supabase │
│ (Redis) │  │ (external)│
└────────┘  └──────────┘
       │
       ▼
┌─────────────┐
│     SQS     │ (Queue for analytics)
└──────┬──────┘
       │
       ▼
┌─────────────┐
│   Lambda    │ (Process click events)
└─────────────┘

Benefits:

Global CDN caching
Auto-scaling Lambda workers
Managed Redis
Free tier eligible

The Reality

AWS deployment hit several roadblocks:

1. IAM Permission Hell

Every service needs specific permissions:

EC2 needs ElastiCache access
Lambda needs SQS read permissions
Lambda needs Supabase network access
CloudWatch needs logging permissions

Creating the right IAM policies took hours of trial and error.

// Example of complex IAM policy needed
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "sqs:ReceiveMessage",
        "sqs:DeleteMessage",
        "sqs:GetQueueAttributes"
      ],
      "Resource": "arn:aws:sqs:us-east-1:123456789:click-queue"
    },
    {
      "Effect": "Allow",
      "Action": ["logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents"],
      "Resource": "*"
    }
  ]
}

2. VPC Networking Complexity

ElastiCache requires VPC
Lambda in VPC can't access internet (needs NAT Gateway)
NAT Gateway costs $32/month (not free tier!)
Security groups need careful configuration

3. Cost Monitoring Anxiety

AWS's pay-as-you-go model is powerful but scary:

Forgot to delete a snapshot → $5 charge
Left NAT Gateway running → $32/month
CloudFront overages → surprise bill

Setting up billing alerts helped, but the anxiety was real.

4. Learning Curve

AWS has 200+ services. Even for a simple app, I needed to understand:

EC2 (compute)
ElastiCache (managed Redis)
SQS (message queue)
Lambda (serverless functions)
CloudFront (CDN)
VPC (networking)
IAM (permissions)
CloudWatch (monitoring)
Route 53 (DNS)

Each service has its own quirks, best practices, and gotchas.

The Decision

After spending 2 days on AWS configuration with limited progress, I made a tough call:

Focus on core functionality first, defer AWS deployment.

Why?

Core app works perfectly locally
Can demonstrate all features
Can always deploy later with better knowledge
Simpler alternatives exist (Vercel, Railway, Render)

Lessons Learned:

AWS is powerful but has a steep learning curve
Start with simpler deployment platforms for MVPs
Document everything (future me will need it)
Don't let DevOps block feature development
Know when to defer and move forward

I documented the entire AWS setup process in AWS-DEPLOYMENT-GUIDE.md for future reference. When budget and time permit, I'll revisit it with a clearer understanding.

🎓 Key Takeaways

Technical Lessons

Caching is Not Optional
- Redis reduced response time by 10x
- Proper cache invalidation is crucial
- Monitor cache hit rates
Async Everything
- Don't block user-facing requests
- Analytics can be processed later
- Use queues for heavy workloads
Security in Layers
- Database-level security (RLS)
- Encrypted storage
- API authentication
- Input validation
Type Safety Saves Time
- TypeScript caught bugs at compile time
- Refactoring is much safer
- IDE auto-completion speeds development
Monitor from Day One
- Logging helped debug issues fast
- Cron job logs showed silent failures
- Performance metrics guide optimization

Project Management Lessons

Start Simple, Iterate
- Built URL shortening first
- Added analytics second
- Deferred complex features (AWS, custom domains)
Perfect is the Enemy of Done
- AWS deployment was blocking progress
- Shipped working product > unfinished AWS setup
- Can always improve later
Document Everything
- README helps others (and future you)
- Code comments explain "why", not "what"
- Architecture diagrams prevent confusion
Know Your Trade-offs
- geoip-lite vs paid service: speed vs accuracy
- Local dev vs cloud deployment: simplicity vs scalability
- Every decision has trade-offs

📈 Performance Metrics

Here's how the system performs:

Metric	Value	Target
Redirect time (cached)	15-50ms	< 100ms ✅
Redirect time (uncached)	80-150ms	< 200ms ✅
Database query time	5-15ms	< 50ms ✅
Analytics calculation	1-3s (10k clicks)	< 5s ✅
Cron job execution	200-500ms	< 1s ✅
Cache hit rate	~85%	> 70% ✅

Load Testing Results (using Artillery):

1,000 concurrent users
10,000 requests over 60 seconds
0% error rate
Average response time: 45ms

🚀 Future Enhancements

Short-Term (Next 2-4 weeks)

[ ] QR code generation endpoint
[ ] Bulk URL import via CSV
[ ] Email notifications for expired URLs
[ ] Rate limiting (prevent abuse)
[ ] User dashboard frontend (React)

Medium-Term (1-3 months)

[ ] Custom domains (go.yourcompany.com/abc123)
[ ] A/B testing (multiple destinations, weighted routing)
[ ] Webhook notifications on click events
[ ] URL preview with Open Graph data
[ ] Mobile app (React Native)

Long-Term (3-6 months)

[ ] AWS deployment with full CI/CD
[ ] Daily aggregation table for faster analytics
[ ] Machine learning: detect malicious URLs
[ ] Multi-region deployment
[ ] GraphQL API

💡 If I Started Over, I'd...

Start with simpler deployment (Railway/Render instead of AWS)
Add tests from day one (I learned this the hard way)
Use a daily aggregation table (for analytics performance)
Implement rate limiting early (prevent abuse before it happens)
Add comprehensive logging (more structured logs, not just console.log)

🎯 Conclusion

Building this URL shortener taught me more than any tutorial could:

Backend architecture: API design, database modeling, caching strategies
Performance optimization: Query optimization, indexing, async processing
Security: Encryption, RLS, authentication best practices
DevOps: Deployment challenges, monitoring, cost management
Project management: Knowing when to defer, when to ship

The app runs smoothly on local environments, efficiently manages multiple requests at once, and delivers detailed analytics. Although deploying it to AWS is still on the roadmap, the core system is fully built and showcases up-to-date backend development techniques.

The biggest lesson? Ship working code. Perfect deployment can wait.

📚 Resources

If you want to build your own:

Documentation:

Libraries Used:

@nestjs/common - Core NestJS framework
@supabase/supabase-js - Supabase client
@nestjs/cache-manager - Redis caching
ua-parser-js - User agent parsing
geoip-lite - IP geolocation
bcrypt - Password hashing

Tutorials That Helped:

🔗 Links

GitHub Repository: github.com/yourusername/url-shortener
Live Demo: Coming soon (post-AWS deployment)
API Documentation: Available locally at localhost:3000/api

💬 Let's Connect!

I'd love to hear your thoughts:

What features would you add?
Have you faced similar AWS challenges?
What's your preferred deployment platform?

Drop a comment below or reach out:

Twitter: @yourhandle
LinkedIn: Your Name
Email: jeremiah.anku.coblah@gmail.com

If you found this helpful, please give it a ❤️ and share it with others!

📝 Changelog

v1.0.0 (October 2025)

✅ Initial release
✅ URL shortening with custom aliases
✅ Redis caching
✅ Comprehensive analytics
✅ Auto-expiration cron job
✅ Supabase Auth & RLS
✅ Swagger documentation
📝 AWS deployment documented (not yet implemented)

Tags: #NestJS #Supabase #Redis #Backend #TypeScript #API #URLShortener #WebDevelopment #PostgreSQL #Caching

Happy coding! 🚀

Building a Real-Time Matchmaking System with NestJS and MongoDB 🎮

Jeremiah Anku Coblah — Sat, 04 Oct 2025 12:20:02 +0000

How I Built a Competitive Gaming Matchmaking API Like Valorant and CS:GO

Have you ever wondered how games like Valorant, CS:GO, or League of Legends instantly find you opponents at your skill level? In this article, I'll walk you through building a production-ready matchmaking system from scratch using NestJS, MongoDB, and intelligent algorithms.

🎯 What We're Building

A backend matchmaking system that:

✅ Automatically pairs players based on skill level (ELO rating)
✅ Groups players by geographic region to minimize lag
✅ Implements a fair queueing system (FIFO - First In, First Out)
✅ Uses smart algorithms to balance wait times vs match quality
✅ Updates player ratings after matches using the ELO system
✅ Handles edge cases like odd numbers of players and long wait times

This isn't just CRUD - it's a real-world system design challenge that requires algorithm thinking, database optimization, and state management.

🏗️ System Architecture

Core Components

┌─────────────┐      ┌──────────────┐      ┌─────────────┐
│   Players   │ ───▶ │  Queue API   │ ───▶ │   Queue DB  │
└─────────────┘      └──────────────┘      └─────────────┘
                            │
                            ▼
                   ┌─────────────────┐
                   │  Matchmaking    │ ◀──── Runs every 10s
                   │    Service      │
                   └─────────────────┘
                            │
                            ▼
                   ┌─────────────────┐
                   │   Match API     │
                   └─────────────────┘
                            │
                            ▼
                   ┌─────────────────┐
                   │   Match DB      │
                   └─────────────────┘

📊 Database Schema Design

User Schema

Stores player profiles and their current state:

{
  _id: ObjectId,
  username: string,
  rating: number,        // ELO rating (starts at 1200)
  region: string,        // Encrypted: "NA", "EU", "ASIA"
  status: string,        // Encrypted: "idle", "searching", "in_match"
  matchHistory: ObjectId[],
  createdAt: Date
}

Why encrypt region and status? Security best practice for sensitive player data.

Queue Schema

The "waiting room" for players looking for matches:

{
  _id: ObjectId,
  userId: ObjectId,      // Reference to User
  username: string,      // Denormalized for faster queries
  rating: number,        // Denormalized to avoid joins
  region: string,
  mode: string,          // "1v1", "2v2", "5v5"
  status: "searching",
  joinedAt: Date
}

Denormalization Trade-off: We duplicate username and rating here for query performance. In matchmaking, speed matters more than storage.

Match Schema

Records of created matches:

{
  _id: ObjectId,
  status: string,        // "pending", "active", "finished"
  mode: string,
  players: [{
    userId: ObjectId,
    username: string,
    rating: number,
    team?: string        // For team-based modes
  }],
  winner: ObjectId,
  result: {
    winnerRating: number,
    loserRating: number,
    ratingChange: number
  },
  createdAt: Date
}

🧠 The Matchmaking Algorithm

This is the heart of the system. Here's how it works:

Step 1: Automatic Scheduling

@Cron(CronExpression.EVERY_10_SECONDS)
async runMatchmaking() {
  this.logger.log('🔍 Starting matchmaking scan...');
  await this.find1v1Matches();
}

Using NestJS's @Cron decorator, the matchmaking service runs automatically every 10 seconds in the background.

Step 2: Fetch and Group Players

const queuedPlayers = await this.queueModel.aggregate([
  { $match: { mode: '1v1' } },
  { $sort: { joinedAt: 1 } },        // FIFO ordering
  {
    $group: {
      _id: '$region',                 // Group by region
      players: { $push: '$$ROOT' }
    }
  }
]);

Why MongoDB Aggregation? It's incredibly efficient for complex queries. We get region-grouped, time-sorted data in a single database call.

Step 3: Compatibility Check

arePlayersCompatible(player1: any, player2: any): boolean {
  // Rule 1: Same region
  if (player1.region !== player2.region) return false;

  // Rule 2: Rating tolerance
  const ratingDiff = Math.abs(player1.rating - player2.rating);
  const RATING_TOLERANCE = 100;

  if (ratingDiff > RATING_TOLERANCE) return false;

  // Rule 3: Wait time flexibility
  const avgWaitTime = (player1WaitTime + player2WaitTime) / 2;
  if (avgWaitTime > 30000) {
    // After 30s, allow ±200 rating difference
    return ratingDiff <= 200;
  }

  return true;
}

Smart Trade-offs:

First 30 seconds: Strict ±100 rating for fair matches
After 30 seconds: Lenient ±200 rating to prevent long waits

Step 4: Match Creation with Transactions

async createMatch(players: any[], mode: string) {
  const session = await this.queueModel.db.startSession();
  session.startTransaction();

  try {
    // 1. Create match document
    const match = new this.matchModel({ /* ... */ });
    await match.save({ session });

    // 2. Remove players from queue
    await this.queueModel.deleteMany({ 
      userId: { $in: userIds } 
    }, { session });

    // 3. Update user statuses to "in_match"
    await this.userModel.updateMany({ 
      _id: { $in: userIds } 
    }, { 
      status: encrypt('in_match') 
    }, { session });

    // 4. Add to match history
    await this.userModel.updateMany({ 
      _id: { $in: userIds } 
    }, { 
      $push: { matchHistory: match._id } 
    }, { session });

    await session.commitTransaction();
    return match;
  } catch (error) {
    await session.abortTransaction();
    throw error;
  } finally {
    session.endSession();
  }
}

Why Transactions? Ensures atomicity. Either all operations succeed, or none do. This prevents bugs like:

Player in two matches simultaneously
Match created but players still in queue
Inconsistent state between collections

🎲 ELO Rating System

After a match finishes, we update ratings using the classic ELO algorithm:

calculateELO(winnerRating: number, loserRating: number) {
  const K = 32; // K-factor (rating change magnitude)

  // Expected win probability
  const expectedWinner = 1 / (1 + Math.pow(10, (loserRating - winnerRating) / 400));
  const expectedLoser = 1 / (1 + Math.pow(10, (winnerRating - loserRating) / 400));

  // Calculate changes
  const winnerChange = Math.round(K * (1 - expectedWinner));
  const loserChange = Math.round(K * (0 - expectedLoser));

  return {
    newWinnerRating: winnerRating + winnerChange,
    newLoserRating: loserRating + loserChange,
    ratingChange: winnerChange,
  };
}

Example:

Player A (1200) beats Player B (1250)
Player A: 1200 → 1225 (+25 points)
Player B: 1250 → 1235 (-15 points)

The upset victory gives A more points since B was favored.

🔐 Security & Performance Features

1. Data Encryption

Sensitive fields like region and status are encrypted at rest:

import { encrypt, decrypt } from 'src/middlewares/encryption/encrypt';

// When saving
const encryptedRegion = encrypt(region);
const encryptedStatus = encrypt(status);

// When reading
const decryptedRegion = decrypt(user.region);

2. Region Validation

We validate region codes to prevent bad data:

if (!isValidCountryCode(region)) {
  throw new BadRequestException('Invalid region code');
}

3. Caching Layer

Frequently accessed user data is cached to reduce database load:

await this.cacheManager.set(
  `user:${username}`,
  { id, username, region },
  3600  // 1 hour TTL
);

4. Input Validation

Every API endpoint validates inputs:

// Can't join queue if already in a match
if (decryptedStatus === "in_match") {
  throw new BadRequestException("User is currently in a match");
}

// Can't join queue twice
const alreadyInQueue = await this.queueModel.findOne({ userId });
if (alreadyInQueue) {
  throw new BadRequestException("User already in queue");
}

🛣️ API Endpoints

User Management

POST   /users/register          # Create new player
GET    /users/:id               # Get player profile

Queue Management

POST   /queue/join              # Join matchmaking queue
POST   /queue/leave             # Leave queue
GET    /queue/status/:userId    # Check queue position

Matchmaking

POST   /matchmaking/trigger     # Manual trigger (testing)
GET    /matchmaking/queue-stats # Queue statistics

Match Management

GET    /matches                 # List all matches (paginated)
GET    /matches/:id             # Get match details
GET    /matches/active/all      # Get active matches
GET    /matches/history/:userId # User's match history
POST   /matches/:id/start       # Start a match
POST   /matches/:id/finish      # Finish match & update ratings
POST   /matches/:id/cancel      # Cancel match (admin)

🧪 Testing the System

Test Scenario: Create a Match

# 1. Create two players
curl -X POST http://localhost:3000/users/register \
  -H "Content-Type: application/json" \
  -d '{"username":"Alice","region":"NA","status":"idle"}'

curl -X POST http://localhost:3000/users/register \
  -H "Content-Type: application/json" \
  -d '{"username":"Bob","region":"NA","status":"idle"}'

# 2. Both join queue
curl -X POST http://localhost:3000/queue/join \
  -H "Content-Type: application/json" \
  -d '{"userId":"ALICE_ID","mode":"1v1"}'

curl -X POST http://localhost:3000/queue/join \
  -H "Content-Type: application/json" \
  -d '{"userId":"BOB_ID","mode":"1v1"}'

# 3. Wait 10 seconds (automatic matching)
# Check server logs:
# 🔍 Starting matchmaking scan...
# ✅ MATCH FOUND: Alice (1200) vs Bob (1200)
# 🎮 Match created: 65a1b2c3d4e5f6789

# 4. Verify match was created
curl http://localhost:3000/matches

# 5. Start the match
curl -X POST http://localhost:3000/matches/MATCH_ID/start

# 6. Finish match (Alice wins)
curl -X POST http://localhost:3000/matches/MATCH_ID/finish \
  -H "Content-Type: application/json" \
  -d '{"winnerId":"ALICE_ID","loserId":"BOB_ID"}'

# 7. Check updated ratings
curl http://localhost:3000/users/ALICE_ID  # Rating: 1225
curl http://localhost:3000/users/BOB_ID    # Rating: 1185

🚀 Key Technical Decisions

Why MongoDB over PostgreSQL?

Flexible Schema: Match results can vary by game mode
Aggregation Pipeline: Perfect for complex matchmaking queries
Denormalization: Trade storage for query speed
Horizontal Scaling: Better for high-concurrency gaming systems

Why NestJS?

Built-in Scheduling: @Cron decorator for background jobs
Dependency Injection: Clean, testable architecture
TypeScript: Type safety for complex algorithms
Mongoose Integration: Seamless MongoDB ODM

Why Transactions?

Gaming systems require strong consistency. A player can't be in two states simultaneously. Transactions ensure atomic operations across multiple collections.

📈 Performance Optimizations

1. Denormalization

// Queue stores username and rating directly
// Avoids JOIN operations during matchmaking
{
  userId: ObjectId,
  username: "Player1",  // ← Denormalized
  rating: 1200          // ← Denormalized
}

2. Indexing Strategy

// Queue collection indexes
queueSchema.index({ region: 1, mode: 1, joinedAt: 1 });
queueSchema.index({ userId: 1 });

// User collection indexes
userSchema.index({ username: 1 }, { unique: true });
userSchema.index({ region: 1, rating: 1 });

3. Aggregation Pipeline

Single query to get grouped, sorted players:

// Instead of:
// 1. Fetch all players
// 2. Group by region in code
// 3. Sort by join time in code

// We do:
$match → $sort → $group  // All in database!

🎓 What I Learned

Algorithm Design

Balancing fairness (skill matching) vs speed (wait times)
Implementing gradual tolerance increases
FIFO queue management

System Design

State machines (idle → searching → in_match → idle)
Background job scheduling
Atomic operations with transactions

Database Optimization

When to denormalize for performance
Effective use of aggregation pipelines
Strategic indexing for query patterns

Real-World Trade-offs

Perfect matches vs reasonable wait times
Data consistency vs system complexity
Storage cost vs query performance

🔮 Future Enhancements

1. WebSocket Integration

Real-time notifications when matches are found:

@WebSocketGateway()
export class MatchmakingGateway {
  @WebSocketServer()
  server: Server;

  notifyMatchFound(match: Match) {
    match.players.forEach(player => {
      this.server.to(player.userId).emit('match-found', match);
    });
  }
}

2. Team-Based Matchmaking

Support for 2v2 and 5v5 modes with team balancing:

async find5v5Matches() {
  // Find 10 players
  // Balance teams so sum(team1.ratings) ≈ sum(team2.ratings)
}

3. Regional Fallback

If no match in primary region after 60s, try nearby regions:

const REGION_FALLBACKS = {
  'NA': ['SA'],
  'EU': ['ME'],
  'ASIA': ['OCE']
};

4. Priority Queue

VIP players or those who've waited longest get priority matching.

5. Anti-Cheat Integration

Track suspicious rating changes, match dodging, etc.

💡 Key Takeaways

Matchmaking is NOT CRUD - It's algorithm design, state management, and real-time decision making
Transactions are critical in systems where consistency matters
Denormalization has a place when query performance is paramount
Background jobs enable autonomous system behavior
Trade-offs are everywhere - perfect vs fast, consistency vs availability

🛠️ Tech Stack

Framework: NestJS (Node.js)
Database: MongoDB with Mongoose ODM
Scheduling: @nestjs/schedule
Caching: @nestjs/cache-manager
Encryption: Custom AES encryption middleware
Validation: class-validator, class-transformer

📦 Repository & Installation

# Clone repository
git clone https://github.com/Jerry-Khobby/matchmaking-system

# Install dependencies
npm install

# Set up MongoDB
# Update connection string in app.module.ts

# Run in development
npm run start:dev

🎯 Conclusion

Building a matchmaking system taught me that backend development goes far beyond CRUD operations. It requires:

Algorithmic thinking for pairing logic
System design for state management
Database optimization for performance
Error handling for edge cases

This project demonstrates production-ready backend skills: handling concurrent users, making intelligent automated decisions, and maintaining data consistency in complex scenarios.

If you're looking to level up from tutorial projects to real system design challenges, building a matchmaking system is an excellent way to do it.

📚 Resources

Questions or suggestions? Drop a comment below! I'd love to discuss system design trade-offs and algorithm optimizations.

Found this helpful? Share it with other developers building real-world backend systems! 🚀

Tags: #NestJS #MongoDB #SystemDesign #Backend #Gaming #Algorithms #NodeJS #TypeScript