DEV Community: Paradane

Building a Production-Ready API in Node.js: A Complete Walkthrough

Paradane — Mon, 15 Jun 2026 15:08:28 +0000

Most Node.js API tutorials stop at "and then you res.json() the data." But a production API needs rate limiting, proper error handling, structured logging, input validation, authentication, and a deployment strategy that won't fall over at 2 AM. At Paradane, we've built dozens of production APIs for clients — from e-commerce backends to SaaS platforms — and we've settled on a stack and patterns that work reliably.

This walkthrough covers building a production-ready REST API from scratch using Express, PostgreSQL, and a handful of battle-tested libraries. No magic frameworks — just the pieces you actually need.

The Stack

Runtime: Node.js 20 LTS
Framework: Express 4 (still the most stable and well-understood)
Database: PostgreSQL with pg (native driver, no ORM overhead)
Validation: Zod (parse, don't validate)
Auth: JWT with jsonwebtoken + bcrypt
Rate limiting: express-rate-limit
Logging: pino (fastest Node.js logger)
Testing: Vitest + Supertest
Deployment: Docker + docker-compose

Project Structure

api/
├── src/
│   ├── index.js          # Entry point, server startup
│   ├── app.js            # Express app setup, middleware
│   ├── config/
│   │   └── env.js        # Environment variable validation
│   ├── middleware/
│   │   ├── auth.js       # JWT verification
│   │   ├── errorHandler.js
│   │   ├── rateLimiter.js
│   │   └── validate.js   # Zod validation middleware
│   ├── routes/
│   │   ├── auth.js
│   │   └── users.js
│   ├── services/
│   │   └── userService.js
│   ├── db/
│   │   ├── pool.js       # pg Pool singleton
│   │   └── migrations/
│   └── utils/
│       └── logger.js
├── tests/
├── docker-compose.yml
├── Dockerfile
└── package.json

Step 1: Environment Config That Won't Bite You

Never trust process.env directly. Validate it at startup so you catch missing variables before they cause cryptic runtime errors:

// src/config/env.js
import { z } from 'zod';

const envSchema = z.object({
  PORT: z.coerce.number().default(3000),
  DATABASE_URL: z.string().url(),
  JWT_SECRET: z.string().min(32),
  NODE_ENV: z.enum(['development', 'production', 'test']).default('development'),
  RATE_LIMIT_WINDOW_MS: z.coerce.number().default(900000),
  RATE_LIMIT_MAX: z.coerce.number().default(100),
});

export const env = envSchema.parse(process.env);

If DATABASE_URL is missing or JWT_SECRET is too short, the app crashes immediately with a clear Zod error — not three hours later when someone tries to log in.

Step 2: Database Pool (Keep It Simple)

A single pg.Pool instance, exported once and imported everywhere:

// src/db/pool.js
import pg from 'pg';
import { env } from '../config/env.js';

const pool = new pg.Pool({
  connectionString: env.DATABASE_URL,
  max: 20,
  idleTimeoutMillis: 30000,
  connectionTimeoutMillis: 2000,
});

pool.on('error', (err) => {
  console.error('Unexpected pool error:', err);
});

export default pool;

No ORM. Raw SQL with parameterized queries. You get full control over query performance, and you never fight an ORM's opinion about how joins should work.

Step 3: Structured Logging with Pino

console.log is fine for debugging. It's not fine for production. Pino gives you structured JSON logs that are fast and machine-parseable:

// src/utils/logger.js
import pino from 'pino';
import { env } from '../config/env.js';

export const logger = pino({
  level: env.NODE_ENV === 'production' ? 'info' : 'debug',
  transport: env.NODE_ENV === 'development'
    ? { target: 'pino-pretty', options: { colorize: true } }
    : undefined,
});

In production, logs go to stdout as JSON. Your log aggregator (CloudWatch, Datadog, Loki) picks them up. In development, pino-pretty makes them readable.

Step 4: Validation Middleware

Zod schemas at the route level. Every incoming request body, query, and param gets validated before it touches business logic:

// src/middleware/validate.js
import { ZodError } from 'zod';

export function validate(schema) {
  return (req, res, next) => {
    try {
      req.validated = schema.parse({
        body: req.body,
        query: req.query,
        params: req.params,
      });
      next();
    } catch (err) {
      if (err instanceof ZodError) {
        return res.status(400).json({
          error: 'Validation failed',
          details: err.errors.map(e => ({
            path: e.path.join('.'),
            message: e.message,
          })),
        });
      }
      next(err);
    }
  };
}

Usage in a route:

import { z } from 'zod';
import { validate } from '../middleware/validate.js';

const createUserSchema = z.object({
  body: z.object({
    email: z.string().email(),
    password: z.string().min(8),
    name: z.string().min(1).max(100),
  }),
});

router.post('/users', validate(createUserSchema), async (req, res) => {
  const { email, password, name } = req.validated.body;
  // ... business logic
});

Step 5: Rate Limiting

express-rate-limit is simple and effective. Configure it globally, then tighten it for auth routes:

// src/middleware/rateLimiter.js
import rateLimit from 'express-rate-limit';
import { env } from '../config/env.js';

export const globalLimiter = rateLimit({
  windowMs: env.RATE_LIMIT_WINDOW_MS,
  max: env.RATE_LIMIT_MAX,
  standardHeaders: true,
  legacyHeaders: false,
  message: { error: 'Too many requests, please try again later.' },
});

export const authLimiter = rateLimit({
  windowMs: 15 * 60 * 1000, // 15 minutes
  max: 10,                  // 10 attempts per window
  message: { error: 'Too many login attempts, please try again later.' },
});

Step 6: Authentication

JWT with access tokens (short-lived) and refresh tokens (long-lived, stored in DB):

// src/middleware/auth.js
import jwt from 'jsonwebtoken';
import { env } from '../config/env.js';

export function authenticate(req, res, next) {
  const header = req.headers.authorization;
  if (!header?.startsWith('Bearer ')) {
    return res.status(401).json({ error: 'Missing or malformed token' });
  }

  try {
    const token = header.slice(7);
    const payload = jwt.verify(token, env.JWT_SECRET);
    req.user = { id: payload.sub, role: payload.role };
    next();
  } catch (err) {
    if (err.name === 'TokenExpiredError') {
      return res.status(401).json({ error: 'Token expired' });
    }
    return res.status(401).json({ error: 'Invalid token' });
  }
}

Password hashing with bcrypt (cost factor 12 — balances security and speed):

import bcrypt from 'bcrypt';

const SALT_ROUNDS = 12;

export async function hashPassword(plain) {
  return bcrypt.hash(plain, SALT_ROUNDS);
}

export async function verifyPassword(plain, hash) {
  return bcrypt.compare(plain, hash);
}

Step 7: Centralized Error Handling

Express's default error handling is barebones. A custom error handler gives you consistent error shapes and proper logging:

// src/middleware/errorHandler.js
import { logger } from '../utils/logger.js';

export function errorHandler(err, req, res, _next) {
  logger.error({ err, reqId: req.id, path: req.path }, 'Unhandled error');

  if (err.status) {
    return res.status(err.status).json({ error: err.message });
  }

  // Don't leak stack traces in production
  const message = process.env.NODE_ENV === 'production'
    ? 'Internal server error'
    : err.message;

  res.status(500).json({ error: message });
}

Create a custom error class for business logic errors:

export class AppError extends Error {
  constructor(message, status = 400) {
    super(message);
    this.status = status;
  }
}

Now your services can throw new AppError('User not found', 404) and the error handler formats it correctly.

Step 8: Putting It All Together — app.js

// src/app.js
import express from 'express';
import { globalLimiter } from './middleware/rateLimiter.js';
import { errorHandler } from './middleware/errorHandler.js';
import { logger } from './utils/logger.js';
import authRoutes from './routes/auth.js';
import userRoutes from './routes/users.js';

export function createApp() {
  const app = express();

  // Trust proxy if behind nginx/load balancer (needed for rate limiter IP detection)
  app.set('trust proxy', 1);

  // Body parsing
  app.use(express.json({ limit: '1mb' }));

  // Request ID for log correlation
  app.use((req, res, next) => {
    req.id = crypto.randomUUID();
    res.setHeader('X-Request-Id', req.id);
    next();
  });

  // Request logging
  app.use((req, res, next) => {
    const start = Date.now();
    res.on('finish', () => {
      logger.info({
        method: req.method,
        path: req.path,
        status: res.statusCode,
        duration: Date.now() - start,
        reqId: req.id,
      }, 'request completed');
    });
    next();
  });

  // Global rate limiter
  app.use(globalLimiter);

  // Health check (no auth required)
  app.get('/health', (_req, res) => {
    res.json({ status: 'ok', timestamp: new Date().toISOString() });
  });

  // Routes
  app.use('/auth', authRoutes);
  app.use('/users', userRoutes);

  // 404 handler
  app.use((_req, res) => {
    res.status(404).json({ error: 'Not found' });
  });

  // Error handler (must be last)
  app.use(errorHandler);

  return app;
}

Step 9: Graceful Shutdown

A production server needs to handle SIGTERM gracefully — finish in-flight requests, close the database pool, then exit:

// src/index.js
import { createApp } from './app.js';
import { env } from './config/env.js';
import pool from './db/pool.js';
import { logger } from './utils/logger.js';

const app = createApp();

const server = app.listen(env.PORT, () => {
  logger.info({ port: env.PORT }, 'server started');
});

async function shutdown(signal) {
  logger.info({ signal }, 'shutdown initiated');

  // Stop accepting new connections
  server.close();

  // Close database pool
  try {
    await pool.end();
    logger.info('database pool closed');
  } catch (err) {
    logger.error({ err }, 'error closing database pool');
  }

  process.exit(0);
}

process.on('SIGTERM', () => shutdown('SIGTERM'));
process.on('SIGINT', () => shutdown('SIGINT'));

Step 10: Docker Deployment

# Dockerfile
FROM node:20-alpine

WORKDIR /app

COPY package*.json ./
RUN npm ci --only=production

COPY src/ ./src/

USER node

EXPOSE 3000

CMD ["node", "src/index.js"]

# docker-compose.yml
services:
  api:
    build: .
    ports:
      - "3000:3000"
    environment:
      - DATABASE_URL=postgresql://user:pass@db:5432/mydb
      - JWT_SECRET=${JWT_SECRET}
      - NODE_ENV=production
    depends_on:
      db:
        condition: service_healthy
    restart: unless-stopped

  db:
    image: postgres:16-alpine
    environment:
      - POSTGRES_USER=user
      - POSTGRES_PASSWORD=pass
      - POSTGRES_DB=mydb
    volumes:
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U user -d mydb"]
      interval: 5s
      retries: 5

volumes:
  pgdata:

What We Skipped (And Why)

ORM (Prisma, Sequelize, TypeORM): They're great for rapid prototyping, but for production APIs with complex queries, raw SQL gives you more control and better performance. We use migration tools (like node-pg-migrate) instead of ORM-managed schemas.
GraphQL: Powerful, but adds complexity. REST with good documentation (OpenAPI/Swagger) serves most use cases. We reach for GraphQL when clients need flexible querying across multiple related resources.
TypeScript: We use it on most projects, but kept this walkthrough in plain JS for clarity. The patterns translate directly — add types to your Zod schemas and service functions.
Redis for rate limiting: express-rate-limit with in-memory storage works for single-instance deployments. Add Redis when you scale horizontally.

Testing (Don't Skip This)

// tests/users.test.js
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import request from 'supertest';
import { createApp } from '../src/app.js';

const app = createApp();

describe('GET /health', () => {
  it('returns 200 with status ok', async () => {
    const res = await request(app).get('/health');
    expect(res.status).toBe(200);
    expect(res.body.status).toBe('ok');
  });
});

The Checklist Before You Ship

[ ] Environment variables validated at startup (Zod schema)
[ ] Rate limiting on all routes, tighter on auth
[ ] All inputs validated (Zod middleware)
[ ] Structured logging (Pino, JSON in production)
[ ] Centralized error handler (no stack traces in production)
[ ] Authentication with short-lived access tokens
[ ] Passwords hashed with bcrypt (cost ≥ 12)
[ ] Health check endpoint (no auth)
[ ] Graceful shutdown (SIGTERM handling)
[ ] Database pool with connection limits and error handling
[ ] Request IDs for log correlation
[ ] Docker deployment with health checks

The Bottom Line

A production API isn't about the framework — it's about the layers around your business logic. Validation, rate limiting, logging, error handling, and graceful shutdown are what keep your API running at 2 AM when you're asleep. The patterns above have served us well across dozens of client projects, from small e-commerce backends to multi-tenant SaaS platforms.

At Paradane, we build production APIs for businesses — REST, GraphQL, and real-time. If you're planning an API project or need help hardening an existing one, we'd be happy to talk.

React Server Components in Production: What We Learned After Migrating a Client Dashboard

Paradane — Mon, 15 Jun 2026 15:07:44 +0000

When React Server Components (RSC) shipped with Next.js 14, the promise was compelling: ship less JavaScript to the browser, render more on the server, and get better performance for free. But moving a real production dashboard from the old Pages Router to the App Router with RSC wasn't free — it was a migration that taught us a lot about what the docs don't tell you.

At Paradane, we recently migrated a client's analytics dashboard — a data-heavy application with real-time charts, filtering, and multi-tenant data isolation — from Next.js 12 (Pages Router) to Next.js 14 with the App Router and React Server Components. Here's what we learned.

The Starting Point

The dashboard had about 15 pages, each pulling data from a PostgreSQL database through tRPC procedures. Every page was a client component by default — useEffect for data fetching, client-side state for filters, and heavy chart libraries (Recharts, D3) rendering entirely in the browser. Page loads were 3-4 seconds on a good connection, and the JavaScript bundle was pushing 400KB gzipped.

What We Expected

Faster initial page loads because data fetching happens on the server
Smaller client bundles because components that don't need interactivity stay on the server
Better SEO for the few public-facing report pages

What Actually Happened

1. The Server/Client Boundary Is the Hardest Part

The docs make it sound simple: add "use client" at the top of files that need interactivity. In practice, the boundary is where most bugs live. A server component imports a client component that imports a server component — and suddenly you have a hydration mismatch because the server rendered something the client can't reconcile.

What worked: We adopted a strict pattern — server components fetch data and pass it as props to client components. Client components never fetch data directly. This single rule eliminated 80% of our hydration issues.

// Server Component — fetches data, renders nothing interactive
import { db } from '@/lib/db';
import { DashboardClient } from './dashboard-client';

export default async function DashboardPage({ params }: { params: { tenantId: string } }) {
  const metrics = await db.query.metrics.findMany({
    where: { tenantId: params.tenantId },
  });
  return <DashboardClient metrics={metrics} />;
}

// Client Component — receives data, handles interactivity
'use client';

import { useState } from 'react';
import { Chart } from './chart';

export function DashboardClient({ metrics }: { metrics: Metric[] }) {
  const [filter, setFilter] = useState('7d');
  const filtered = metrics.filter(m => m.period === filter);
  return (
    <div>
      <FilterBar value={filter} onChange={setFilter} />
      <Chart data={filtered} />
    </div>
  );
}

2. Chart Libraries Are Still a Problem

Recharts and D3 need the DOM. You can't render charts on the server (well, you can, but they won't be interactive). Our solution: wrap every chart in a client component with dynamic(() => import(...), { ssr: false }). This adds a loading skeleton while the chart bundle loads, but the tradeoff is worth it — no hydration mismatches, and the rest of the page renders instantly on the server.

3. Data Fetching Got Simpler (Eventually)

Moving from tRPC to direct server-side database calls was the biggest win. No more API route boilerplate, no more client-side loading states for data that could have been fetched at request time. But we had to restructure our data access layer — tRPC procedures that assumed a client context (like session cookies) needed to be rewritten as server-side functions that accept the session directly.

4. Bundle Size Dropped, But Not as Much as Expected

We went from ~400KB gzipped to ~280KB. Good, but not the 50% reduction some blog posts promised. Why? Because interactive components (charts, filters, forms) still ship their code to the client. RSC helps most with content-heavy pages — dashboards with heavy interactivity see smaller gains.

5. Streaming Is the Real Performance Win

React Suspense with streaming changed the perceived performance dramatically. Instead of waiting for all data before showing anything, we could stream the page shell immediately and fill in data as it arrived:

export default function DashboardPage({ params }: { params: { tenantId: string } }) {
  return (
    <DashboardShell>
      <Suspense fallback={<MetricsSkeleton />}>
        <MetricsSection tenantId={params.tenantId} />
      </Suspense>
      <Suspense fallback={<ChartSkeleton />}>
        <ChartsSection tenantId={params.tenantId} />
      </Suspense>
    </DashboardShell>
  );
}

Users saw the dashboard shell in under 500ms, with data populating progressively. Time-to-interactive dropped from 3.2s to 1.1s.

The Numbers

Metric	Before (Pages Router)	After (App Router + RSC)
First Contentful Paint	2.1s	0.4s
Time to Interactive	3.2s	1.1s
JS Bundle (gzipped)	398KB	281KB
Lighthouse Performance	62	91
Build time	45s	68s

Build time increased — server components mean more work at build time. Not a dealbreaker, but worth knowing.

What We'd Do Differently Next Time

Start with the data flow, not the component tree. Map out which data each component needs and whether it can be fetched server-side before writing any code.
Use dynamic(() => import(...), { ssr: false }) for ALL third-party interactive libraries. Don't try to make them work with SSR — it's not worth the debugging time.
Embrace streaming early. Suspense boundaries aren't an afterthought — they're the architecture. Plan them before you write components.
Keep the server/client boundary rule simple. Server components fetch and pass data. Client components receive data and handle interactivity. No exceptions until you have a very good reason.

The Bottom Line

React Server Components are a real improvement, not just hype. The performance gains are measurable, and the programming model — once you internalize the server/client boundary — is cleaner than the old useEffect-for-everything pattern. But the migration isn't free, and the learning curve is steeper than the docs suggest.

If you're considering the move, start with a content-heavy page (blog, docs, marketing pages) where RSC shines brightest. Save the interactive dashboard migration for when you've built some intuition about the boundary.

At Paradane, we build and migrate web applications for businesses — from e-commerce platforms to SaaS dashboards. If you're planning a Next.js migration or building a new React application, we'd be happy to share what we've learned.

Debugging a Production Memory Leak: A Step-by-Step Walkthrough

Paradane — Mon, 15 Jun 2026 08:34:51 +0000

The 3 AM Alert

It started with a PagerDuty alert at 3 AM. Our client's Node.js API server had been restarted by the process manager — again. This was the third OOM (Out of Memory) kill this week, and the team was running out of patience.

The application was an Express.js API handling about 2,000 requests per minute. Nothing crazy. But every 6-8 hours, memory usage would climb past the 512MB container limit, and Docker would unceremoniously kill the process.

Here's exactly how we found the leak, fixed it, and what we learned along the way.

Step 1: Confirm It's Actually a Leak

First rule of memory debugging: make sure you're not just under-provisioned. A growing heap doesn't always mean a leak — it could be legitimate cache growth or increased traffic.

We checked the metrics:

# Container memory over 24 hours
docker stats --no-stream

The pattern was unmistakable: steady, linear growth with no plateau. A healthy application's memory should level off after warm-up. Ours never did.

// Quick and dirty heap size check we added to a health endpoint
app.get('/health/debug', (req, res) => {
  const used = process.memoryUsage();
  res.json({
    heapUsed: `${Math.round(used.heapUsed / 1024 / 1024)} MB`,
    heapTotal: `${Math.round(used.heapTotal / 1024 / 1024)} MB`,
    external: `${Math.round(used.external / 1024 / 1024)} MB`,
    rss: `${Math.round(used.rss / 1024 / 1024)} MB`,
  });
});

Hitting this endpoint every 30 seconds confirmed: heap used was climbing ~5MB per hour with no signs of garbage collection bringing it back down.

Step 2: Take a Heap Snapshot

Node.js has built-in heap snapshot capabilities via the v8 module and Chrome DevTools. We added a snapshot endpoint (behind auth, obviously — snapshots are expensive):

const v8 = require('v8');
const fs = require('fs');

app.get('/admin/heap-snapshot', async (req, res) => {
  const snapshot = v8.writeHeapSnapshot();
  const filename = snapshot;
  res.download(filename);
});

We took two snapshots 30 minutes apart, downloaded them, and loaded them into Chrome DevTools (Memory tab → Load).

Step 3: Compare Snapshots in Chrome DevTools

This is where the real debugging happens. In DevTools:

Load the first snapshot
Load the second snapshot
Switch to "Comparison" view
Sort by "Delta" (new objects) or "Size Delta"

The comparison view shows exactly what objects were allocated between the two snapshots and — crucially — what was NOT garbage collected.

Immediately, one thing jumped out: (closure) objects were dominating the delta. Thousands of closures, each holding references to request-scoped data.

Step 4: Trace the Closures

We drilled into the closure objects and followed the retaining paths. The DevTools "Retainers" view shows what's keeping each object alive. For our closures, the chain looked like:

closure → context → array → EventEmitter._events → ... → global

The EventEmitter._events was the smoking gun. Something was attaching event listeners that never got removed.

We searched the codebase for .on( and .addListener( calls. Found the culprit in our request logging middleware:

// THE BUG: middleware that attached a listener but never cleaned up
app.use((req, res, next) => {
  const requestId = uuid.v4();

  // This listener is attached per-request but never removed
  process.on('uncaughtException', (err) => {
    logger.error({ requestId, error: err.message });
  });

  res.on('finish', () => {
    logger.info({ requestId, status: res.statusCode });
  });

  next();
});

Every single request added a new uncaughtException listener to the global process object. Those listeners were closures capturing requestId and logger. After 100,000 requests, that's 100,000 listeners — each holding a closure with references that the GC couldn't touch because process is a global that never goes out of scope.

Step 5: The Fix

The fix was straightforward once we understood the problem:

// FIXED: use once() or manage listener lifecycle
app.use((req, res, next) => {
  const requestId = uuid.v4();

  // Option A: Use process.once() if you genuinely need per-request handling
  // (but you probably don't — uncaughtException handlers should be global)

  // Option B: Move the handler to app startup, pass context differently
  // This is what we actually did:

  res.on('finish', () => {
    logger.info({ requestId, status: res.statusCode });
  });

  next();
});

// Single global handler at startup — no per-request listeners
process.on('uncaughtException', (err) => {
  logger.error({ error: err.message, stack: err.stack });
});

We also added a safeguard: a periodic check that warns if the listener count grows unexpectedly:

setInterval(() => {
  const listenerCount = process.listenerCount('uncaughtException');
  if (listenerCount > 5) {
    logger.warn({ listenerCount }, 'Possible listener leak detected');
  }
}, 60000);

Step 6: Verify the Fix

After deploying, we watched the heap for 48 hours:

Before fix:  ~80MB → ~480MB over 8 hours (OOM kill)
After fix:   ~80MB → ~120MB over 48 hours (stable, GC working)

The heap stabilized around 120MB with normal GC sawtooth patterns. No more 3 AM alerts.

What We Learned

Heap snapshots are your best friend. The comparison view in Chrome DevTools makes leaks obvious. Don't guess — take snapshots.
Event listeners on global objects are dangerous. process, global, and long-lived EventEmitter instances will keep your closures alive forever. Always clean up with removeListener or use once().
Per-request resource allocation needs per-request cleanup. If you create something during a request, make sure it dies with the request. The res.on('finish') pattern is fine — res gets garbage collected. process does not.
Add listener count monitoring. A simple process.listenerCount() check in your health endpoint can catch leaks before they take down production.

At Paradane, we've debugged memory leaks across dozens of client applications — Node.js, Python, PHP, you name it. The pattern is almost always the same: something global holding references to things that should be ephemeral. If you're dealing with a production memory leak and need a second pair of eyes, our custom software development team has seen it all.

Tools We Use for Memory Profiling

Chrome DevTools (for Node.js heap snapshots) — free, built into Node
clinic.js — excellent for spotting event loop delays and memory issues
pm2 with --max-memory-restart — buys you time while you debug
Prometheus + Grafana — for long-term memory trend visualization

Have you debugged a production memory leak? What tools worked for you? Drop your war stories in the comments.

How We Reduced API Response Time by 80% for a Client — A Real-World Optimization Walkthrough

Paradane — Mon, 15 Jun 2026 05:02:45 +0000

The Problem: An API That Was Costing Real Money

A few months ago, a client came to us with a problem that was bleeding revenue. Their e-commerce platform's product search API was taking 2.3 seconds on average — and during peak traffic, it spiked to 4+ seconds. For an online store, every 100ms of delay costs conversions. At 2.3 seconds, they were losing customers before the page even loaded.

The API served product listings with inventory, pricing, and availability across multiple warehouses. It was built on Node.js + Express with PostgreSQL, and it had grown organically over two years without much performance attention. The code worked — it just worked slowly.

Here's exactly how we diagnosed the bottlenecks and brought response times down to under 400ms.

Step 1: Measure Before You Touch Anything

You can't optimize what you don't measure. We instrumented the API with OpenTelemetry tracing and added request-level timing logs. Within an hour of deploying instrumentation, the picture was clear:

Component	Avg Time	% of Total
Database queries	1,420ms	62%
External inventory API call	480ms	21%
Response serialization	210ms	9%
Auth/validation	120ms	5%
Other	70ms	3%

The database was the obvious villain. But the external inventory call was also a problem — it was synchronous and blocking.

Step 2: Fix the Database Queries

Digging into the PostgreSQL query logs, we found three issues:

N+1 Query Problem

The product listing endpoint was fetching products, then looping through each one to fetch its variants, then looping through variants to fetch inventory. Classic N+1. A page of 20 products was triggering 60+ queries.

Fix: We rewrote the data access layer to use JOINs and fetch everything in 2-3 queries:

-- Before: 1 product query + 20 variant queries + 40 inventory queries
-- After: 3 queries total
SELECT p.*, 
       json_agg(json_build_object('id', v.id, 'sku', v.sku, 'price', v.price)) as variants
FROM products p
LEFT JOIN variants v ON v.product_id = p.id
WHERE p.category_id = $1
GROUP BY p.id
LIMIT 20;

Missing Indexes

The WHERE clause was filtering on category_id and is_active, but neither column had an index. A sequential scan on 200K+ products was happening on every request.

Fix: Two composite indexes:

CREATE INDEX idx_products_category_active ON products(category_id, is_active) WHERE is_active = true;
CREATE INDEX idx_variants_product ON variants(product_id);

Unnecessary Data Fetching

The API was returning full product descriptions (sometimes 2KB+ of HTML) in the listing endpoint. Nobody reads descriptions in a search result.

Fix: We created a lightweight product_summary view that excluded heavy text fields and computed pre-aggregated values.

Result: Database time dropped from 1,420ms to ~180ms.

Step 3: Parallelize the External Call

The inventory availability check was calling a third-party warehouse API synchronously — the request sat idle waiting for a response that took 400-500ms.

Fix: We moved the inventory check to a background job using Redis-backed queues (BullMQ). The API now returns product listings immediately with a inventory_status: 'pending' flag. A separate WebSocket channel pushes real-time inventory updates to the frontend within 1-2 seconds.

This was a UX tradeoff — the product appears instantly, and inventory data fills in a moment later. The client's users preferred this dramatically over waiting 2+ seconds for a complete response.

Result: The 480ms blocking call was eliminated from the critical path.

Step 4: Response Serialization

We found that JSON.stringify() on large response objects was taking 200ms+. The culprit: the API was serializing full product objects with nested relations, many of which the frontend never used.

Fix: We implemented a response shaping layer using DTOs (Data Transfer Objects) that only included fields the frontend actually consumed. We also added response compression (gzip at the Express level), which cut payload size by 70%.

Result: Serialization dropped to ~40ms.

Step 5: Add Caching (But Carefully)

With the database queries optimized, we added a Redis cache layer for product listings. The key insight: cache at the right granularity.

We cached at the category + filter combination level with a 60-second TTL. This meant:

First request: ~200ms (database)
Subsequent requests within 60s: ~15ms (Redis)
Cache invalidation on product update via database triggers → Redis pub/sub

We deliberately kept the TTL short because inventory changes frequently. A longer TTL would have served stale data.

The Final Numbers

Metric	Before	After	Improvement
Avg response time	2,300ms	380ms	83.5%
p95 response time	4,100ms	620ms	84.9%
p99 response time	5,800ms	890ms	84.7%
DB queries per request	60+	3	95%
Payload size	48KB	14KB	71%

The client saw an immediate 12% increase in product page conversions and a 9% drop in search abandonment. The infrastructure cost actually went down because fewer database CPU cycles were burned on inefficient queries.

What We Learned

Instrument first. You can't fix what you can't see. OpenTelemetry + query logging gave us the map.
The database is usually the bottleneck. In our experience at Paradane, 7 out of 10 slow APIs are slow because of database access patterns, not application code.
N+1 queries are everywhere. ORMs make them easy to write and hard to spot. Always check your query logs.
Synchronous external calls are API killers. If you're calling a third-party service in the request path, you're borrowing their latency.
Cache with intent. Don't just slap Redis on everything. Think about TTL, invalidation, and what staleness means for your users.

Is Your API Slow?

If your API response times are creeping up, start with instrumentation. The fixes are usually simpler than you think — indexes, query consolidation, and removing blocking I/O solve most problems. And if you need help diagnosing or optimizing your stack, we do this kind of work at Paradane every day.

Have you tackled a similar optimization? What was your biggest win? Drop a comment below.