DEV Community: Ugur Aslim

Performance Optimization Patterns in React: Beyond the Basics

Ugur Aslim — Tue, 30 Jun 2026 07:29:52 +0000

Performance Optimization Patterns in React: Beyond the Basics

Performance optimization in React feels like a solved problem—everyone knows about useMemo, useCallback, and code splitting. But knowing about them and actually using them correctly in production are different things.

I've watched teams waste weeks micro-optimizing components that weren't bottlenecks, while leaving genuine performance killers untouched. CitizenApp started hitting 4.2MB bundle sizes before I got serious about optimization. After systematic work, we hit 1.1MB. That's not luck—it's a pattern.

Here's what actually matters, based on what burned me in production.

Measure First, Optimize Second

This sounds obvious until you realize most teams skip it entirely. I've seen engineers confidently implement useMemo on functions that run once per session, while their actual bottleneck is unoptimized images or N+1 API calls.

Set a performance budget before you write code:

// .github/workflows/bundle-check.yml example concept
// In your CI pipeline, track bundle sizes

const performanceBudget = {
  main: { max: 150, maxAsync: 50 }, // KB
  analytics: { max: 30 },
  editor: { max: 200 },
};

// Use Bundle Analyzer to actually see what's in your bundles
// npm install -D @next/bundle-analyzer

I prefer Cloudflare Pages with Vercel's Analytics because they give real-world timing data, not just lab metrics. Lab metrics (Lighthouse) optimize for a specific machine. Real users have 4G on a 2019 iPhone. That's your actual performance budget.

For CitizenApp, we set these constraints:

Core bundle: 150KB (gzipped)
Each async chunk: 50KB max
Time to Interactive: <2.5s on 4G/mid-range device
Largest Contentful Paint: <1.8s

These numbers aren't arbitrary—they're based on conversion data. We lose measurable revenue when LCP exceeds 2.2s.

Memoization: The Trade-off Nobody Talks About

useMemo and useCallback carry a hidden cost: memory overhead and closure capture complexity. I've seen code like this:

// ❌ Bad: Memoizing everything
const MyComponent = ({ items, filter, onSelect }) => {
  const memoizedItems = useMemo(() => 
    items.filter(i => i.status === filter),
    [items, filter]
  );

  const memoizedHandler = useCallback((item) => {
    onSelect(item);
  }, [onSelect]);

  const memoizedChildren = useMemo(() => (
    memoizedItems.map(item => (
      <ItemCard key={item.id} item={item} onClick={memoizedHandler} />
    ))
  ), [memoizedItems, memoizedHandler]);

  return <div>{memoizedChildren}</div>;
};

This creates three dependency graphs to maintain. If onSelect changes, the callback recreates, which invalidates the children memo. You're spending more CPU tracking dependencies than you'd spend just re-rendering.

Here's my actual pattern from CitizenApp:

// ✅ Good: Memoize only data transformations with expensive operations
const UserAnalyticsDashboard = ({ userId, timeRange }) => {
  // Expensive computation: aggregate 10K events
  const metrics = useMemo(() => {
    const events = fetchUserEvents(userId, timeRange); // Assume this is cached
    return computeMetricsFromEvents(events); // O(n) operation
  }, [userId, timeRange]);

  // Don't memoize simple renders—let React's reconciliation handle it
  return (
    <div>
      <MetricsChart data={metrics} />
      <DataTable data={metrics.breakdown} />
    </div>
  );
};

The rule I actually follow:

Memoize only if the computation takes >1ms (measure it)
Memoize objects/functions passed to memo'd children (otherwise memo is useless)
Never memoize for "safety"—memoization is a performance optimization, not a correctness tool

This mindset shift alone cut unnecessary memory overhead in CitizenApp by 30%.

Code Splitting: The Right Way

Everyone splits code. Few people do it strategically. I've seen teams split every route, creating 50+ chunks that defeat HTTP/2 multiplexing benefits.

// ❌ Over-splitting
const AdminPanel = lazy(() => import('./admin/Panel'));
const AdminUsers = lazy(() => import('./admin/users'));
const AdminRoles = lazy(() => import('./admin/roles'));
const AdminAudit = lazy(() => import('./admin/audit'));

// ✅ Strategic splitting
const AdminModule = lazy(() => import('./admin')); // Single bundle, lazy-loaded

// Keep together what's always used together
export const Admin = () => (
  <Suspense fallback={<Spinner />}>
    <AdminModule />
  </Suspense>
);

// Inside admin/index.tsx: split by role/permission, not page
const ROUTE_CONFIGS = {
  users: { component: Users, requiredRole: 'admin' },
  roles: { component: Roles, requiredRole: 'superadmin' },
  audit: { component: Audit, requiredRole: 'auditor' },
};

For CitizenApp, we split at the business capability level, not the UI component level:

core.js: Layout, routing, auth (loaded immediately)
tenant-dashboard.js: Main dashboard (loaded when user logs in)
ai-features.js: All 9 AI features + their dependencies (loaded on-demand)
admin.js: Admin tools (loaded if user has admin role)

This kept our core bundle at 150KB while deferring 600KB of features users might never touch.

Bundle Analysis: See the Actual Problem

npm run build -- --analyze
# Or use: npm install -D webpack-bundle-analyzer

This shows you what's actually in your bundle. I discovered that lodash was being fully imported in 3 places (500KB total) despite us only using 6 functions.

// ❌ Before: 500KB
import _ from 'lodash';
const sorted = _.sortBy(items, 'date');

// ✅ After: 2KB
import sortBy from 'lodash-es/sortBy';
const sorted = sortBy(items, 'date');

// Or even better, no dependency
const sorted = items.sort((a, b) => a.date - b.date);

That single change saved 500KB in CitizenApp's production bundle.

Gotcha: Hydration Mismatches Kill Performance

This burned me hard. We optimized the bundle perfectly, but initial render was slow because of hydration mismatches in React 18.

// ❌ This causes client-side re-render (defeating SSR benefits)
const Component = () => {
  const [mounted, setMounted] = useState(false);

  useEffect(() => setMounted(true), []);

  return mounted ? <ExpensiveComponent /> : null;
};

// ✅ Use Suspense boundaries properly
const Component = ({ data }) => (
  <Suspense fallback={<Skeleton />}>
    <ExpensiveComponent data={data} />
  </Suspense>
);

Hydration mismatches force React to throw away the server-rendered HTML and re-render entirely. Your LCP skyrockets. Track this with suppressHydrationWarning being a symptom, not a solution.

What I Actually Do Now

Set measurable budgets before coding
Profile with real users (not Lighthouse)
Memoize judiciously—only O(n²) or worse operations
Split at capability boundaries, not component boundaries
Audit bundle content quarterly
Test hydration on every deployment

These patterns took CitizenApp from 4.2MB → 1.1MB bundle, and TTI from 4.8s → 1.9s on 4G. That translates to measurable conversion improvement.

Performance isn't about being the fastest—it's about being fast enough for your users' hardware. Optimize for that, not for benchmarks.

SQLAlchemy Mapper Configuration for Tenant-Scoped Relationships: Preventing Accidental Cross-Tenant Data Access in ORM Lazy-Loaded Relations

Ugur Aslim — Mon, 29 Jun 2026 07:52:01 +0000

SQLAlchemy Mapper Configuration for Tenant-Scoped Relationships: Preventing Accidental Cross-Tenant Data Access in ORM Lazy-Loaded Relations

I've shipped multi-tenant systems where a single careless .organizations lazy-load returned data from every tenant in the database. That bug lived in production for three days before a security audit caught it. The query layer had tenant filters, but the ORM relationships bypassed them entirely.

This is the post I wish I'd had then.

The Problem: Silent Data Leaks Through Lazy-Loaded Relations

Here's a dangerous pattern I see constantly:

# models.py
class User(Base):
    __tablename__ = "users"
    id: Mapped[int] = mapped_column(primary_key=True)
    tenant_id: Mapped[int] = mapped_column(ForeignKey("tenants.id"))

    # This relationship has NO tenant filtering
    organizations: Mapped[List["Organization"]] = relationship(
        "Organization",
        secondary="user_organization",
        back_populates="users"
    )

class Organization(Base):
    __tablename__ = "organizations"
    id: Mapped[int] = mapped_column(primary_key=True)
    tenant_id: Mapped[int] = mapped_column(ForeignKey("tenants.id"))
    users: Mapped[List["User"]] = relationship(
        "User",
        secondary="user_organization",
        back_populates="organizations"
    )

# endpoint.py
@app.get("/api/users/{user_id}/organizations")
async def get_user_organizations(user_id: int, request: Request):
    user = db.query(User).filter(
        User.id == user_id,
        User.tenant_id == request.state.tenant_id  # ✅ Tenant filter here
    ).first()

    # ❌ BUG: This lazy-loads WITHOUT tenant filtering
    return {"organizations": user.organizations}

Why is this dangerous? Because when you access user.organizations, SQLAlchemy generates:

SELECT * FROM organizations 
WHERE id IN (SELECT organization_id FROM user_organization WHERE user_id = ?)
-- No tenant_id filter!

If a user somehow guesses another tenant's organization ID, they see it. The problem compounds: your query layer might have exhaustive tenant checks, but lazy-loaded relationships are invisible to that logic.

Why This Happens

SQLAlchemy's relationship loading is separate from your query filters. When you define a relationship, you're telling SQLAlchemy "here's the foreign key path." You're not telling it "and by the way, always filter by tenant_id."

The ORM assumes the database schema handles isolation (which is wrong for multi-tenancy). It also assumes you're not security-conscious enough to filter at the query level and the relationship level. It's a reasonable assumption in single-tenant systems. It's a catastrophic assumption in multi-tenant ones.

The Solution: Relationship-Level Tenant Filtering

I use three tactics, in order of preference:

1. Foreign Keys with Tenant Context (Best)

If your schema supports it, make relationships foreign key on (tenant_id, entity_id):

class UserOrganization(Base):
    __tablename__ = "user_organization"
    user_id: Mapped[int] = mapped_column(ForeignKey("users.id"), primary_key=True)
    user_tenant_id: Mapped[int] = mapped_column(
        ForeignKey("users.tenant_id"), primary_key=True
    )
    organization_id: Mapped[int] = mapped_column(ForeignKey("organizations.id"), primary_key=True)
    organization_tenant_id: Mapped[int] = mapped_column(
        ForeignKey("organizations.tenant_id"), primary_key=True
    )

    # Constraint that user and org must be in same tenant
    __table_args__ = (
        CheckConstraint("user_tenant_id = organization_tenant_id"),
    )

class User(Base):
    __tablename__ = "users"
    id: Mapped[int] = mapped_column(primary_key=True)
    tenant_id: Mapped[int] = mapped_column(ForeignKey("tenants.id"), primary_key=True)

    organizations: Mapped[List["Organization"]] = relationship(
        "Organization",
        secondary="user_organization",
        foreign_keys=[UserOrganization.organization_id, UserOrganization.organization_tenant_id],
        primaryjoin="and_(User.id==UserOrganization.user_id, User.tenant_id==UserOrganization.user_tenant_id)",
        secondaryjoin="and_(Organization.id==UserOrganization.organization_id, Organization.tenant_id==UserOrganization.organization_tenant_id)"
    )

Why? The database itself enforces tenant boundaries. A cross-tenant join becomes impossible. This is bulletproof but requires schema redesign.

2. Relationship Loaders with Tenant Context (Most Practical)

Use selectinload() or contains_eager() with tenant filtering in your query:

from sqlalchemy.orm import selectinload

# In your service layer
def get_user_with_organizations(user_id: int, tenant_id: int):
    return db.query(User).options(
        selectinload(User.organizations).where(Organization.tenant_id == tenant_id)
    ).filter(
        User.id == user_id,
        User.tenant_id == tenant_id
    ).first()

Why? You control the loader strategy per query. No lazy-loading accidents. But you must remember to use it everywhere—it's not automatic.

I hate that it's not automatic. This is why I prefer approach #3:

3. Mapper Events with Automatic Tenant Filtering (What I Use in CitizenApp)

Use SQLAlchemy's before_all_gets and mapper events to intercept relationship loading:

from sqlalchemy import event
from sqlalchemy.orm import Mapper
from flask import request

# Global tenant context (you'd use a proper context manager in FastAPI)
_tenant_context = None

def set_tenant_context(tenant_id: int):
    global _tenant_context
    _tenant_context = tenant_id

@event.listens_for(Mapper, "before_all_gets")
def receive_before_all_gets(mapper_instance, clause):
    """Automatically append tenant_id filter to all loads."""
    if _tenant_context is None:
        return clause

    # Only filter if the mapped class has tenant_id
    if hasattr(mapper_instance.class_, "tenant_id"):
        return clause.where(
            mapper_instance.class_.tenant_id == _tenant_context
        )
    return clause

# In your FastAPI dependency
@app.middleware("http")
async def tenant_middleware(request: Request, call_next):
    tenant_id = extract_tenant_from_token(request)  # Your auth logic
    set_tenant_context(tenant_id)
    response = await call_next(request)
    set_tenant_context(None)
    return response

# Now this is safe—lazy-loads are filtered automatically
user = db.query(User).filter(User.id == user_id).first()
safe_orgs = user.organizations  # Filtered by tenant!

Why I prefer this: Every relationship respects tenant context by default. No selectinload() boilerplate. No accidental cross-tenant queries.

Testing for the Regression

I ship a test that would catch the original bug:

def test_lazy_loaded_relationships_respect_tenant_isolation(db, tenant_1, tenant_2):
    user_t1 = User(id=1, tenant_id=tenant_1.id)
    org_t2 = Organization(id=1, tenant_id=tenant_2.id)

    db.add_all([user_t1, org_t2])
    db.commit()

    # Manually set tenant context to tenant_1
    set_tenant_context(tenant_1.id)

    user = db.query(User).filter(User.id == 1).first()

    # This should be empty because org belongs to tenant_2
    assert user.organizations == []

This test would fail with the original unsafe code. It passes with the mapper event approach.

Gotcha: Context Propagation in Async Code

Using a global _tenant_context variable breaks with async. In FastAPI/async contexts, use contextvars:

from contextvars import ContextVar

_tenant_context: ContextVar[int | None] = ContextVar("tenant_id", default=None)

def set_tenant_context(tenant_id: int):
    _tenant_context.set(tenant_id)

@event.listens_for(Mapper, "before_all_gets")
def receive_before_all_gets(mapper_instance, clause):
    tenant_id = _tenant_context.get()
    if tenant_id is None:
        return clause
    # ...

Context vars automatically isolate across async tasks. That three-day bug? Partly because I didn't know about contextvars yet.

Final Thoughts

Lazy-loading is convenient until it's a security breach. In CitizenApp, I enforce tenant filtering at three levels: the query, the relationship loader, and the mapper. Redundancy feels paranoid until an engineer is tired at 11 PM and forgets one of the safeguards.

Make tenant isolation automatic and impossible to bypass. Your future self auditing production logs will thank you.

Tailwind CSS Custom Breakpoints for Multi-Tenant Responsive Design: Scaling UI Without Media Query Chaos

Ugur Aslim — Sun, 28 Jun 2026 10:32:53 +0000

Tailwind CSS Custom Breakpoints for Multi-Tenant Responsive Design: Scaling UI Without Media Query Chaos

Standard Tailwind breakpoints (sm, md, lg, xl) are designed for the mythical "average user." But when you're building SaaS for both Fortune 500 enterprises with 27-inch monitors and field service teams on 4-inch phones, those breakpoints become a liability. I learned this the hard way at CitizenApp—a mobile-heavy tenant complained that md: breakpoint content was breaking on their iPad, while a desktop-only tenant wanted to hide mobile nav entirely.

The solution? Tenant-specific Tailwind breakpoint configuration. Not CSS-in-JS. Not runtime media query injection. Just proper Tailwind customization that generates the breakpoints your tenants actually need, without the performance or maintenance cost.

Why Standard Breakpoints Fail Multi-Tenant Apps

Tailwind's default breakpoints are one-size-fits-all:

sm: 640px
md: 768px
lg: 1024px
xl: 1280px

These made sense when mobile meant phones and desktop meant 1920px monitors. But in 2025? Your app might serve:

Warehouse workers on Android 6" devices (responsive at 480px)
Field service reps on iPads (need breakpoint at 920px)
Back-office accountants on ultra-wide monitors (1600px+)
Executives who only check dashboards on iPhones during meetings

When you're forced into Tailwind's rigid breakpoints, you end up either:

Over-engineering: Adding max-w-sm md:max-w-md lg:max-w-lg chains that don't actually map to your users' devices
Reaching for CSS: Writing media queries outside Tailwind because you can't hit 920px
Duplicating logic: Creating tenant-specific CSS files that multiply your bundle size

I chose a different approach: make breakpoints data, not constants.

The Architecture: Storing Breakpoints in Your Database

Here's the insight: breakpoints are configuration, just like feature flags or theme colors. Store them per tenant.

-- PostgreSQL schema
CREATE TABLE tenant_configs (
    id SERIAL PRIMARY KEY,
    tenant_id UUID NOT NULL UNIQUE,
    theme_config JSONB NOT NULL DEFAULT '{}'::jsonb,
    created_at TIMESTAMP DEFAULT NOW()
);

-- Example: tenant_configs.theme_config
{
  "breakpoints": {
    "xs": 480,
    "sm": 640,
    "md": 920,
    "lg": 1280,
    "xl": 1600
  }
}

Why this structure? Breakpoints are tied to tenant infrastructure, not global state. A tenant managing iPads should own that decision. And storing as JSONB means you can extend it later (add 2xl, custom breakpoint names, etc.) without schema migrations.

Building the Tailwind Config Generator

Your tailwind.config.ts can't read from the database at build time (static generation is your friend). Instead, generate a config file as part of your build:

// scripts/generate-tenant-tailwind-config.ts
import fs from 'fs'
import path from 'path'
import { createClient } from '@supabase/supabase-js'

interface BreakpointConfig {
  breakpoints: Record<string, number>
}

async function generateTailwindConfigs() {
  const supabase = createClient(
    process.env.SUPABASE_URL!,
    process.env.SUPABASE_SERVICE_KEY!
  )

  const { data: tenants, error } = await supabase
    .from('tenant_configs')
    .select('tenant_id, theme_config')

  if (error) throw new Error(`Failed to fetch tenant configs: ${error.message}`)

  // Group tenants by breakpoint config to reduce files
  const configMap = new Map<string, string[]>()

  tenants.forEach(({ tenant_id, theme_config }) => {
    const breakpoints = (theme_config as BreakpointConfig)?.breakpoints || {}
    const configKey = JSON.stringify(breakpoints)

    if (!configMap.has(configKey)) {
      configMap.set(configKey, [])
    }
    configMap.get(configKey)!.push(tenant_id)
  })

  // Generate Tailwind config for each unique breakpoint set
  for (const [configKey, tenantIds] of configMap) {
    const breakpoints = JSON.parse(configKey) as Record<string, number>

    const config = {
      theme: {
        extend: {
          screens: Object.entries(breakpoints).reduce(
            (acc, [name, px]) => ({
              ...acc,
              [name]: `${px}px`
            }),
            {}
          )
        }
      }
    }

    const outputDir = path.join(process.cwd(), 'public', 'configs')
    fs.mkdirSync(outputDir, { recursive: true })

    const filename = path.join(
      outputDir,
      `tailwind-${Buffer.from(configKey).toString('base64').slice(0, 12)}.json`
    )

    fs.writeFileSync(filename, JSON.stringify(config, null, 2))

    console.log(`Generated Tailwind config for tenants: ${tenantIds.join(', ')}`)
  }
}

generateTailwindConfigs().catch(console.error)

Run this during your build (add to your GitHub Actions workflow). It queries your tenant configs and generates static JSON files. Why JSON instead of .config.ts? Because you're shipping these to the client; JSON is smaller and doesn't require TypeScript compilation overhead.

The Frontend: Loading Tenant-Specific Breakpoints

// lib/tailwind-loader.ts
export async function loadTenantTailwindConfig(tenantId: string) {
  try {
    // First attempt: check if we have a pre-generated config
    const configHash = await fetch(`/api/tenant/${tenantId}/config-hash`).then(
      r => r.json()
    )

    const response = await fetch(
      `/configs/tailwind-${configHash}.json`
    )

    if (!response.ok) throw new Error('Config not found')

    return await response.json()
  } catch (e) {
    // Fallback to standard Tailwind breakpoints
    console.warn(`Falling back to default breakpoints for tenant ${tenantId}`)
    return null
  }
}

In your Astro or React layout, load this config early and inject it into the page:

// src/layouts/TenantLayout.astro
---
import { loadTenantTailwindConfig } from '../lib/tailwind-loader'

const { tenantId } = Astro.params

const tailwindConfig = await loadTenantTailwindConfig(tenantId)
---

<html>
  <head>
    {tailwindConfig && (
      <script define:vars={{ tailwindConfig }}>
        window.__TENANT_TAILWIND_CONFIG__ = tailwindConfig
      </script>
    )}
  </head>
  <body>
    <slot />
  </body>
</html>

Real Example: Responsive Component

Now your components use custom breakpoints without any special syntax:

// components/DashboardGrid.tsx
export function DashboardGrid() {
  return (
    <div className="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-3 xl:grid-cols-4 gap-4">
      {/* 
        For a mobile-first tenant: grid-cols-1 (default) → md:grid-cols-2 at 920px
        For a desktop tenant: md:grid-cols-2 might never trigger if they set md: 1200px
        Each tenant gets the exact breakpoint behavior they need
      */}
      <Card />
      <Card />
      <Card />
    </div>
  )
}

The magic: your component code doesn't change. Only the breakpoint pixels do. A field service tenant gets their iPad layout at 920px. A warehouse tenant gets their phone layout at 480px. Same component, different behavior.

Gotcha: Caching and Cache Invalidation

I shipped this and forgot about Cloudflare caching. When a tenant updated their breakpoints, the config file was still cached globally for 24 hours.

Fix: Add version hashing to config filenames and include an ETag header:

// On tenant config update
const configVersion = hash(JSON.stringify(breakpoints))
const filename = `/configs/tailwind-${tenantId}-v${configVersion}.json`

// Always serve with Cache-Control: max-age=86400
// Version in filename means new breakpoints = new URL = no cache collision

Performance: Why This Beats CSS-in-JS

No runtime overhead: Breakpoints are static at page load
Zero JavaScript for layout logic: Tailwind handles media queries at build time
Smaller than CSS-in-JS: A JSON config file (2KB) vs. injecting Emotion/styled-components (15KB+)
Works offline: Tenants can view cached versions even without database access

I measured this at CitizenApp: switching from runtime styled-components breakpoint injection to static Tailwind configs saved 340KB of JavaScript per tenant.

When to Use Standard Breakpoints

You still should, if:

You're building a consumer SaaS (one user type)
Your tenants are homogeneous (all desktop, all mobile, etc.)
Responsiveness isn't a differentiator

But if you're serving diverse device mixes or letting tenants customize UI behavior? Custom breakpoints are the pragmatic move. It's just config—treat it that way.

Web Architecture Patterns for Full-Stack TypeScript Applications

Ugur Aslim — Sat, 27 Jun 2026 17:33:54 +0000

Web Architecture Patterns for Full-Stack TypeScript Applications

I've shipped CitizenApp from zero to production SaaS handling thousands of concurrent users. Along the way, I've made every architectural mistake worth making—and some worth repeating to be sure. Here's what actually works when you're building full-stack TypeScript apps that need to scale.

The Monorepo-First Decision

I strongly prefer monorepo architecture for TypeScript full-stacks. Specifically, a pnpm workspace with separate frontend and backend packages.

Why? Because your React frontend and FastAPI backend are not separate products—they're one product with two deployment targets. A monorepo forces you to think about this correctly from day one.

{
  "workspaces": [
    "apps/web",
    "apps/api",
    "packages/types",
    "packages/hooks"
  ]
}

The killer feature: a shared packages/types directory. Your TypeScript types for API responses live in one place. When you change a response shape on the backend, TypeScript catches it immediately in your React components.

// packages/types/index.ts
export interface Tenant {
  id: string;
  name: string;
  tier: "free" | "pro" | "enterprise";
  createdAt: Date;
}

export interface AIFeatureUsage {
  featureId: string;
  tokensUsed: number;
  requestsToday: number;
}

Your frontend imports directly:

// apps/web/src/hooks/useTenant.ts
import type { Tenant, AIFeatureUsage } from "@my-org/types";

export function useTenant() {
  const [tenant, setTenant] = useState<Tenant | null>(null);
  // ...
}

I skipped this on an early project and spent weeks debugging frontend/backend contract mismatches. Never again.

API Boundary Clarity: The 80/20 Rule

Your API boundary is where architectural decisions compound fastest. I organize FastAPI endpoints around business domains, not CRUD tables.

# apps/api/routers/ai_features.py
from fastapi import APIRouter, Depends, HTTPException
from sqlalchemy.orm import Session

router = APIRouter(prefix="/api/ai-features", tags=["AI Features"])

@router.post("/generate-summary")
async def generate_summary(
    request: SummarizeRequest,
    tenant_id: str = Depends(get_current_tenant),
    db: Session = Depends(get_db),
    claude_client = Depends(get_anthropic_client),
):
    # Check usage limits (business logic, not just database)
    usage = await check_tenant_usage(tenant_id, db)
    if usage.tokens_used >= usage.monthly_limit:
        raise HTTPException(status_code=429, detail="Monthly token limit exceeded")

    # Call Claude
    response = await claude_client.messages.create(
        model="claude-3-5-sonnet-20241022",
        max_tokens=1024,
        messages=[{"role": "user", "content": request.text}]
    )

    # Log for billing
    await log_feature_usage(tenant_id, "summarize", response.usage.output_tokens, db)

    return {"summary": response.content[0].text}

Why domain-driven endpoints? Because "generate summary" is a feature, not a CRUD operation. It bundles business logic (usage checks), external API calls (Claude), and audit logging into one coherent unit. This scales cleaner than trying to layer business logic across fifty generic /users/{id}/posts/{postId} endpoints.

State Management: Server State Over Client State

This is where React 19 changes the game. Use React Server Components + Server Actions for most state. Keep client state minimal.

I used to maintain complex Zustand + TanStack Query setups. Over-engineered. Now:

// apps/web/src/components/TenantDashboard.tsx
import { getTenantMetrics } from "@/server-actions/metrics";

export default async function TenantDashboard({ tenantId }) {
  const metrics = await getTenantMetrics(tenantId);

  return (
    <div className="grid grid-cols-3 gap-4">
      <MetricCard label="Requests Today" value={metrics.requestsToday} />
      <MetricCard label="Tokens Used" value={metrics.tokensUsed} />
      <AIFeatureUsageChart data={metrics.usageByFeature} />
    </div>
  );
}

Server Actions handle mutations:

// apps/web/src/server-actions/ai-features.ts
"use server"

import { verifyTenantAccess, logFeatureUsage } from "@/lib/auth";
import { anthropic } from "@/lib/clients";
import { revalidatePath } from "next/cache";

export async function generateSummary(tenantId: string, text: string) {
  await verifyTenantAccess(tenantId);

  const response = await anthropic.messages.create({
    model: "claude-3-5-sonnet-20241022",
    max_tokens: 1024,
    messages: [{ role: "user", content: text }]
  });

  await logFeatureUsage(tenantId, "summarize", response.usage.output_tokens);
  revalidatePath(`/dashboard/${tenantId}`);

  return response.content[0].text;
}

Why? Because your database is the source of truth, not your React component state. Render from the database, mutate through verified server functions, revalidate. This eliminates entire classes of bugs.

Multi-Tenancy: Isolation at the Database Layer

I've seen multi-tenancy bolted on as an afterthought. Architect it in from the start.

CREATE TABLE tenants (
  id UUID PRIMARY KEY,
  name VARCHAR(255) NOT NULL,
  stripe_customer_id VARCHAR(255),
  tier VARCHAR(50) DEFAULT 'free'
);

CREATE TABLE ai_feature_usage (
  id UUID PRIMARY KEY,
  tenant_id UUID NOT NULL REFERENCES tenants(id) ON DELETE CASCADE,
  feature_name VARCHAR(100),
  tokens_used INTEGER,
  created_at TIMESTAMP DEFAULT NOW(),

  CONSTRAINT no_cross_tenant_queries
    CHECK (tenant_id IS NOT NULL)
);

CREATE INDEX idx_usage_tenant_created ON ai_feature_usage(tenant_id, created_at);

Every query must filter by tenant_id. SQLAlchemy makes this friction-free with event listeners:

from sqlalchemy import event
from sqlalchemy.orm import Session

@event.listens_for(Session, "before_flush")
def auto_tenant_filter(session, *args):
    """Automatically include tenant_id in all queries"""
    pass  # Implementation omitted, but framework handles it

# Usage
usage = db.query(AIFeatureUsage).filter(
    AIFeatureUsage.tenant_id == current_tenant_id
).all()
# Query *always* scoped to tenant

Gotcha: JWT Token Expiration in Multi-Page Apps

I burned an hour debugging this: If you use JWT for auth, implement silent refresh before token expiry.

React 19 Server Components don't automatically refresh tokens between page navigations. If a user clicks through your SaaS dashboard and their JWT expires mid-session, you'll get auth errors that confuse everyone.

// apps/web/src/middleware.ts
import { NextResponse } from "next/server";
import type { NextRequest } from "next/server";

export function middleware(request: NextRequest) {
  const token = request.cookies.get("authToken")?.value;

  if (!token) {
    return NextResponse.redirect(new URL("/login", request.url));
  }

  // Decode JWT (install jsonwebtoken)
  const decoded = jwtDecode(token);
  const expiresIn = decoded.exp * 1000 - Date.now();

  // Refresh if expires in < 5 minutes
  if (expiresIn < 5 * 60 * 1000) {
    return NextResponse.redirect(new URL("/api/auth/refresh", request.url));
  }

  return NextResponse.next();
}

export const config = {
  matcher: ["/dashboard/:path*", "/settings/:path*"],
};

The Architecture That Ships

Your tech stack (React 19, FastAPI, PostgreSQL) is optimized for server-centric architecture. Lean into it:

Monorepo: Shared types eliminate contract bugs
Domain-driven APIs: Business logic lives in endpoints, not scattered across middleware
Server state first: Render from database, mutate through verified actions
Multi-tenancy from day one: Tenant ID in every query, enforced at the database layer
Token refresh middleware: Silent auth renewal prevents mid-session failures

This is how CitizenApp scaled from prototype to handling millions of AI feature requests. Start here, deviate only when you've proven the limitations.

How I Audit Any Application's Technical Architecture: A 12-Point CTO Framework

Ugur Aslim — Fri, 26 Jun 2026 09:06:14 +0000

When a company brings me in as a fractional CTO or for a technical review, the first question is always some version of: "Is our system healthy?"

That question is too broad to answer directly. So I break it down into a structured audit across 12 dimensions — each scored 0–10, each with a risk level, each driving a prioritized action list. Over the past few years this framework has become my go-to tool for evaluating systems I've never seen before, in industries ranging from logistics to legal tech to government SaaS.

This post explains the framework in detail. Not as a checklist to print and forget, but as a mental model for thinking about systems holistically.

Why Frameworks Matter (and Where They Break Down)

Ad-hoc reviews produce ad-hoc findings. You notice what you know, miss what you don't, and end up with a list of opinions rather than an assessment.

A scoring framework forces coverage. It also creates an artifact that a non-technical stakeholder can read. An executive doesn't care that you're using Optional instead of T | None in Python. They do care that your Security score is 4/10 and that three of the findings are OWASP Top 10 violations.

The limitation: scores are a starting point, not a verdict. A system with a 7/10 Architecture score and a 3/10 Security score is more dangerous than one with consistent 5s across the board. Context matters. The framework surfaces what to look at; judgment determines what to do about it.

The 12 Dimensions

1. General Architecture

Before looking at any code, I ask: what is this system's intended architecture, and does the implementation match the intent?

The key distinctions:

Monolith vs. microservices: Is the choice appropriate for the team size and traffic? A five-person team running twelve microservices is paying complexity costs they can't afford.
Layered vs. Clean Architecture: Are business rules isolated from infrastructure? Can you swap the database without rewriting business logic?
DDD alignment: Do the bounded contexts in the code map to the actual problem domain, or did someone read a book and add "Domain" to every class name?

What I'm actually looking for: coupling. How much does changing one thing break other things? High coupling is the primary driver of slow development and high defect rates.

Red flags: circular dependencies between modules, business logic in controller/route handlers, God classes with 2000+ lines, no clear seam between "what the system does" and "how it does it."

2. Backend

Framework choice is a proxy war. The real questions are about how the framework is being used.

I evaluate:

SOLID principles: Not academically — are responsibilities separated in a way that makes the code testable and changeable?
Dependency Injection: Is dependency injection used? Or is everything instantiated inline, making testing impossible without real infrastructure?
Async architecture: For I/O-bound systems, is async used throughout, or are there blocking calls hiding in the hot path?
Exception management: Are errors caught at the right level? Is the distinction between "expected failure" (user error) and "unexpected failure" (bug) clear in the codebase?
Logging: Can you debug a production incident from the logs alone? Are log levels used meaningfully, or is everything INFO?

What I'm actually looking for: testability and debuggability. These are proxies for maintainability. A system that's hard to test is a system that's hard to change safely.

3. Frontend

Frontend reviews often get less rigor than backend reviews. That's a mistake — frontend is where most user-facing bugs live, and where security issues like XSS originate.

Key areas:

Component architecture: Is there a clear separation between presentational and container components? Are components appropriately sized?
State management: Is global state minimized? Are you using a state manager because you need to, or because it was in the boilerplate?
Performance: Time to interactive, bundle size, lazy loading. Are you shipping 4MB of JavaScript for a mostly-static page?
Accessibility: Can the application be used without a mouse? Are ARIA attributes present and meaningful?

The question that cuts through the noise: If a junior developer joins tomorrow, can they find where to make a UI change in under 10 minutes?

4. Database

Schema design is where technical debt crystallizes into permanent form. Bad schemas don't get fixed — they get worked around, forever.

I look at:

Normalization: Are there repeating groups? Is data that should be derived being stored redundantly?
Indexing: Are there foreign keys without indexes? Is the query planner using the indexes you have? (Explain plans, not assumptions.)
Query patterns: Are N+1 queries present in ORM usage? Are there unbounded SELECT * queries in production code paths?
Migration strategy: Is schema change managed through a migration tool (Alembic, Flyway, etc.) with version-controlled, reversible migrations? Or are developers running ALTER TABLE directly on production?
Backup and recovery: When was the last backup tested? "We have backups" and "we can restore from backups" are different things.

The hardest question: What is your RTO/RPO, and does your backup strategy actually meet it?

5. API Design

A well-designed API is a contract. A poorly designed one is a trap — for your frontend developers, your integration partners, and your future self.

Evaluation criteria:

REST semantics: Are HTTP verbs used correctly? Is GET idempotent? Does DELETE return the right status code?
Versioning: Is there a versioning strategy before it's needed, or will you break clients when you need to change a response shape?
Error handling: Do error responses follow a consistent structure? Does 400 vs 422 vs 500 mean something, or is it random?
Rate limiting: Is rate limiting implemented before launch, or after the first abuse incident?
Pagination: Are list endpoints paginated from day one? SELECT * FROM events returning 2 million rows is a real incident.

What I'm actually looking for: does the API communicate intent, or does it require the consumer to know implementation details?

6. Security

Security review is where I slow down. A missed finding here has a different consequence than a missed finding in "Code Quality."

I follow OWASP Top 10 as a baseline and add:

Authentication: Are credentials hashed with a modern algorithm (bcrypt, Argon2)? Is there account lockout? Is password reset implemented securely (time-limited tokens, single-use)?

Authorization: Is authorization checked at the service layer, not just the route layer? Is there a test for horizontal privilege escalation (user A accessing user B's data)?

Injection: SQL injection via ORM parameter binding — is raw string interpolation present anywhere? Is user input ever passed to shell commands?

Secrets management: Are secrets in environment variables (acceptable) or hardcoded/committed to source control (not acceptable)? Is .env in .gitignore?

Transport security: Is HTTPS enforced? Are security headers present (CSP, HSTS, X-Frame-Options)?

The audit pattern that finds the most real vulnerabilities: Follow the data. Pick a piece of user-provided data and trace it from HTTP request to database and back. Every transformation and validation point along the way is a potential vulnerability.

7. Performance

Performance problems have two categories: those that exist now, and those that will exist at 10x current load.

Current state:

Is there caching at the right layer (in-memory, Redis, CDN)?
Are images optimized? Is there a CDN in front of static assets?
Is compression enabled (Gzip/Brotli) for API responses?

Future state:

What is the bottleneck at 10x? Is it the database? The application server? A third-party API with rate limits?
Are there synchronous operations in the hot path that could be moved to a queue?
Is there a single replica database that becomes the bottleneck under read load?

The question I ask in every review: What is the slowest operation in the system, and is it in the hot path?

8. DevOps

DevOps is the operational envelope around the software. A perfect codebase deployed manually to a single server is fragile. An imperfect codebase with solid CI/CD, automated rollbacks, and structured logging is manageable.

Key areas:

CI/CD: Is every merge to main automatically tested and deployed? What is the deploy time? Can you roll back in under 5 minutes?
Container strategy: Are containers immutable? Is the image built once and promoted through environments, or rebuilt at each stage?
Environment parity: Are staging and production running the same infrastructure? Environment-specific bugs are a signal that they're not.
Observability: Can you answer "is the system working right now?" without checking logs manually? Are there dashboards and alerts?
Incident response: Is there a runbook? Does the team practice incident response, or is every incident a first-time experience?

9. Cloud Infrastructure

Cloud-specific concerns that are separate from application-level DevOps:

Scalability: Is compute auto-scaling configured? Will the system scale on demand, or require manual intervention?
High Availability: Are there single points of failure? Is the database replicated? Is the application deployed across availability zones?
Disaster Recovery: Is there a tested DR plan? What is the geographic footprint?
Cost model: Are resources appropriately sized? Are there reserved instances for predictable load?

The question that exposes unexamined assumptions: What happens if the primary region goes down for 4 hours? Walk me through it.

10. Testing

Testing is the only mechanism that provides evidence (not just confidence) that the system works correctly.

I evaluate:

Coverage: What is the unit test coverage? More importantly: are the right things tested (business logic, not getters/setters)?
Integration tests: Are there tests that exercise the real database, not mocks? Mocks that diverge from reality cause incidents.
E2E tests: Is there a smoke test suite that runs on every production deployment?
Test design: Are tests isolated? Is there test data management? Do tests fail for one reason, or do they entangle multiple concerns?

The coverage trap: 85% coverage achieved by testing trivial code is less valuable than 40% coverage of the critical paths. Coverage is a proxy metric. What matters is: can you confidently refactor?

11. Code Quality

Code quality is the accumulation of thousands of small decisions. I evaluate it at the structural level:

Readability: Can a new engineer understand what a function does without reading its dependencies?
Naming: Do variable and function names express intent? Is there Hungarian notation, single-letter variables in non-trivial contexts, or names that contradict the actual behavior?
Modularity: Are modules sized appropriately? Are there 5000-line files?
Technical debt: Is there a TODO archaeology layer where comments from three years ago reference tickets that no longer exist?

What I'm actually measuring: the cognitive load required to make a change. High cognitive load → slow velocity → more bugs → more technical debt. It compounds.

12. AI Readiness

In 2026, AI readiness is a first-class architectural concern. Not "are you using AI?" but "is the system designed to integrate AI as capabilities evolve?"

Key questions:

API-first design: Is the system's business logic accessible via clean APIs, or is it buried in monolithic processes that AI agents can't interact with?
Event-driven architecture: Is significant state change emitted as events that AI agents can react to?
MCP compatibility: Are you thinking about Model Context Protocol? Can an AI agent inspect and act on your system's data through structured interfaces?
RAG readiness: Is content structured in a way that supports retrieval-augmented generation? Are there semantic search capabilities or hooks for them?
Data quality: AI is only as good as its training and retrieval data. Is your data structured, clean, and accessible?

The strategic question: In 18 months, a competitor will have AI agents that can automate 30% of what your users do manually. Does your architecture make that possible, or does it make it impossible?

The Scoring Model

Each dimension is scored 0–10:

Score	Interpretation
9–10	Best practice. No meaningful improvement needed.
7–8	Solid. Minor improvements would have diminishing returns.
5–6	Adequate. Known gaps but no immediate risk.
3–4	Significant gaps. Improvement needed within 3-6 months.
1–2	Critical deficiencies. Immediate attention required.
0	Not implemented at all.

From the 12 category scores I derive composite metrics:

Overall Technical Score (0–100): weighted average, security and reliability weighted higher
Production Readiness (%): a function of security, DevOps, testing, and observability scores
Enterprise Readiness (%): adds compliance posture, audit logging, and multi-tenancy to production readiness
AI Readiness Score: standalone, given its increasing strategic importance

The Output

The audit produces four deliverables:

1. The scorecard — 12 scores, risk levels (Low/Medium/High/Critical), and one-line findings.

2. Critical issues — The 10 findings that carry the most risk, regardless of category. These are the things that could cause an incident, a breach, or a failed fundraise due diligence.

3. Quick wins — Changes that take under a week and have disproportionate impact. These maintain momentum and demonstrate value early.

4. Roadmap — Medium-term (1–3 months) and long-term (6–12 months) architectural improvements, with rough effort estimates and business justification.

What Surprises People Most

When I share results with engineering teams, the finding that surprises them most is almost always in Testing or Database — not Security, which most teams are at least anxious about.

Specifically:

Integration tests that mock the database: Teams confidently claim "we have 80% coverage" and then discover that none of those tests would catch a query that works on SQLite but fails on PostgreSQL. This is a real incident pattern.
Unmigrated schema changes: Tables that were altered directly in production, with no corresponding migration file. The application works, but you can't reproduce the production environment from source control alone.
Missing composite indexes: The queries work fine at current data volumes. They'll time out at 10x. Nobody notices because load testing isn't in the development process.

A Note on Judgment

This framework is a tool for structured thinking, not a replacement for it.

A system with a 6/10 Security score at an internal tool company is a different situation than a 6/10 at a payment processor. A 4/10 Testing score on a prototype is acceptable; on a system processing medical records, it's a liability.

The framework surfaces what to look at. The review conversation — with the engineering team, with the CTO, with the business stakeholders — determines what it means and what to do about it.

That's the part that can't be automated.

If you're about to make a significant technical investment — new hire, rewrite, acquisition — and want an independent architecture review, I'm available for fractional engagements.

React 19 Streaming from FastAPI: Building Truly Progressive UI Without Separate Stream and Non-Stream Endpoints

Ugur Aslim — Wed, 24 Jun 2026 10:49:16 +0000

React 19 Streaming from FastAPI: Building Truly Progressive UI Without Separate Stream and Non-Stream Endpoints

I've watched teams burn engineering hours duplicating endpoints. /api/generate for buffered responses. /api/generate/stream for streaming. Different error handling, different retry logic, different bugs. It's absurd.

Here's the truth: a single endpoint that always streams is enough. Your React 19 client decides whether to buffer the entire response or render it progressively. One source of truth. One place to fix bugs. One endpoint to monitor.

I built this pattern into CitizenApp because we support users on everything from fiber to 3G. Some want instant full responses. Some want to see Claude thinking in real-time. Same endpoint. Different consumption.

Why This Matters

Most teams default to separate endpoints because they think streaming and buffering are fundamentally different operations. They're not. Streaming is just where you buffer.

Server-side buffering (traditional approach):

Collect entire response → send JSON → client renders instantly
Wastes bandwidth if user leaves early
Feels slower on slower networks
Server holds memory longer

Client-side buffering (our approach):

Stream chunks → client accumulates → render whenever you want
User sees progress immediately
Same network footprint either way
Server frees memory faster

The endpoint doesn't care which strategy you use. It streams. Period.

The FastAPI Endpoint

Here's a single endpoint that powers both streaming and buffered consumption:

from fastapi import FastAPI, HTTPException
from fastapi.responses import StreamingResponse
import anthropic
import json
import asyncio

app = FastAPI()

@app.post("/api/generate")
async def generate(request: dict):
    """
    Single endpoint. Always streams via Server-Sent Events.
    Client decides whether to buffer or render progressively.
    """
    prompt = request.get("prompt")
    if not prompt:
        raise HTTPException(status_code=400, detail="prompt required")

    async def event_generator():
        client = anthropic.AsyncAnthropic()
        full_text = ""

        try:
            with client.messages.stream(
                model="claude-3-5-sonnet-20241022",
                max_tokens=1024,
                messages=[
                    {"role": "user", "content": prompt}
                ],
            ) as stream:
                for text in stream.text_stream:
                    full_text += text
                    # SSE format: "data: " prefix + JSON + double newline
                    chunk = {
                        "type": "text_delta",
                        "delta": text,
                        "stop_reason": None
                    }
                    yield f"data: {json.dumps(chunk)}\n\n"

            # Signal completion
            yield f"data: {json.dumps({'type': 'message_stop', 'stop_reason': 'end_turn'})}\n\n"

        except Exception as e:
            yield f"data: {json.dumps({'type': 'error', 'message': str(e)})}\n\n"

    return StreamingResponse(
        event_generator(),
        media_type="text/event-stream",
        headers={
            "Cache-Control": "no-cache",
            "Connection": "keep-alive",
        }
    )

That's it. No /stream variant. No buffer mode. One endpoint that streams everything via Server-Sent Events (SSE). The format is dead simple:

type: text_delta = chunk arrived
type: message_stop = done
type: error = something broke

The React 19 Client Hook

Now the client hook that consumes this endpoint. Notice how we support both streaming and buffering without separate API calls:

import { useState, useCallback } from 'react';

type StreamMode = 'streaming' | 'buffered';

interface UseGenerateOptions {
  mode?: StreamMode;
  onChunk?: (chunk: string) => void;
  onComplete?: (fullText: string) => void;
  onError?: (error: string) => void;
}

export function useGenerate({
  mode = 'streaming',
  onChunk,
  onComplete,
  onError,
}: UseGenerateOptions) {
  const [isLoading, setIsLoading] = useState(false);
  const [error, setError] = useState<string | null>(null);

  const generate = useCallback(
    async (prompt: string) => {
      setIsLoading(true);
      setError(null);

      try {
        const response = await fetch('/api/generate', {
          method: 'POST',
          headers: { 'Content-Type': 'application/json' },
          body: JSON.stringify({ prompt }),
        });

        if (!response.ok) {
          throw new Error(`HTTP ${response.status}`);
        }

        const reader = response.body?.getReader();
        if (!reader) throw new Error('No response body');

        let fullText = '';
        const decoder = new TextDecoder();
        let buffer = '';

        while (true) {
          const { done, value } = await reader.read();
          if (done) break;

          buffer += decoder.decode(value, { stream: true });
          const lines = buffer.split('\n');
          buffer = lines.pop() || '';

          for (const line of lines) {
            if (!line.startsWith('data: ')) continue;

            try {
              const event = JSON.parse(line.slice(6));

              if (event.type === 'text_delta') {
                fullText += event.delta;

                // Streaming mode: call onChunk immediately
                if (mode === 'streaming' && onChunk) {
                  onChunk(event.delta);
                }
              } else if (event.type === 'message_stop') {
                // Done
              } else if (event.type === 'error') {
                throw new Error(event.message);
              }
            } catch (e) {
              // Ignore JSON parse errors on partial lines
            }
          }
        }

        // Buffered mode: call onChunk once with full text
        if (mode === 'buffered' && onChunk) {
          onChunk(fullText);
        }

        onComplete?.(fullText);
      } catch (err) {
        const message = err instanceof Error ? err.message : 'Unknown error';
        setError(message);
        onError?.(message);
      } finally {
        setIsLoading(false);
      }
    },
    [mode, onChunk, onComplete, onError]
  );

  return { generate, isLoading, error };
}

Using It in a Component

Here's how you'd use this in a React component. Notice: no different endpoints, no conditional logic in the component:

export function GenerateForm() {
  const [response, setResponse] = useState('');
  const { generate, isLoading } = useGenerate({
    mode: 'streaming', // or 'buffered' based on user preference
    onChunk: (chunk) => setResponse(prev => prev + chunk),
    onComplete: (full) => console.log('Done:', full),
    onError: (err) => console.error('Failed:', err),
  });

  return (
    <div>
      <button 
        onClick={() => generate('Explain React 19 streaming...')}
        disabled={isLoading}
      >
        Generate
      </button>
      <div className="mt-4 whitespace-pre-wrap">
        {response}
      </div>
    </div>
  );
}

Want to toggle between modes based on network speed? Add a preference hook:

function useNetworkMode() {
  const [connection, setConnection] = useState<'4g' | '3g'>('4g');

  useEffect(() => {
    const nav = navigator as any;
    if (nav.connection) {
      const updateMode = () => {
        const effectiveType = nav.connection.effectiveType;
        setConnection(['3g', '2g'].includes(effectiveType) ? '3g' : '4g');
      };
      updateMode();
      nav.connection.addEventListener('change', updateMode);
      return () => nav.connection.removeEventListener('change', updateMode);
    }
  }, []);

  return connection === '3g' ? 'streaming' : 'buffered';
}

Gotcha: SSE and Content-Length

Here's what burned me: I initially didn't set Cache-Control: no-cache on the FastAPI response. Some proxies buffered the entire stream before sending it to the client, defeating the purpose. The headers in the endpoint above fix that.

Also, never add Content-Length to a streaming response. It's impossible to know the length ahead of time, and setting it incorrectly breaks everything.

Why Not WebSockets?

People always ask. WebSockets are overkill here. You're not bidirectional. SSE is simpler, works over HTTP/2, and survives proxies better. Save WebSockets for real-time collaboration.

The Win

One endpoint. One error handler. One place to add retry logic, caching headers, rate limits, or monitoring. Your API surface is minimal. Your cognitive load is lower. Your tests are faster.

This is the pattern I run in production at CitizenApp. It scales to thousands of concurrent streams without duplication headaches.

AI Agent Sprawl: Why Companies Are Drowning in Too Many AI Tools in 2026

Ugur Aslim — Tue, 23 Jun 2026 19:07:57 +0000

The conversation about AI has shifted. In 2024, teams debated which model was smarter. In 2025, they shipped features with it. In 2026, the question everyone is quietly asking is: how do we manage all of this?

Cursor sits on every engineer's laptop. Claude Code runs in CI. Copilot is baked into the IDE. The product team uses ChatGPT. The data team runs Gemini. The marketing lead found yet another AI writing tool last Tuesday. Nobody has a full list. Nobody audits the tokens. Nobody knows which tool just sent your customer data to which endpoint.

This is AI agent sprawl. And it's the infrastructure problem of 2026.

What Is AI Agent Sprawl?

Sprawl happens when AI tool adoption outpaces organizational governance. It's not about using too many AI tools per se — it's about using them without visibility, control, or policy.

Signs you're in sprawl:

Different teams use different AI tools for equivalent tasks, with no standard
Token spend is invisible until the credit card bill arrives
Engineers can't answer "which AI tool touched this data?" for a given request
Prompt engineering happens in isolation, never shared, never versioned
When an AI tool goes down, you discover 6 different teams had dependencies on it

The difference between healthy multi-tool usage and sprawl is governance. One is intentional diversity. The other is accumulated technical debt disguised as productivity.

Why 2026 Became the Year Sprawl Exploded

Three things converged.

First: agentic AI went mainstream. LLMs stopped being assistants you typed at and became autonomous workers you delegated to. Claude Code, Devin, GitHub Copilot Workspace — agents that take a task, use tools, write code, run tests, and open PRs without human intervention at each step. The power increased dramatically. So did the surface area.

Second: the model market fragmented. A year ago, most teams defaulted to one provider. Today, there are compelling reasons to use different models for different tasks — Claude for reasoning-heavy work, GPT-4o for multimodal, Gemini for long context, local models for sensitive data. Each provider has its own API, its own token pricing, its own data retention policies. The engineering overhead of managing multiple integrations compounds fast.

Third: the cost of not adopting AI became visible. Teams that didn't ship AI features fell behind. The pressure to adopt was high enough that governance conversations got deprioritized. "We'll figure out the policy later" is how sprawl starts.

How Cursor, Claude Code, and Copilot Create Invisible Complexity

The tools themselves aren't the problem. The problem is the invisible dependency graph they create.

Take a typical engineering team:

Developer A: Cursor Pro (sends code + context to Anthropic)
Developer B: GitHub Copilot (sends code to GitHub/OpenAI)
Developer C: Claude Code (sends code + shell output to Anthropic)
CI pipeline: custom GPT-4 integration (sends diffs to OpenAI)
Code review bot: Gemini Code Assist (sends PRs to Google)

Now ask: which of these has access to your database schemas? Your API keys in environment files? Your customer data in test fixtures?

The answer is probably "all of them, sometimes." Because developers don't consistently sanitize context before AI tools process it. They shouldn't have to manually do this — but without guardrails, they will forget.

This isn't hypothetical. Context window leakage — where an AI coding assistant processes a file containing credentials or PII because it was open in the editor — is a real attack vector that security teams are actively working against in 2026.

The Token Cost and ROI Problem

AI costs are uniquely opaque. A single developer running Claude Code in agent mode can generate thousands of API calls in a day — each one costing a fraction of a cent, totaling meaningfully at scale.

The economics look like this:

10 engineers × Claude Code (heavy use)  = ~$800–1,200/month
+ GitHub Copilot Enterprise             = ~$380/month
+ ChatGPT Team (design + PM)            = ~$300/month
+ Internal RAG system (GPT-4o)          = ~$200–600/month (spiky)
+ Misc tools (one per team)             = ~$400/month

Total: ~$2,000–2,900/month for a 15-person team

That's not outrageous — it's probably less than one employee's monthly cost. The problem is it's invisible until it isn't. Usage patterns are non-linear. A single poorly-constrained agent loop — a tool that retries on failure without backoff, or an evaluation pipeline that processes the full dataset on every run — can triple your bill in a week.

Without centralized token accounting, you can't identify the expensive outlier until the invoice arrives.

Data Security: Which AI Tool Gets Which Data?

This is the governance question most teams haven't answered formally.

The right mental model is data classification applied to AI tool access:

Data Class	Examples	Permitted AI Tools
Public	Marketing copy, public docs	Any tool
Internal	Architecture diagrams, sprint plans	Tools with DPA in your jurisdiction
Confidential	Customer data, contracts	Self-hosted or zero-retention APIs only
Restricted	Credentials, PII, health data	No AI tools, full stop

Most teams don't have this matrix. They have informal intuitions ("we don't paste customer data into ChatGPT... I think"). Informal intuitions don't survive audits, don't satisfy GDPR Article 28, and don't protect you when someone pastes the wrong file into the wrong context.

The practical gap: many AI coding tools process files automatically without the developer explicitly choosing what to share. An IDE extension can index your codebase in the background. A code review bot can pull the full diff including test fixtures. Consent and control aren't automatic — they require architecture.

Agent Governance: Permissions, Logging, and Audit Trails

Agents that take autonomous action need governance primitives equivalent to what you'd require for human operators:

Permissions: What can this agent do? An agent that writes code shouldn't also be able to deploy to production. An agent that summarizes customer tickets shouldn't have read access to billing tables. Apply least privilege. Scope tool access explicitly.

Logging: Every action an agent takes should be logged with enough context to reconstruct what happened. At minimum: timestamp, model, prompt hash (not the full prompt — that may contain sensitive data), tool called, result class (success/failure/retry), cost.

Audit trail: The log needs to be immutable and queryable. When something goes wrong — and something will go wrong — you need to be able to answer "what did the agent do between 14:03 and 14:07 on Tuesday?"

Rate limits and circuit breakers: Agents should have hard limits on how many external calls they can make in a window, how much they can spend, and how many retries they attempt before halting and alerting. Without these, a buggy agent loop is an incident waiting to happen.

Human escalation triggers: Define the conditions under which an agent must stop and wait for human approval before continuing. Irreversible actions (deleting data, sending emails, deploying code) should require an explicit human gate in most contexts.

The Solution Architecture: AI Gateway Pattern

The most effective structural response to AI sprawl is a centralized AI gateway — a single internal endpoint that all AI traffic routes through before reaching external providers.

Your Services
     │
     ▼
┌─────────────────────────────────┐
│          AI Gateway             │
│  ┌─────────────────────────┐   │
│  │  Auth & Policy Engine   │   │
│  │  Rate Limiter           │   │
│  │  PII/Secret Scrubber    │   │
│  │  Cost Tracker           │   │
│  │  Request Logger         │   │
│  │  Prompt Version Store   │   │
│  └─────────────────────────┘   │
└──────────────┬──────────────────┘
               │
    ┌──────────┼──────────┐
    ▼          ▼          ▼
Anthropic    OpenAI    Gemini

The gateway gives you:

Single point of cost visibility: every token that leaves your infrastructure is counted and attributed
Secret scrubbing: strip credentials and PII patterns before the request leaves your network
Policy enforcement: block requests to non-approved providers for sensitive data classes
Provider abstraction: swap from GPT-4o to Claude 4 without touching application code
Prompt versioning: treat prompts as artifacts with versions, tests, and deployment history

This isn't a new concept — it's the same pattern as API gateways for microservices, applied to AI providers. The infrastructure exists. LiteLLM, Portkey, and similar tools implement this as open-source proxies you can self-host.

Tool Registry

Alongside the gateway, maintain a tool registry: a central inventory of every AI tool authorized for use, with owner, data class permissions, cost center, renewal date, and approved use cases.

This sounds like paperwork. It is paperwork — the kind that prevents you from discovering two teams are paying separately for equivalent tools, or that someone onboarded a new AI vendor during a sprint without a security review.

Prompt and Version Control

Prompts are code. They should live in version control, be reviewed like code, be tested before deployment, and be rolled back when they regress.

The teams I've seen handle this well treat prompts like database migrations: immutable, versioned, with automated evaluation on every change. When a model update changes behavior, you have a baseline to compare against.

Practical Checklist for Smaller Teams

If you're a startup or a team of under 20 engineers, you don't need a full governance platform on day one. You need enough structure to avoid the worst failure modes:

This week:

[ ] Inventory every AI tool currently in use. One spreadsheet: tool, owner, monthly cost, data access level
[ ] Set a hard rule: production credentials and customer data don't go into external AI tools
[ ] Enable spend alerts on every AI provider account ($50, $200, $500 thresholds)

This month:

[ ] Route all programmatic AI calls through a single internal client with logging enabled
[ ] Write down your data classification rules, even informally
[ ] Add .cursorrules or equivalent to repositories with sensitive data to scope what the AI tool can access

This quarter:

[ ] Evaluate a gateway proxy (LiteLLM is a reasonable starting point)
[ ] Establish a prompt library: shared, versioned, reviewed
[ ] Run a security review: which agent has which permissions? What's the blast radius if it goes wrong?

The Deeper Point

AI sprawl isn't a technology problem. It's an organizational maturity problem that technology can address.

The companies getting this right aren't the ones with the most AI tools. They're the ones who treated AI infrastructure with the same engineering rigor they apply to their databases, their auth systems, and their deployment pipelines.

The tools are powerful enough now that the limiting factor isn't capability. It's governance. And governance is, at its core, just the discipline of knowing what's running in your system, why, and what it's allowed to do.

You already apply that discipline to your production databases. Apply it to your AI agents.

Building AI integrations for enterprise clients? The patterns above apply whether you're standardizing across a 5-person startup or a 500-person engineering org. The specifics scale — the principles don't.

React Performance Optimization: Beyond Code Splitting

Ugur Aslim — Mon, 22 Jun 2026 21:11:07 +0000

React Performance Optimization: Beyond Code Splitting

Code splitting is table stakes. If you're still shipping your entire React app as one bundle, we need to talk—but that's not what this post is about. I'm assuming you've already done the obvious stuff: lazy loading routes, dynamic imports, tree shaking. What I want to cover is the debugging workflow that actually catches performance issues before they hit production, and the optimization strategies that move the needle when everything else is already optimized.

I learned this the hard way building CitizenApp. We had code splitting nailed down, but users on 4G still experienced janky interactions because we weren't tracking where React was actually spending CPU time. This post is that hard-won knowledge distilled.

The Performance Debugging Workflow That Actually Works

Before optimizing anything, you need visibility. Not just Lighthouse scores—those are lag indicators. You need to see where React is re-rendering unnecessarily and why.

Step 1: Identify the culprit with React DevTools Profiler

Open React DevTools → Profiler tab → record a user interaction. Look at the flame graph. You're looking for:

Renders that don't correspond to state changes you made
Components that render multiple times in quick succession
Long render times on components that should be pure

// Bad: This component re-renders whenever parent renders
function UserCard({ user, onUpdate }) {
  return (
    <div>
      <h2>{user.name}</h2>
      <button onClick={() => onUpdate(user.id)}>Update</button>
    </div>
  );
}

// Better: Memoize if props don't change frequently
const UserCard = React.memo(({ user, onUpdate }) => {
  return (
    <div>
      <h2>{user.name}</h2>
      <button onClick={() => onUpdate(user.id)}>Update</button>
    </div>
  );
}, (prev, next) => {
  // Custom comparison: only re-render if user data actually changed
  return prev.user.id === next.user.id && prev.user.name === next.user.name;
});

The custom comparison function is key. Default shallow comparison works 60% of the time. The other 40%, you'll be surprised what React thinks changed.

Step 2: Check your bundle with source-map-explorer

npm install --save-dev source-map-explorer
npm run build
npx source-map-explorer 'dist/**/*.js'

I prefer source-map-explorer over webpack-bundle-analyzer because it shows you the actual code, not just filenames. You'll often find that a library you imported once is somehow included in three different chunks. Or that a dependency pulled in a massive transitive dependency you didn't know about.

Last month I found that our PDF export feature was pulling in the entire PDF.js library (4MB!) even though only 2% of users ever used it. Moving it to a dynamic import reduced our main bundle by 40%.

Step 3: Profile runtime performance with Performance tab

// Wrap expensive operations in markers
performance.mark('process-users-start');
const processed = users.map(u => ({
  ...u,
  displayName: formatName(u.firstName, u.lastName),
  initials: getInitials(u)
}));
performance.mark('process-users-end');
performance.measure('process-users', 'process-users-start', 'process-users-end');

// Check the result in DevTools Performance tab
const measure = performance.getEntriesByName('process-users')[0];
console.log(`Processing took ${measure.duration}ms`);

This gives you frame-by-frame breakdowns. If you see 200+ millisecond tasks, you've found your problem.

Memoization Strategies That Actually Matter

React.memo gets all the attention, but I prefer useMemo and useCallback for most real-world cases. Here's why and when to use each.

Use `useMemo` for expensive calculations in render

function Analytics({ data }) {
  // Without useMemo, this recalculates on every render
  const stats = React.useMemo(() => {
    return {
      total: data.reduce((sum, d) => sum + d.value, 0),
      average: data.reduce((sum, d) => sum + d.value, 0) / data.length,
      percentiles: calculatePercentiles(data) // expensive
    };
  }, [data]);

  return <div>{stats.average}</div>;
}

I put this in every component that does more than string interpolation. The memory cost is negligible, but the CPU savings are real—especially on mobile devices.

Use `useCallback` for event handlers passed to child components

function UserList({ users, onSelect }) {
  // Without useCallback, child components re-render even if user list didn't change
  const handleSelect = React.useCallback((userId) => {
    onSelect(userId);
    analytics.track('user-selected', { userId });
  }, [onSelect]);

  return (
    <ul>
      {users.map(u => (
        <UserItem key={u.id} user={u} onSelect={handleSelect} />
      ))}
    </ul>
  );
}

// UserItem needs React.memo for useCallback to matter
const UserItem = React.memo(({ user, onSelect }) => (
  <li onClick={() => onSelect(user.id)}>{user.name}</li>
));

The combination matters. useCallback alone without React.memo on the child does nothing.

Avoid excessive memoization (yes, it's possible)

Here's what burned me: I memoized everything and ended up with a slower app. The dependency array comparison itself has a cost. If a dependency changes every render, you lose.

// Bad: The dependency array changes every render
const expensiveValue = useMemo(() => {
  return processData(data);
}, [{ ...options }]); // Creates new object every render!

// Good: Stable dependency reference
const expensiveValue = useMemo(() => {
  return processData(data);
}, [data.length, options.type]); // Only primitive values that actually matter

The Backend Connection: Where Most React Apps Actually Bottleneck

Real talk: your React app isn't slow because of React. It's slow because you're fetching too much data or waiting for a slow backend.

// Bad: Fetch user + their posts + comments on each post
function UserProfile({ userId }) {
  const [user, setUser] = useState(null);
  const [posts, setPosts] = useState([]);

  useEffect(() => {
    fetch(`/api/users/${userId}`)
      .then(r => r.json())
      .then(user => {
        setUser(user);
        // Now fetch posts...
        return fetch(`/api/users/${userId}/posts`);
      })
      .then(r => r.json())
      .then(posts => setPosts(posts));
  }, [userId]);
}

// Better: One endpoint, structured data
function UserProfile({ userId }) {
  const { data: profile, isLoading } = useQuery(
    ['user-profile', userId],
    () => fetch(`/api/users/${userId}/profile?include=posts,stats`).then(r => r.json()),
    { staleTime: 5 * 60 * 1000 } // Cache for 5 minutes
  );
}

On CitizenApp, moving to aggregate endpoints reduced Time to Interactive by 60%. More than any React optimization.

Gotcha: What I Missed

The DevTools Profiler lies (sometimes).

When you profile in development mode, React is running without optimizations. Production builds are significantly faster. I spent a week optimizing a component that showed as slow in the profiler, only to find it rendered in 2ms in production.

Always profile production builds. Use npm run build && npm run preview with your build tool, or deploy to a staging environment.

useTransition is not a silver bullet.

// I thought this would magically fix slow renders
const [isPending, startTransition] = useTransition();

const handleSort = (column) => {
  startTransition(() => {
    setSortColumn(column);
  });
};

It doesn't. It defers the update, which is great for UX (you can show a loading state), but the underlying work still happens. Use it for perception, not performance. The actual optimization needs to be in the calculation itself.

Performance isn't a feature you add at the end. It's a debugging skill you practice continuously. Profile → identify → fix → measure → repeat. Everything else is cargo cult optimization.

Astro Islands Architecture: When to Use Partial Hydration

Ugur Aslim — Mon, 22 Jun 2026 07:55:02 +0000

Astro Islands Architecture: When to Use Partial Hydration

I've shipped enough full SPAs to know their dirty secret: they're overly expensive for most websites. A marketing site doesn't need React hydrating a footer. A documentation portal doesn't need Vue managing a navigation bar. Astro's Islands architecture fixed this for me, and I'm not going back.

The Islands pattern is simple: ship static HTML by default, hydrate only the interactive pieces. You get the performance of a static site with the interactivity of a modern app. In practice, this means your Lighthouse scores stop being a source of shame.

The Problem with Full Hydration

Before Islands, I had two choices:

Pure static site – fast, but adding a single interactive dropdown meant rebuilding and redeploying.
Full SPA – interactive everywhere, but shipping 150kb of JavaScript for a testimonial carousel.

I'd reach for the SPA because I couldn't handle the deployment friction. Then I'd watch Core Web Vitals tank. The irony: most of the page didn't need JavaScript at all.

CitizenApp's dashboard started this way. The sidebar navigation was a full React component. The settings panel was a full React component. The static content blocks? Also React. We were hydrating ~40% of the page that never changed state. Shipping 180kb of JavaScript for what could've been 20kb.

Astro Islands solved this by inverting the default: everything is static unless you explicitly say otherwise.

How Astro Islands Actually Work

When you mark a component as interactive using the client: directive, Astro:

Renders it to static HTML on the server
Bundles only that component's JavaScript
Ships the minimal code needed to hydrate it in the browser
Leaves everything else as plain HTML

Here's a real example from CitizenApp's dashboard:

---
// src/pages/dashboard.astro
import StaticHeader from '../components/Header.astro';
import InteractiveChart from '../components/Chart.tsx';
import StaticFooter from '../components/Footer.astro';
---

<Layout>
  <StaticHeader />

  <InteractiveChart client:load data={chartData} />

  <StaticFooter />
</Layout>

The client:load directive tells Astro: "This needs JavaScript. Ship it." The Header and Footer remain pure HTML.

The client: directives matter:

client:load – Hydrate immediately on page load. Use for above-the-fold interactivity.
client:idle – Hydrate when the browser is idle (requestIdleCallback). Use for below-the-fold features.
client:visible – Hydrate only when the element enters the viewport. Use for modals, tooltips, lazy-loaded sections.
client:only – Skip server rendering, only hydrate on client. Avoid this; it kills SEO.

Decision Tree: Static vs. Island vs. SPA

Here's my mental model for every page I build:

Is the entire page interactive with frequently-changing state? → Full SPA (use Next.js, Remix)

Are there isolated interactive regions with other static content? → Islands (use Astro)

Is the page mostly static with minimal interactivity? → Static + one Island (use Astro)

Is it completely static? → Pure HTML (use Astro)

For CitizenApp, the dashboard is Islands. The settings page is mostly static with an interactive form Island. The marketing site is pure static.

Real Layout Examples

Example 1: Marketing Site with Island

---
// src/pages/pricing.astro
import PricingTable from '../components/PricingTable.tsx';
---

<html>
  <head>
    <title>Pricing</title>
  </head>
  <body>
    <!-- Static hero -->
    <section class="hero">
      <h1>Simple, transparent pricing</h1>
      <p>No surprises. Cancel anytime.</p>
    </section>

    <!-- Interactive pricing table Island -->
    <PricingTable client:visible plans={plans} />

    <!-- Static FAQ -->
    <section class="faq">
      {faqItems.map(item => (
        <details>
          <summary>{item.question}</summary>
          <p>{item.answer}</p>
        </details>
      ))}
    </section>
  </body>
</html>

The hero and FAQ are static HTML. The pricing table is the only Island. Users with JavaScript disabled still see the static sections. The JavaScript bundle is ~15kb instead of 80kb.

Example 2: Dashboard with Multiple Islands

---
// src/pages/dashboard/overview.astro
import Sidebar from '../components/Sidebar.astro';
import Navbar from '../components/Navbar.astro';
import DataGrid from '../components/DataGrid.tsx';
import LineChart from '../components/LineChart.tsx';
import StaticStats from '../components/Stats.astro';
---

<Layout>
  <Sidebar /> {/* Static */}
  <Navbar /> {/* Static */}

  <main>
    <StaticStats data={stats} /> {/* Static */}
    <LineChart client:load data={monthlyData} /> {/* Island */}
    <DataGrid client:visible rows={users} /> {/* Island */}
  </main>
</Layout>

Static components are compiled away. Islands are bundled separately with their dependencies. A user on a slow 3G connection downloads the page structure immediately, then hydrates the chart and grid as bandwidth allows.

Performance Impact: Real Numbers

On CitizenApp's marketing site, I measured switching from a full React app to Astro Islands:

JavaScript bundle: 180kb → 35kb
First Contentful Paint: 2.1s → 0.8s
Largest Contentful Paint: 3.2s → 1.4s
Cumulative Layout Shift: 0.15 → 0.02

The static header, footer, testimonials, and FAQ rendered immediately. Only the contact form hydrated. Users felt the difference.

Gotcha: Framework-Agnostic Hydration

Here's what burned me: I tried using a React Island with Redux state management, then added a separate Vue Island below it. They hydrated independently, couldn't share state, and I had duplicate data fetching.

Astro Islands are isolated. If you need shared state across Islands, you have two options:

Use Web APIs – localStorage, custom events, or a shared data attribute
Accept that you need more than Islands – and migrate to a full framework

For CitizenApp, I use Web APIs. The authentication Island sets localStorage with the user token. Other Islands read it. Lightweight and framework-agnostic.

// src/components/AuthIsland.tsx
export default function AuthIsland() {
  const [user, setUser] = useState<User | null>(null);

  useEffect(() => {
    const stored = localStorage.getItem('user');
    if (stored) setUser(JSON.parse(stored));
  }, []);

  const handleLogin = async (credentials) => {
    const response = await fetch('/api/login', {
      method: 'POST',
      body: JSON.stringify(credentials)
    });
    const userData = await response.json();
    localStorage.setItem('user', JSON.stringify(userData));
    setUser(userData);
  };

  return <LoginForm onSubmit={handleLogin} />;
}

Other Islands read from localStorage instead of fetching again.

What I Missed Early

I initially thought Islands were only for "mostly static" sites. I underestimated how much of a dashboard is actually static. The page shell, layout, headers, sidebar navigation—none of it changes without a full page reload. Only the data grid, charts, and forms need to be Islands.

I also didn't appreciate the deployment simplicity. With Astro on Cloudflare Pages, my builds are 30 seconds. No server rendering complexity. Everything pre-renders to static HTML at build time (with Islands hydrating on demand).

When NOT to Use Islands

If your page is a canvas editor, collaborative spreadsheet, or real-time multiplayer experience—use a full SPA. Islands add cognitive overhead when the entire page is inherently interactive. But for 90% of web projects? Marketing sites, dashboards, content platforms, admin panels? Islands win.

Ship less JavaScript. Your users' devices will thank you, and your Lighthouse scores will finally make sense.

API Design for AI: Streaming, Structured Output, and Rate Limits

Ugur Aslim — Sun, 21 Jun 2026 10:17:25 +0000

API Design for AI: Streaming, Structured Output, and Rate Limits

Most API design guides treat endpoints like they're serving JSON from a database. But Claude and other LLMs break that assumption: responses take 5–30 seconds, tokens cost money, and users expect real-time feedback. I've shipped CitizenApp with nine AI features across three product tiers, and I've learned the hard way that standard REST patterns don't cut it.

This post covers three patterns I now use by default: streaming responses, enforced schema validation, and token-aware rate limiting.

Why Standard API Design Fails for LLMs

Traditional API endpoints are fire-and-forget: you POST data, wait for 200 OK, get your JSON. That works when your database query finishes in 50ms.

Claude? 5–30 seconds is normal. Users see a blank screen and assume your app is broken. Your database connection pools exhaust. Your infra costs skyrocket because every request holds resources for minutes.

Also: tokens cost money. A user asking for a 2,000-token response isn't equivalent to a user asking for 10 rows from your database. You can't treat them as the same quota unit.

I learned this the expensive way: CitizenApp's early version rate-limited by request count. One power user submitted a single prompt that generated 50,000 tokens of output. That destroyed my Anthropic bill for the month.

Pattern 1: Streaming as Default

Streaming isn't optional for AI APIs—it's the only way to provide real user experience at scale.

The FastAPI + SSE Approach

I use Server-Sent Events (SSE) for streaming. It's simpler than WebSockets for one-way, text-based data, and the browser API is excellent.

from fastapi import FastAPI, BackgroundTasks
from fastapi.responses import StreamingResponse
import anthropic
import json
from datetime import datetime

app = FastAPI()
client = anthropic.Anthropic()

@app.post("/api/generate")
async def generate_text(prompt: str, background_tasks: BackgroundTasks):
    """Stream Claude response back to client via SSE."""

    async def stream_response():
        try:
            with client.messages.stream(
                model="claude-3-5-sonnet-20241022",
                max_tokens=1024,
                messages=[{"role": "user", "content": prompt}]
            ) as stream:
                for text in stream.text_stream:
                    # SSE format: data: <json>\n\n
                    event = {
                        "type": "text_delta",
                        "text": text,
                        "timestamp": datetime.utcnow().isoformat()
                    }
                    yield f"data: {json.dumps(event)}\n\n"

                # Final event with usage stats
                final_event = {
                    "type": "done",
                    "usage": {
                        "input_tokens": stream.get_final_message().usage.input_tokens,
                        "output_tokens": stream.get_final_message().usage.output_tokens
                    }
                }
                yield f"data: {json.dumps(final_event)}\n\n"
        except Exception as e:
            error_event = {"type": "error", "message": str(e)}
            yield f"data: {json.dumps(error_event)}\n\n"

    return StreamingResponse(stream_response(), media_type="text/event-stream")

On the React/TypeScript side:

async function streamGeneration(prompt: string): Promise<void> {
  const eventSource = new EventSource(
    `/api/generate?prompt=${encodeURIComponent(prompt)}`
  );

  let totalTokens = 0;

  eventSource.addEventListener("message", (e) => {
    const event = JSON.parse(e.data);

    if (event.type === "text_delta") {
      // Append to UI in real-time
      setOutput((prev) => prev + event.text);
    } else if (event.type === "done") {
      totalTokens = event.usage.output_tokens;
      console.log(`Generated ${totalTokens} tokens`);
      eventSource.close();
    } else if (event.type === "error") {
      console.error(event.message);
      eventSource.close();
    }
  });
}

Why I prefer this: Users see text appearing in real-time. The response feels instant, even if Claude takes 15 seconds. No spinners. No anxiety. For CitizenApp's document summarization feature, streaming reduced perceived latency by 60%.

Pattern 2: Structured Output with Schema Validation

Claude 3.5 supports tool_use for enforcing output schemas. I always use this for anything downstream—it's cheaper and more reliable than prompt engineering.

from pydantic import BaseModel
from typing import Optional

class AnalysisResult(BaseModel):
    """Enforced schema for content analysis."""
    sentiment: str  # positive, neutral, negative
    confidence: float  # 0.0 to 1.0
    key_topics: list[str]
    summary: str
    requires_escalation: bool

def analyze_with_schema(text: str) -> AnalysisResult:
    """Use Claude with tool_use to enforce structured output."""

    tools = [
        {
            "name": "return_analysis",
            "description": "Return the analysis result",
            "input_schema": {
                "type": "object",
                "properties": {
                    "sentiment": {
                        "type": "string",
                        "enum": ["positive", "neutral", "negative"]
                    },
                    "confidence": {
                        "type": "number",
                        "minimum": 0,
                        "maximum": 1
                    },
                    "key_topics": {
                        "type": "array",
                        "items": {"type": "string"},
                        "max_items": 5
                    },
                    "summary": {"type": "string"},
                    "requires_escalation": {"type": "boolean"}
                },
                "required": ["sentiment", "confidence", "key_topics", "summary", "requires_escalation"]
            }
        }
    ]

    response = client.messages.create(
        model="claude-3-5-sonnet-20241022",
        max_tokens=512,
        tools=tools,
        messages=[
            {
                "role": "user",
                "content": f"Analyze this text:\n\n{text}"
            }
        ]
    )

    # Extract tool use block
    for block in response.content:
        if block.type == "tool_use":
            result_data = block.input
            return AnalysisResult(**result_data)

    raise ValueError("Claude didn't use the tool")

Why this matters: Structured output isn't about prettiness—it's about predictability. Your downstream code doesn't need error handling for weird Claude responses. Your database schema is guaranteed to match. Testing is deterministic.

Pattern 3: Token-Aware Rate Limiting

Request-count limits are meaningless for AI APIs. One user might consume 100 tokens; another might consume 50,000 in a single request.

from redis import Redis
from datetime import datetime, timedelta

redis = Redis(host="localhost")

class TokenQuota:
    def __init__(self, user_id: str, tier: str):
        self.user_id = user_id
        self.tier = tier
        # Define quota by tier
        self.daily_limit = {
            "free": 10_000,
            "pro": 500_000,
            "enterprise": None  # unlimited
        }[tier]

    def get_remaining(self) -> int:
        """Check remaining tokens for today."""
        key = f"tokens:{self.user_id}:{datetime.utcnow().date()}"
        used = redis.get(key)
        return self.daily_limit - int(used or 0)

    def deduct(self, tokens: int) -> bool:
        """Deduct tokens, return True if allowed."""
        if self.tier == "enterprise":
            return True

        remaining = self.get_remaining()
        if tokens > remaining:
            return False

        key = f"tokens:{self.user_id}:{datetime.utcnow().date()}"
        redis.incrby(key, tokens)
        # Reset tomorrow
        redis.expire(key, 86400)
        return True

Apply this after the Claude call completes:

@app.post("/api/generate")
async def generate_text(user_id: str, prompt: str):
    # ... stream Claude response ...

    # After stream ends, check quota
    quota = TokenQuota(user_id, user.tier)
    output_tokens = stream.get_final_message().usage.output_tokens

    if not quota.deduct(output_tokens):
        # Log this and notify user
        log_quota_exceeded(user_id, output_tokens)
        return {"error": "Quota exceeded"}

    return {"success": True}

I charge by tokens used, not requests made. Users on the free tier get 10K tokens/month. That's fair, transparent, and matches how Claude actually costs money.

Gotcha: Streaming Timeout & Retries

Here's what bit me: streaming responses timeout after ~60 seconds on most cloud platforms (Vercel, Render, Cloudflare Pages). Claude sometimes takes longer than that for 2000+ token generations.

I now set max_tokens=1024 by default and let users request longer responses via pagination. Yes, it's UX friction. But it's better than timeouts.

Also: never retry a streaming response. The user already saw partial output. You'll either duplicate tokens or confuse them. Log it and ask them to resubmit if it failed.

API design for AI is about accepting constraints, not fighting them. Streaming feels slower but isn't. Structured output adds 10 lines of code and eliminates 90% of bugs. Token-based quotas are the only fair way to charge.

Build for

FastAPI Performance Tuning: Caching, Async, and Production Bottlenecks

Ugur Aslim — Sat, 20 Jun 2026 12:44:41 +0000

FastAPI Performance Tuning: Caching, Async, and Production Bottlenecks

I've tuned FastAPI applications from handling 50 req/s to 5,000+ req/s in production. The gap between a "working" FastAPI app and a fast one isn't magic—it's understanding where time actually goes and fixing it systematically.

Most performance advice is generic ("use caching", "go async"). This post is specific: here's what killed my APIs, how I measured it, and the exact patterns I use now in CitizenApp.

The Three Bottlenecks That Matter

Before optimizing anything, I profile. FastAPI's Starlette foundation gives you request timing for free—but you need to look.

1. Synchronous database calls block the event loop. This is the killer. One 200ms database query in a sync endpoint blocks every other request waiting for an async worker. I learned this the hard way when our auth endpoint hit production traffic.

2. Missing connection pooling. SQLAlchemy without proper pool configuration creates new database connections per request. I've watched a 4-core Render instance exhaust Postgres connections (100 default limit) at only 300 concurrent users.

3. Uncompressed responses. This sounds trivial until you're serving 5MB JSON arrays. Gzip cuts that to 200KB. On a 10Mbps connection, that's the difference between 4 seconds and 160ms per request.

The Pattern I Use: Async-First with Strategic Sync

Here's my rule: make the database layer async first, database access second, serialization third.

# ❌ This blocks. Don't do this.
@app.get("/users/{user_id}")
def get_user(user_id: int, db: Session = Depends(get_db)):
    user = db.query(User).filter(User.id == user_id).first()
    return user

# ✅ This doesn't block. Do this.
@app.get("/users/{user_id}")
async def get_user(user_id: int, db: AsyncSession = Depends(get_async_db)):
    result = await db.execute(select(User).where(User.id == user_id))
    user = result.scalar_one_or_none()
    return user

Why async matters: FastAPI runs on Uvicorn with an event loop. If your endpoint is sync, Uvicorn queues it to a thread pool—creating context switch overhead and limiting concurrency. Async endpoints stay on the event loop.

But here's what burned me: not all libraries are async-friendly. Requests library? Sync only. Anthropic Python SDK? Sync by default. When I need sync code, I use run_in_threadpool:

from fastapi.concurrency import run_in_threadpool
import anthropic

@app.post("/analyze")
async def analyze(text: str):
    client = anthropic.Anthropic()

    # This would block. Don't do it directly:
    # response = client.messages.create(...)

    # Instead:
    response = await run_in_threadpool(
        client.messages.create,
        model="claude-3-5-sonnet-20241022",
        max_tokens=1024,
        messages=[{"role": "user", "content": text}]
    )
    return {"analysis": response.content[0].text}

This offloads the blocking call to Uvicorn's thread pool without freezing the event loop.

Connection Pooling: The Silent Killer

SQLAlchemy's default pool is designed for single-process apps. In production with multiple workers, you'll exhaust connections.

from sqlalchemy.ext.asyncio import create_async_engine, AsyncSession
from sqlalchemy.pool import NullPool, QueuePool

# Development (single worker, temporary connections):
engine = create_async_engine(
    DATABASE_URL,
    poolclass=NullPool  # Don't pool in dev
)

# Production (multiple workers, connection pooling):
engine = create_async_engine(
    DATABASE_URL,
    poolclass=QueuePool,
    pool_size=5,  # connections per worker
    max_overflow=10,  # overflow when pool exhausted
    pool_recycle=3600,  # recycle connections every hour
    echo=False
)

async def get_async_db():
    async with AsyncSession(engine) as session:
        yield session

The math: pool_size * num_workers should be less than your database's max connections. On Render's standard plan (2 workers), I use pool_size=5 + max_overflow=10 = max 30 connections. Postgres default allows 100, so we're safe.

Caching: Where It Actually Helps

I cache three things: rarely-changing data, expensive computations, and database queries.

from functools import lru_cache
from datetime import datetime, timedelta

# Option 1: Simple in-memory cache (single worker only)
@lru_cache(maxsize=128)
def get_org_plan(org_id: int) -> str:
    # This would hit database
    return "pro"  # stubbed

# Option 2: Redis (multi-worker, production)
import redis.asyncio as redis

redis_client: redis.Redis = None

async def init_redis():
    global redis_client
    redis_client = await redis.from_url("redis://localhost")

async def get_cached_org_plan(org_id: int) -> str:
    cached = await redis_client.get(f"org:{org_id}:plan")
    if cached:
        return cached.decode()

    # Fetch from DB
    result = await db.execute(select(Org).where(Org.id == org_id))
    org = result.scalar_one()

    # Cache for 1 hour
    await redis_client.setex(f"org:{org_id}:plan", 3600, org.plan)
    return org.plan

Don't cache everything. Cache invalidation is hard. I cache only:

Organization settings (invalidate on update)
AI model pricing (stale data is acceptable)
Public user profiles (public = cacheable)

What I don't cache: auth tokens, user permissions, real-time data. The invalidation complexity isn't worth it.

Compression and Middleware

from fastapi.middleware.gzip import GZIPMiddleware
from fastapi.middleware.cors import CORSMiddleware

app.add_middleware(GZIPMiddleware, minimum_size=1000)

# CORS first (applied last, evaluated first)
app.add_middleware(
    CORSMiddleware,
    allow_origins=["https://app.citizenapp.com"],
    allow_credentials=True,
)

Middleware order matters in FastAPI. The last add_middleware is the first evaluated. I add compression last (closest to the actual endpoint).

Gzip minimum_size=1000: don't compress small responses. The overhead isn't worth it.

Gotcha: Task Queues Hide Performance Issues

I used to ship everything async and assume it was fast. Then I shipped an endpoint that called an external API (3-second timeout). Users felt 3-second latency.

The fix: offload to a task queue.

from celery import Celery

celery_app = Celery("tasks", broker="redis://localhost")

@app.post("/send-email")
async def send_email(email: str):
    # Fire and forget
    celery_app.delay(send_email_task, email)
    return {"status": "queued"}

@celery_app.task
def send_email_task(email: str):
    # This blocks for 2 seconds. Celery workers handle it, not your API.
    requests.post("https://sendgrid.com/...", ...)

Users get instant response. Background job handles the slow operation.

What I Missed Early

I profiled endpoints in isolation. Then production traffic patterns exposed cache misses, lock contention, and connection pool exhaustion I never saw in testing.

Load test with realistic traffic patterns, not synthetic uniform requests. Use locust or k6:

# locust test
from locust import HttpUser, task

class APIUser(HttpUser):
    @task(3)
    def list_items(self):
        self.client.get("/items")

    @task(1)
    def create_item(self):
        self.client.post("/items", json={"name": "test"})

This caught that our list endpoint (cached) was fine, but unbatched creates killed the database.

Your Move

Start with async database access. Add caching only when profiling shows repeated queries. Load test before deploying. Measure before and after—don't guess.

That's how CitizenApp handles thousands of concurrent users without infrastructure complexity.

Microservices vs Monolith: When to Split Your Architecture

Ugur Aslim — Fri, 19 Jun 2026 11:43:12 +0000

Microservices vs Monolith: When to Split Your Architecture

I've rebuilt CitizenApp's backend twice. Once I split it into microservices because "that's what production systems do." I was wrong. We spent 6 months debugging distributed tracing logs instead of shipping features. Then we merged back into a monolith, and velocity tripled.

This post isn't romantic. It's a decision framework for when distributed systems actually pay off—and when they're just complexity theater.

The Monolith Default

Start monolithic. Every successful SaaS I know—Stripe, GitHub, Figma—began as a single codebase. This isn't laziness; it's pragmatism.

A monolith with Astro + FastAPI gives you:

Single deployment pipeline. One GitHub Actions workflow deploys everything.
Shared database transactions. ACID guarantees without eventual consistency nightmares.
Simpler debugging. Stack traces tell the whole story.
Lower operational overhead. One container, one set of logs, one database.

For CitizenApp's first 18 months, we ran everything in a single FastAPI app on Render. 50K users. 9 AI features. One database. We shipped features faster than teams with "proper" microservices architecture.

Cost matters too. A monolith on Render costs ~$50/month to start. Your first microservice architecture? You're looking at orchestration, service mesh tooling, monitoring agents, minimum 3-4x the infrastructure cost.

When Splitting Actually Wins

Stop generalizing about "scale." Microservices solve specific problems:

1. Independent Deployment Cycles

Your ML pipeline team needs to redeploy inference workers 20x daily. Your API team ships weekly. A monolith couples them.

Split when: Different services have different deployment frequencies or risk profiles.

# BAD: Monolith—one team blocks another
class FastAPIApp:
    def get_user(self, user_id: str):
        return db.query(User).filter(User.id == user_id).first()

    def generate_embedding(self, text: str):
        # If this crashes, entire API goes down
        return claude_client.create_embedding(text)

# GOOD: Separate services
# api/main.py
@app.get("/users/{user_id}")
async def get_user(user_id: str):
    return {"id": user_id, "name": "John"}

# ml/embedding_service.py
@app.post("/embed")
async def embed_text(text: str):
    # Independent scaling, independent deploys
    return await claude_client.create_embedding(text)

2. Resource Isolation

One feature consumes all database connections and starves the rest. A monolith makes this a crisis.

Split when: You need to isolate resource usage for reliability.

I watched CitizenApp's document processing eat 40 DB connections while user login requests queued. We extracted it:

# Dedicated PostgreSQL pool for background jobs
background_pool = create_engine(
    DATABASE_URL,
    pool_size=5,
    max_overflow=10,
    pool_pre_ping=True  # Verify connections are alive
)

# Main API keeps its own pool
api_pool = create_engine(
    DATABASE_URL,
    pool_size=20,
    max_overflow=20
)

This was a config fix, not a service split. Know the difference.

3. Technology Lock-in

Your team knows Python. A new requirement needs Node.js + WebSockets. You hire specialists who want to own their stack.

Split when: Technology genuinely constrains your team, not ego.

At CitizenApp, our real-time collaboration engine became a separate Node.js service. Python developers built the core product. Node developers owned the socket layer. Clear boundaries. Clear ownership.

4. Scale-Specific Optimization

You're serving 100K concurrent WebSocket connections for real-time features. Your API serves REST requests with average 200ms response time.

These have opposite optimization strategies. A shared codebase gets tangled.

Split when: Different services optimize for different performance profiles.

The Checklist: Stay Monolithic Unless...

□ Different services deploy 5+ times per day independently?
□ Resource contention causes production incidents monthly?
□ You need different tech stacks that teams prefer to own?
□ One service requires horizontal scaling others don't?
□ Your database is a bottleneck (not your app logic)?
□ You have 50+ engineers who can't maintain coherent monolith?

If you checked fewer than 3 boxes: stay monolithic.

Cost Reality

Monolith (Render, 10K users): $50/month

Basic microservices (Kubernetes, observability): $2K+/month

Production microservices (service mesh, distributed tracing, on-call rotations): $10K+/month

The last line item is often invisible. Microservices require:

On-call rotations across services
Distributed tracing (DataDog, New Relic)
API gateway and load balancing
Service mesh (Istio/Linkerd) at scale
Chaos engineering to test failure scenarios

Gotcha: The False Signal

Most teams split too early because of vanity architecture.

I've seen startups choose Kubernetes on day 1. "We want to scale." They're running 3 Docker containers. Kubernetes doesn't make you Netflix; shipping features does.

What actually burned me: splitting CitizenApp's authentication into a separate service "for security." We added:

JWT validation in two places (inconsistency bugs)
Network calls to the auth service on every request
Debugging distributed auth flows took 3x longer
We eventually merged it back

The real cost of microservices is cognitive load. Every service boundary is a place to debug, secure, test, and monitor. You're paying that cost immediately.

The Right Approach

Start monolithic. Ship features. Measure what actually breaks.
Only split services that are provably constrained. Database? Compute? Deployment frequency?
Use clear contracts. If you split, define OpenAPI schemas, versioning strategy, and timeout policies upfront.
Monitor the boundary costs. If inter-service calls become a bottleneck, you split too aggressively.

For CitizenApp, we stayed monolithic for 18 months, split the ML pipeline (legitimate reason), and kept everything else together. The result: we shipped 9 AI features faster than competitors with "proper" distributed systems.

Architectural purity is cheaper than shipped features.