How to design systems for your actual scale: a practical system design tutorial

Section: Core mindset and requirements

• Start with requirements, not patterns. Clarify functional needs, latency targets, availability, data freshness, and failure modes.
• Distinguish between blocking user workflows and background tasks. If a task isn’t user-blocking, consider asynchronous handling early.
• Avoid FAANG-grade overengineering. Design for your actual traffic, budget, and team capabilities.

Section: Capacity planning and sizing

• Estimate traffic: users per second, requests per user, peak vs average, and growth trajectory.
• Define service-level objectives (SLOs) and translate them into acceptable latency (e.g., 95th percentile 200 ms) and availability (e.g., 99.9% uptime).
• Start with modest hardware and scale vertically (more CPU/RAM) before adding more machines; only then scale horizontally when needed.
• Plan for failure: assume a portion of components will fail; design with redundancy and graceful degradation.

Section: Architecture sketch for a typical everyday app

• Front-end gateway: simple load balancer with health checks, TLS termination, and basic rate limiting.
• API layer: stateless services to ease scaling; clear API contracts; idempotent operations where possible.
• Data layer: choose a primary database with sensible consistency needs; add a caching layer for hot reads; use read replicas for scale if the workload justifies it.
• Background tasks: move non-urgent work to queues or scheduled jobs to avoid blocking user requests.
• Observability: structured logs, metrics, tracing, and alerting tuned to real failures and latency outliers.

Section: Caching vs database-first thinking

• Use the cache as a speed booster, not the primary source of truth. Start by indexing and query optimization on the database.
• Cache strategies:
  • Cache-aside (lazy loading): app checks cache first; on miss, load from DB and populate cache.
  • Write-through: writes go to DB and cache atomically; keeps cache in sync.
• Cache invalidation is the hard part. Favor short TTLs for data with frequent updates; implement explicit invalidation on writes when possible.
• Use a distributed cache for multi-instance apps and CDN caching for static assets or edge-available responses.

Section: When to queue vs when to sync

• Queue non-critical tasks that don’t need immediate user feedback (e.g., image thumbnail generation, analytics aggregation).
• Keep user-critical paths synchronous when needed for correctness or user experience.
• For high-stakes updates, consider transactional patterns that preserve data integrity; avoid race conditions with proper idempotency keys.

Section: Data choices

• Relational databases for strong consistency and complex queries; NoSQL for flexible schemas and high write throughput when the data model fits.
• Evaluate CAP implications: if you need strong consistency, accept latency/availability trade-offs; if you can tolerate eventual consistency, you may gain resilience and speed.
• For simple, scalable backends, consider managed services that handle maintenance, backups, and scaling.

Section: Practical example: a small SaaS analytics dashboard

• Functional requirements: user accounts, dashboards, filters, export data.
• Non-functional: 99.9% uptime, sub-400 ms UI interactions, daily data refresh.
• Architecture: gateway -> stateless API services -> read-replica database -> cache layer for dashboards -> background jobs to compute aggregates -> notification service for alerts.
• Decisions:
  • Start with PostgreSQL for structured data; add Redis for caching frequently accessed dashboards.
  • Use queueing for export generation and large analytics tasks.
  • Implement API versioning and feature flags to reduce deployment risk.
• Outcomes: measurable speedups for hot dashboards and smoother user experience during peak times.

Section: Practical checklist

• What’s my actual traffic? Measure before planning.
• Can I optimize queries first? Add proper indexes and avoid N+1 queries.
• Is the task blocking the user? If not, queue it.
• What’s the cost of failure? Prioritize database-backed correctness for critical data.
• Can I scale vertically first? Upgrade RAM/CPU before adding servers.
• Is maintenance feasible for the team? Avoid unnecessary complexity.

Section: Realistic patterns that pay off

• Stateless services with clear boundaries to simplify scaling.
• Read replicas to handle reporting and dashboards without affecting write latency.
• A modest caching layer to reduce repeated DB hits and improve perceived performance.
• Observability baked in from day one to catch latency spikes and reliability issues early.
• Gradual, data-driven capacity growth; avoid big-bang migrations or monolithic rewrites.

Section: Simple decision framework for design choices

• If latency is the bottleneck, investigate caching and query optimization first.
• If throughput is the bottleneck, consider horizontal scaling, sharding, or read replicas.
• If data freshness is critical, prefer write-through caching and stronger consistency guarantees.
• If cost is constraining, start with the simplest possible architecture that satisfies requirements and measure before expanding.

Illustration: practical path from idea to deployment

• MVP with core features -> measure traffic and latency -> identify hotspots (API endpoints, heavy queries) -> apply targeted optimizations (indexes, caching) -> decide on queueing for non-critical tasks -> add read replicas or simple shard if needed -> introduce observability and alerting -> iterate.

Would you like me to tailor this to a specific domain (e.g., e-commerce checkout, internal tooling, or a mobile app backend) and include a concrete component diagram and a cost-focused capacity plan? If so, which domain and approximate monthly active users are you targeting?

-

Rizwan Saleem | https://rizwansaleem.co