DEV Community: Matt Frank

Day 155: GDPR Compliance Platform - AI System Design in Seconds

Matt Frank — Tue, 08 Sep 2026 13:11:36 +0000

GDPR Compliance Platform Architecture: Handling the Right to Erasure at Scale

Building a GDPR compliance platform is fundamentally about respecting user rights while managing complex, distributed data landscapes. The stakes are high: companies face fines up to 4% of global revenue for non-compliance, but more importantly, users deserve control over their personal data. The real architectural challenge isn't just storing consent preferences or logging data requests, it's orchestrating reliable erasure across dozens of microservices, databases, and backup systems that were never designed to coordinate deletion workflows.

Architecture Overview

A robust GDPR compliance platform sits at the intersection of multiple systems. The core components include a consent management engine that tracks what data users have authorized, a data subject rights service that handles access and deletion requests, a data mapping layer that catalogs where personal data lives across your infrastructure, and a workflow orchestrator that coordinates the actual erasure processes.

The platform receives requests through a central gateway, which routes them to appropriate handlers. Consent management maintains a source-of-truth for user preferences and integrates with feature flags to enforce data usage policies in real-time. The data mapping component is particularly critical, functioning as a living inventory of databases, caches, data warehouses, and third-party services that hold personal information. Without accurate mapping, deletion requests become incomplete and leave compliance gaps.

The architecture emphasizes resilience and auditability. Every deletion workflow generates an immutable audit trail documenting which systems were targeted, what status each returned, and any failures encountered. This traceability is essential for regulators and for your own compliance verification. The system uses a choreography pattern where services communicate asynchronously through event streams, reducing tight coupling and allowing independent scaling of each compliance function.

The Right to Erasure: A Deep Technical Challenge

Here's where it gets interesting. A right-to-erasure request doesn't just mean deleting records from production databases. The system implements a multi-stage deletion strategy that addresses the full data lifecycle. When a deletion request is initiated, it triggers handlers for each registered data source. Primary databases receive immediate delete commands with transaction guarantees. However, the request also cascades to read-only replicas, cache layers like Redis or Memcached, and search indices like Elasticsearch, each of which maintains stale copies of the deleted data.

The backup challenge requires particular attention. Most organizations maintain snapshots and incremental backups for disaster recovery, often stored in immutable or write-once storage. The GDPR compliance platform tracks backup manifests and coordinates with backup systems to either suppress access to backups containing the user's data or, preferably, purge the user's records from backup files where technically feasible. For backups where deletion is impossible, the platform documents these cases with regulatory justification and implements access controls to prevent accidental restoration of deleted user data. A deletion workflow isn't considered complete until all known copies, including backups within their retention windows, have been addressed or documented. This often requires integrating with infrastructure teams and establishing clear protocols between compliance and disaster recovery systems.

Watch the Full Design Process

See how we designed this system in real-time using AI, from initial requirements to complete architecture diagram:

Try It Yourself

This is Day 155 of our 365-day system design challenge. The complexity of compliance platforms demonstrates why visual architecture design matters, especially for intricate workflows spanning multiple systems.

Head over to InfraSketch and describe your system in plain English. In seconds, you'll have a professional architecture diagram, complete with a design document.

Day 154: Secrets Management - AI System Design in Seconds

Matt Frank — Mon, 07 Sep 2026 13:03:28 +0000

Imagine your database password gets compromised. You need to rotate it immediately, but your application can't afford a second of downtime. How do you pull it off? Secrets management systems solve this critical problem by providing a centralized, secure way to store, retrieve, and rotate credentials without interrupting production services. In today's world of microservices and distributed systems, this isn't optional, it's essential.

Architecture Overview

A robust secrets management system operates as the trusted vault for your organization's sensitive data. At its core, you need three key layers working in harmony: a secure storage backend (encrypted at rest), a distribution mechanism (encrypted in transit), and rotation orchestration (automated key lifecycle management). The system acts as an intermediary between applications and their credentials, ensuring no secrets ever get hardcoded into code repositories or configuration files.

The architecture typically includes several critical components. The storage layer uses encryption with hardware-backed key management, often integrated with cloud provider services or on-premises HSMs (hardware security modules). The API layer provides authenticated access for applications to request secrets, logging every access for audit compliance. The rotation engine automatically generates new credentials on a schedule, updates them in the target system, and distributes them to dependent applications. Finally, you need a policy engine that controls who can access which secrets and under what conditions, implementing the principle of least privilege throughout.

The design decisions here reflect real-world constraints. Rather than storing secrets in your application configuration or environment variables, the system makes applications request credentials at runtime. This means compromised code can't leak hardcoded secrets. By centralizing rotation logic, you eliminate the fragmented, manual processes that lead to forgotten password updates and security gaps. Adding audit trails for every secret access creates accountability and makes compliance audits straightforward.

Design Insight: Zero-Downtime Password Rotation

Here's where things get clever. You rotate a database password without downtime by implementing a dual-credential approach. The database maintains both the old and new passwords simultaneously during the rotation window. Your application receives the new password from the secrets manager before rotation completes, allowing it to begin using it immediately. Meanwhile, other services still connected with the old password continue operating without interruption. Once all applications have been updated and verified, the rotation engine revokes the old password. This staged approach requires tight coordination between the secrets manager and your applications, but it's the gold standard for production environments where every second of downtime has real costs.

Watch the Full Design Process

We built this architecture in real-time using AI-powered diagram generation. Watch as we explore how secrets flow through the system, how rotation gets orchestrated, and how we handle the tricky edge cases:

Watching the architecture come together visually really highlights why secrets management matters, especially when you see how many components need to communicate securely.

Try It Yourself

Want to design a secrets management system tailored to your infrastructure? Head over to InfraSketch and describe your system in plain English. In seconds, you'll have a professional architecture diagram, complete with a design document. Whether you're building for compliance, scale, or security, let InfraSketch help you visualize your secrets strategy.

Day 153: Zero Trust Architecture - AI System Design in Seconds

Matt Frank — Sun, 06 Sep 2026 13:04:26 +0000

Traditional network security operates on a perimeter defense model: build strong walls around your infrastructure and trust everything inside. But modern threats, cloud adoption, and remote work have exposed this approach as fundamentally broken. Zero Trust Architecture eliminates implicit trust entirely, requiring every user, device, and service to prove its identity and authorization before accessing any resource, regardless of location. This shift from "trust but verify" to "never trust, always verify" represents one of the most significant security paradigm changes in decades.

Architecture Overview

Zero Trust networks are built on continuous verification at every layer. The core principle divides your infrastructure into micro-segments, where each segment functions as its own isolated zone with strict access controls. Rather than a single perimeter, you create multiple enforcement points. Every request, whether from a user or a service, must be authenticated and authorized based on context like device health, user identity, location, time of access, and behavioral patterns. This context-aware approach means that even if an attacker gains credentials, they can't simply move laterally through your network.

The typical Zero Trust architecture includes several interconnected components working in harmony. An identity provider manages user and service authentication, often supplemented by multi-factor authentication. A policy enforcement point acts as your gatekeeper, evaluating every request against your access policies. Network segmentation tools isolate workloads and services, preventing lateral movement. Continuous monitoring and analytics feed data back into the system, allowing policies to adapt based on real-time threat detection. Device posture checking ensures that only healthy, compliant endpoints gain access. These components create a closed loop where security decisions are made dynamically, not statically.

The design philosophy prioritizes assumption of breach. Rather than asking "how do we prevent attackers from entering," Zero Trust asks "how do we operate safely assuming attackers are already inside." This mindset fundamentally changes how you architect systems. You segment aggressively, minimize trust relationships, monitor everything, and make every access decision provable and auditable.

Design Insight: Service-to-Service Communication

Here's where Zero Trust gets truly interesting at the infrastructure level: how do you handle the thousands of internal service calls happening every second without creating administrative nightmares? The answer lies in mutual TLS (mTLS) combined with service meshes and fine-grained policies. Every service authenticates itself to every other service using cryptographic certificates, not just network position. A service mesh like Istio or Linkerd intercepts all traffic between services, enforcing encryption and identity verification automatically. Policies define exactly which services can communicate with which other services, and access is logged for audit trails. This approach means a compromised service can't freely pivot to others, and you maintain the same "never trust" principle internally that you enforce at the network edge.

Watch the Full Design Process

See how these concepts come together in real-time as we generate a complete Zero Trust architecture diagram with comprehensive design documentation:

Try It Yourself

Ready to design your own Zero Trust architecture? Head over to InfraSketch and describe your system in plain English. In seconds, you'll have a professional architecture diagram, complete with a design document.

This is Day 153 of our 365-day system design challenge. Each day, we're exploring critical architecture patterns that shape modern infrastructure. Zero Trust isn't just a security trend, it's becoming a baseline expectation for enterprise systems. Start experimenting with these concepts today.

Day 152: Vulnerability Scanner - AI System Design in Seconds

Matt Frank — Sat, 05 Sep 2026 13:33:18 +0000

Security teams are drowning in vulnerability alerts. Without smart prioritization, organizations waste resources patching low-risk issues while critical vulnerabilities slip through the cracks. A well-designed vulnerability scanner doesn't just find problems, it surfaces the threats that matter most, turning noise into actionable intelligence.

Architecture Overview

A modern vulnerability scanning platform operates across four distinct scanning layers, each targeting different attack surfaces. The code scanner analyzes source repositories for secrets and insecure patterns, the container scanner inspects Docker images for vulnerable packages, the infrastructure scanner audits cloud configurations for misconfigurations, and the dependency scanner tracks third-party libraries against known vulnerability databases. These layers feed into a central orchestration engine that coordinates timing, handles authentication, and manages computational resources across distributed workers.

The architecture hinges on a critical insight: scanning and remediation are separate concerns. Raw scan results flow into a processing pipeline that enriches findings with contextual data before they reach the prioritization engine. This pipeline enriches each vulnerability with exploitability metrics, CVSS scores, and asset criticality ratings. By decoupling the scanning workers from the intelligence layer, the system scales horizontally as scanning demands grow without bottlenecking the analysis pipeline.

The platform exposes findings through three key interfaces: a real-time dashboard for urgent threats, scheduled report generation for compliance, and API access for integration with ticketing systems and incident response tools. A feedback loop is essential here, allowing security teams to track which vulnerabilities get patched and which are accepted as business risks. This historical data trains the prioritization engine to improve over time, making the system smarter with every decision.

Designing for Scale and Signal

The Prioritization Challenge

Finding vulnerabilities is easy. Finding the right vulnerabilities to fix is hard. A typical enterprise might scan hundreds of applications daily, generating tens of thousands of findings. Most are low-risk, but a handful could lead to data breaches. The prioritization engine solves this by applying a multi-factor scoring model that considers severity, exploitability, asset importance, and environmental context.

The key is context awareness. A critical vulnerability in a rarely-used staging environment deserves less attention than a medium-severity bug in a customer-facing service. The system maintains an asset inventory tied to business criticality, allowing the prioritization engine to downrank findings in low-impact services and surface those affecting production systems. Additionally, exploitability data from threat intelligence feeds helps distinguish between theoretical vulnerabilities and those actively targeted in the wild. This multi-dimensional scoring transforms a firehose of alerts into a prioritized risk register.

Watch the Full Design Process

Want to see how this architecture came together? I designed this system in real-time using AI assistance, capturing the iterative process of turning requirements into a scalable architecture. Watch how each component fits together and how we solved the prioritization challenge:

Try It Yourself

This is Day 152 of a 365-day system design challenge, and I'm using InfraSketch to generate architecture diagrams in real-time. The platform turns natural language descriptions into professional system designs in seconds, complete with component relationships and design documentation.

Ready to design your own system? Head over to InfraSketch and describe your system in plain English. In seconds, you'll have a professional architecture diagram, complete with a design document.

Day 151: Data Loss Prevention - AI System Design in Seconds

Matt Frank — Fri, 04 Sep 2026 13:03:23 +0000

Every day, organizations lose millions to data breaches that slip past traditional security systems. Most data loss prevention (DLP) solutions focus on text-based detection, missing the visual threats hidden in screenshots, PDFs, and images. Today's advanced DLP architectures need multi-layered intelligence to catch sensitive data regardless of format, making this one of the most critical security challenges in modern infrastructure.

Architecture Overview

A robust DLP system operates as a distributed detection network sitting at three critical perimeters: email gateways, cloud storage APIs, and endpoint agents. The architecture doesn't rely on a single detection method but instead orchestrates multiple specialized processors working in parallel. When content flows through any channel, it's immediately tagged with metadata, scanned by content analyzers, and cross-referenced against policy databases before a decision is made to allow, quarantine, or block the data.

The system's core strength lies in its separation of concerns. Content ingestion layers normalize data from diverse sources (email attachments, cloud uploads, clipboard operations) into a standardized format. This feeds into a rules engine that evaluates both signature-based patterns and behavioral anomalies. For example, detecting a sudden spike in file uploads containing customer records triggers behavioral analysis, even if individual files pass content scans. The architecture also includes a machine learning pipeline that continuously learns from detected threats and false positives, improving detection accuracy over time.

A critical design decision is the use of queuing systems to prevent bottlenecks. Scanning large files or images can be compute-intensive, so the system decouples content reception from analysis using asynchronous workers. This ensures user experience isn't degraded while maintaining security rigor. Additionally, the system maintains an audit trail of all scanning decisions, enabling security teams to investigate incidents and tune policies based on real-world patterns.

Detecting Sensitive Data in Images and Screenshots

Here's where modern DLP gets sophisticated. Visual content requires a fundamentally different approach than text extraction. The system employs optical character recognition (OCR) to convert images into machine-readable text, but OCR alone isn't enough because handwritten documents, logos, and structured layouts carry meaning that plain text misses. The architecture incorporates computer vision models trained to recognize patterns specific to sensitive documents: the visual structure of credit card layouts, the characteristic formatting of social security numbers, or even the watermark patterns on classified documents.

More advanced implementations use multi-modal AI models that understand both the textual and visual context simultaneously. When an image contains text saying "CONFIDENTIAL" alongside financial data, the system weighs both the semantic content and the visual indicators of sensitivity. The system also maintains image hashing to detect when sensitive images are being copied or cropped, catching attempts to obscure detection. All image analysis happens in isolated sandbox environments, ensuring privacy while maintaining detection capability.

Watch the Full Design Process

See how this entire DLP architecture was designed in real-time using AI-powered diagram generation:

Try It Yourself

This is Day 151 of our 365-day system design challenge, and we're constantly pushing the boundaries of what's possible with architecture visualization. Want to design your own DLP system or explore a different security architecture?

Head over to InfraSketch and describe your system in plain English. In seconds, you'll have a professional architecture diagram, complete with a design document. Whether you're building enterprise security infrastructure or learning system design, InfraSketch makes it effortless to visualize and iterate on your ideas.

Day 150: SIEM Platform - AI System Design in Seconds

Matt Frank — Thu, 03 Sep 2026 13:04:23 +0000

SIEM Architecture: Detecting Threats Without Drowning in False Alarms

Modern security teams face an overwhelming deluge of alerts. A typical enterprise generates millions of security events daily, and without intelligent filtering, analysts spend more time chasing false positives than hunting real threats. A well-designed SIEM platform doesn't just collect and correlate events, it learns to distinguish genuine security incidents from the noise of normal business operations, transforming raw data into actionable intelligence.

Architecture Overview

A production SIEM platform operates as a distributed system with clearly separated concerns. At its core, you have multiple data sources feeding into centralized collection points: network devices, servers, applications, cloud services, and endpoints all stream their logs and events through agents or syslog protocols. These raw events flow into an ingestion layer that performs initial parsing, normalization, and enrichment. This is critical because a login event from Active Directory looks different from one captured by a web application, yet they represent the same security concept.

Once normalized, events move through a correlation engine, the intelligent heart of the system. This component applies rules and machine learning models to identify patterns that indicate compromise or policy violations. Think of it as teaching the system to recognize attack signatures, privilege escalation attempts, and anomalous behavior. The correlation layer doesn't work in isolation, though. It draws context from multiple enrichment sources: asset inventories, threat intelligence feeds, user directories, and baseline behavior models. This contextual awareness is what separates a reactive alert system from a true threat detection platform.

Behind these real-time components sits a scalable storage layer built for both immediate querying and long-term compliance archiving. The architecture uses time-series databases for metrics and analytics, traditional storage for structured logs, and object storage for compliance retention. A separate alerting and response subsystem consumes correlation output, manages alert queues, applies escalation policies, and integrates with ticketing systems and incident response tools. The entire platform needs monitoring dashboards that let security teams visualize network security posture at a glance.

Design Insight: Reducing False Positives Through Behavioral Baselining

The false positive problem is fundamentally about context. A user downloading 500 files at 3 AM looks suspicious until you learn that user is a data analyst running their nightly ETL job. A traditional SIEM would fire an alert every time. An intelligent SIEM builds behavioral baselines for users, machines, and applications, then flags deviations rather than absolute rules. The system learns what "normal" looks like for your specific environment by analyzing historical activity patterns over weeks or months. When you layer this with time-aware rules that understand business cycles, exception handling for known legitimate activities, and peer-group analysis that compares user behavior against similar users in the organization, you dramatically reduce false positives. Additionally, machine learning models can score alerts by confidence level, allowing analysts to focus on high-confidence threats first. The key architectural decision is treating the SIEM as a learning system that adapts to your environment rather than a rigid ruleset that fires on static indicators.

Watch the Full Design Process

See how this entire architecture comes together in real-time as an AI generates a complete SIEM design:

Try It Yourself

This is Day 150 of a 365-day system design challenge. Ready to design your own security platform? Head over to InfraSketch and describe your system in plain English. In seconds, you'll have a professional architecture diagram, complete with a design document.

Day 149: WAF (Web Application Firewall) - AI System Design in Seconds

Matt Frank — Wed, 02 Sep 2026 13:03:13 +0000

Every second, web applications face thousands of malicious requests, from SQL injections to DDoS attacks. A Web Application Firewall (WAF) stands between your users and these threats, intelligently inspecting traffic and enforcing security policies before attacks ever reach your application. Understanding how a WAF works is essential for anyone designing secure systems.

Architecture Overview

A Web Application Firewall operates as a reverse proxy or inline filter that examines HTTP requests against a set of predefined and custom rules. The core architecture typically consists of four key layers: request ingestion, parsing and normalization, rule evaluation, and action execution. When a request arrives, the WAF first normalizes it by decoding various encodings, handling Unicode transformations, and standardizing formats. This prevents attackers from obfuscating their payloads.

The rule evaluation engine is where the real intelligence happens. The WAF maintains multiple rule sets, including built-in signatures for known attack patterns (like OWASP Top 10 vulnerabilities), rate limiting policies, and custom business logic rules. Rules can be simple string matches or complex expressions that evaluate multiple request attributes simultaneously. A request might be evaluated against dozens of rules in parallel, checking headers, body content, query parameters, cookies, and URL paths. The system assigns a threat score or applies immediate actions like blocking, logging, or rate limiting.

The final component is the action engine, which decides what happens to a request based on rule matches. Common actions include allowing the request through, blocking it with an error response, rate limiting the client, challenging with a CAPTCHA, or logging for manual review. Modern WAFs also include feedback mechanisms that learn from false positives and security analysts can refine rules over time. This flexibility makes WAFs adaptable to evolving threats while minimizing false positives that frustrate legitimate users.

Design Insight: Distinguishing Large File Uploads from Injection Attacks

This is where WAF design becomes particularly nuanced. A legitimate file upload might be several hundred megabytes, while a malicious request body injection attack could be just kilobytes but contain encoded SQL or script payloads. The WAF doesn't rely on size alone. Instead, it examines content-type headers, checks if the request declares itself as multipart form data, analyzes the actual content structure, and applies different rule sets based on context.

For a legitimate upload, the WAF validates that the request follows proper multipart boundaries, contains file metadata headers, and lacks suspicious patterns in non-file portions. For potential injection attacks, it inspects the request body itself for dangerous patterns, encoding sequences, or script signatures, regardless of size. Context matters enormously. A request claiming to be a JSON API call shouldn't have multipart boundaries, while a file upload missing proper boundary markers is immediately suspicious. Additionally, WAFs track upload history per user, applying behavioral analysis to detect anomalies like sudden massive uploads from unusual locations or at unusual times. This layered approach ensures large files pass through while tiny, malicious payloads get caught.

Watch the Full Design Process

See how this architecture comes together in real-time with AI-powered diagram generation:

Try It Yourself

Ready to design your own security systems? Head over to InfraSketch and describe your system in plain English. In seconds, you'll have a professional architecture diagram, complete with a design document.

This is Day 149 of a 365-day system design challenge. Each day brings new architectural patterns and real-world challenges to solve.

Day 148: API Gateway Security - AI System Design in Seconds

Matt Frank — Tue, 01 Sep 2026 13:04:13 +0000

API Gateway Security: Your First Line of Defense

Every request entering your system passes through a chokepoint, and what happens there determines whether your application thrives or crashes. An API gateway sits at this critical junction, filtering threats, managing load, and validating requests before they ever touch your backend services. Without proper gateway architecture, you're essentially leaving your front door unlocked while hoping intruders behave themselves.

Architecture Overview

An effective API gateway isn't just a single component, it's an orchestrated system of defensive layers working in concert. The gateway receives incoming requests and immediately processes them through an authentication layer that verifies user identity and checks credentials against your identity provider. Simultaneously, a rate limiting module tracks request frequency per client, preventing any single user or malicious actor from overwhelming your system. These first-line defenses happen in milliseconds, before any real business logic gets involved.

Behind authentication and rate limiting sits a request validation engine that ensures incoming data matches your API contract. This component checks headers, payload structure, and content types, rejecting malformed requests before they waste backend resources. Running validation at the gateway means your downstream services can assume all data is well-formed and safe, reducing redundant validation code scattered across multiple microservices.

The threat detection system continuously monitors patterns and anomalies in incoming traffic. It looks for suspicious behaviors like SQL injection attempts, unusual header combinations, or request patterns that deviate from normal usage. This layer doesn't just block obvious attacks, it also learns and adapts, flagging edge cases for security teams to investigate. When a potential threat is detected, the gateway can immediately respond with an error, log the incident for audit purposes, and potentially trigger alerting systems.

All these components feed into a central logging and monitoring pipeline that gives you complete visibility into what's happening at your API boundary. Every decision the gateway makes, from authentication success to rate limit rejections, creates a valuable audit trail for compliance and incident response.

Design Insight: Handling Slow Backends

What happens when your backend suddenly starts responding slowly? A well-designed API gateway implements timeout policies and circuit breakers to prevent cascading failures. When a backend service begins taking longer than expected to respond, the gateway doesn't patiently wait forever. Instead, it cuts the connection after a configured threshold, returns an error response to the client, and tracks this failure.

After a certain number of failures within a time window, the circuit breaker "opens," meaning the gateway stops sending new requests to that backend entirely. It serves cached responses when possible, or returns a graceful degradation message to clients. This prevents the gateway itself from becoming a bottleneck of stuck connections waiting for a struggling backend to recover. Meanwhile, the backend can breathe and recover without being crushed by incoming requests. When the backend stabilizes, the circuit breaker gradually returns to normal operation through a "half-open" state, testing the waters with a few requests before fully resuming traffic.

Watch the Full Design Process

Want to see how this entire architecture comes together in real-time? Watch the AI-powered design process on your preferred platform:

Try It Yourself

Ready to design your own API gateway architecture? Head over to InfraSketch and describe your system in plain English. In seconds, you'll have a professional architecture diagram, complete with a design document. Whether you're building your first gateway or optimizing an existing one, letting AI generate your initial architecture is a game-changer.

This is Day 148 of the 365-day system design challenge, and every day brings new opportunities to explore critical infrastructure patterns. Your gateway security posture directly impacts your entire system's resilience, so invest the time to get it right.

Day 147: Certificate Authority - AI System Design in Seconds

Matt Frank — Mon, 31 Aug 2026 13:04:24 +0000

Managing certificates across dozens of microservices sounds straightforward until 3 AM when a service goes down mid-renewal and your entire infrastructure grinds to a halt. An internal Certificate Authority (CA) with automated lifecycle management eliminates this nightmare by centralizing certificate issuance, renewal, and revocation across your entire infrastructure. This is one of those system design challenges that separates "it works" from "it works reliably at scale."

Architecture Overview

A robust internal CA system consists of several interconnected layers. At the core, you have the CA itself, which signs and manages the certificate lifecycle. This connects to a certificate storage layer that keeps issued certificates safe and accessible, an enrollment service that handles initial certificate requests from services, and a renewal engine that proactively tracks expiration dates and reissues certificates before they expire. Each component needs to communicate securely, track state accurately, and handle failures gracefully.

The key design decision here is automation over manual intervention. Rather than requiring operators to manually trigger renewals or handle revocations, the system continuously monitors certificate validity periods and pushes renewals to services automatically. A distributed configuration management system ensures that each service receives its updated certificate without service interruption. This architecture also separates concerns: the CA itself remains isolated and secure, while renewal workers, enrollment handlers, and revocation coordinators operate independently and can scale horizontally.

Security is baked in from the start. The CA stores its private key in a hardware security module or encrypted vault, never exposed to the network directly. Services authenticate to the enrollment service using existing credentials or mutual TLS with bootstrap certificates. Audit logging captures every certificate operation for compliance and forensics. The system also maintains a certificate revocation list, published regularly so services can validate that other certificates haven't been compromised.

Design Insight: Handling Temporarily Unreachable Services

Here's where things get interesting. When a service is temporarily unreachable during renewal, you can't just fail and wait for the next retry. The renewal engine needs a sophisticated retry strategy with exponential backoff that respects both certificate expiration deadlines and service availability patterns.

The elegant solution uses a tiered approach. First, the renewal engine attempts direct connection with immediate retries over minutes. If that fails, it escalates to a distributed message queue where renewal tasks wait for the service to reappear, with backpressure to prevent queue explosion. Critically, the system maintains a safety buffer: renewals are triggered well before expiration (often 30 days early) so temporary outages don't become emergencies. If a service remains unreachable as expiration approaches, the system can either pre-stage the certificate for immediate installation when the service recovers, or trigger alerting to operators with clear context about what's needed. Some teams even implement grace periods where the CA continues trusting recently-expired certificates from unavailable services, buying time for recovery without breaking authentication chains.

Watch the Full Design Process

This architecture didn't emerge from theory alone. We built it live, starting with plain English requirements and evolving it through real design decisions. Watch how a Certificate Authority system develops from concept to production-ready blueprint:

Try It Yourself

Ready to design your own Certificate Authority system? Head over to InfraSketch and describe your system in plain English. In seconds, you'll have a professional architecture diagram, complete with a design document. Whether you're managing 10 services or 1000, this tool helps you think through the layers and connections that matter before you write a single line of infrastructure code.

This is Day 147 of the 365-day system design challenge. Each day explores a new architecture pattern in real-time.

Day 146: Session Management - AI System Design in Seconds

Matt Frank — Sun, 30 Aug 2026 13:04:24 +0000

Session Management in Distributed Systems

As your application scales across multiple services and regions, managing user sessions becomes exponentially more complex. A single user might have sessions across mobile apps, web browsers, and API clients, all requiring instant invalidation when security events occur. This is where a well-designed distributed session management system becomes your safeguard against unauthorized access and data breaches.

Architecture Overview

A robust distributed session management system sits at the intersection of security and scalability, requiring careful orchestration between multiple components. At its core, you'll have a Session Store (typically Redis or a similar fast data store) that maintains active sessions with their metadata and expiration times. This sits behind a Session Manager service that handles creation, validation, and lifecycle operations. Additionally, you need an Authentication Service that generates session tokens, an Event Bus for broadcasting invalidation events across services, and a User Service that manages user accounts and password changes.

The architecture follows a publish-subscribe pattern where critical events like password changes trigger cascading invalidation across the entire session ecosystem. When a user initiates a password change, the User Service doesn't just update credentials, it broadcasts an event that the Session Manager consumes immediately. Each microservice that validates sessions listens to these events, ensuring no orphaned sessions persist in distributed caches. This event-driven approach eliminates the race condition problem you'd encounter with synchronous calls, where some services might still accept an invalidated session while others reject it.

Token design plays a crucial role in this architecture. Rather than storing all session data server-side, many distributed systems use signed JWT tokens with short expiration windows paired with refresh tokens that have longer lifespans. This reduces Session Store load and allows offline validation in some scenarios. However, the invalidation challenge becomes more critical, since a token remains cryptographically valid until its expiration time passes. The Session Manager maintains a revocation list for these tokens, checking it during validation to catch any sessions that should have been terminated early.

The Password Change Scenario

When a user changes their password, invalidating all active sessions requires atomic coordination across your distributed system. Here's how it flows: the User Service receives the password change request and, after updating credentials, publishes a "UserPasswordChanged" event with the user ID. The Session Manager subscribes to this event and immediately queries the Session Store for all active sessions belonging to that user. It then removes each session record and publishes a "SessionsRevoked" event containing the user ID and session IDs. Every service that caches session validation results receives this event and purges their local caches, preventing stale session acceptance. For JWT-based systems, the revoked session IDs get added to a temporary revocation list (stored with a TTL matching the JWT expiration time) that validators check during authentication. This multi-layer invalidation ensures that even if a user's token was just validated microseconds before the password change, the next request will fail because that session now appears in the revocation list.

Watch the Full Design Process

Watch how these concepts come together in real-time as we build this architecture from scratch. See how each component fits into the larger system and understand the tradeoffs between consistency, performance, and operational complexity.

Try It Yourself

Want to design your own session management system or tackle a different architecture challenge? Head over to InfraSketch and describe your system in plain English. In seconds, you'll have a professional architecture diagram, complete with a design document. Whether you're working through the 365-day system design challenge or solving a real-world problem, let AI help you visualize and refine your ideas faster than traditional whiteboarding ever could.

Day 145: API Key Management - AI System Design in Seconds

Matt Frank — Sat, 29 Aug 2026 13:03:19 +0000

API keys are the gatekeepers of your infrastructure, and when one gets compromised, every second counts. A poorly designed key management system can force you to choose between security and availability, but a well-architected one handles revocation instantly across your entire service fleet. Today we're exploring a system that makes this possible without breaking a sweat.

Architecture Overview

An effective API key management system sits at the intersection of security, performance, and operational simplicity. The core components work together like a distributed security checkpoint: a key generation service that creates cryptographically secure tokens with embedded metadata, a scoping engine that defines what each key can access, a central revocation cache that acts as a high-speed blocklist, and usage tracking infrastructure that monitors every request in real-time.

The architecture typically revolves around three critical layers. The first is the control plane, where administrators and services manage keys through a secure API. This is where key creation happens, scopes are assigned, and rotation policies are defined. The second is the data plane, where your services validate incoming API keys before processing requests. The third is the intelligence layer, which tracks usage patterns, detects anomalies, and automatically triggers alerts when something looks suspicious.

One of the biggest design challenges is balancing centralized control with distributed performance. Services can't afford to check a remote database on every single request, so the system needs to cache key information locally while still respecting real-time revocation signals. This tension drives the architecture's most interesting patterns.

How Instant Revocation Works Without Downtime

Here's where things get elegant. When a compromised key is detected, the system doesn't rely on services polling a database for updates. Instead, it uses a multi-layered invalidation strategy. The revocation immediately updates a distributed cache layer (think Redis cluster or similar) that services query with sub-millisecond latency. Simultaneously, the system publishes invalidation events through a message broker to all connected services, triggering their local cache invalidation.

Services maintain a local in-memory cache of valid keys with a short time-to-live (usually 30-60 seconds). When a key is revoked, these services receive a push notification rather than waiting for the cache to expire naturally. This combination of push notifications for critical events and pull-based validation on regular intervals creates a system where revocation typically propagates within milliseconds. The key insight is that you don't need 100% of services to know about the revocation instantly. You need the vast majority to know within seconds, and any request using the compromised key to fail gracefully. Because the cache is distributed and the revocation event is broadcast immediately, there's effectively no downtime, just a controlled failure of bad requests.

Watch the Full Design Process

Want to see how we built this architecture from scratch? I recently demonstrated the complete design process using AI in real-time. You can watch the full walkthrough on your preferred platform:

The video shows how these components interact, the message flows during a revocation event, and the trade-offs we made between consistency and availability.

Try It Yourself

This is Day 145 of the 365-day system design challenge, and we're building momentum. If you want to design your own API key management system or any other architecture quickly, head over to InfraSketch and describe your system in plain English. In seconds, you'll have a professional architecture diagram, complete with a design document. No more blank whiteboards or hours spent in Figma. Let AI help you explore design possibilities faster.

Day 128: Load Balancer - AI System Design in Seconds

Matt Frank — Wed, 12 Aug 2026 13:07:54 +0000

Load balancing is the invisible hero keeping your application responsive when traffic spikes. Without intelligent distribution across multiple servers, even the most powerful backend would crumble under user demand. Today, let's explore a production-grade architecture that handles complexity at multiple layers, ensuring both performance and reliability.

Architecture Overview

A multi-layer load balancing system operates at two distinct levels, each solving different problems. Layer 4 (L4) load balancers work at the transport level, making routing decisions based on IP protocol data like TCP and UDP. Layer 7 (L7) load balancers operate at the application level, understanding HTTP headers, cookies, and request content. This two-tier approach gives you the raw speed of L4 for initial distribution while enabling intelligent routing decisions at L7 based on actual application behavior.

The system typically flows like this: incoming traffic hits the L4 load balancer first, which performs fast, connection-level distribution across multiple L7 instances. These L7 balancers then apply sophisticated routing rules, such as path-based routing (sending API requests to one backend cluster and web requests to another) or content-based decisions. Session affinity comes into play here, ensuring that requests from the same user stick to the same backend server, which is crucial for stateful applications that rely on in-memory sessions.

Health checking runs continuously across this entire architecture, probing backends at regular intervals to detect failures. Beyond simple binary checks (up or down), modern systems implement graduated health assessment, categorizing servers as healthy, degraded, or unhealthy. When a server fails health checks, it's immediately removed from the rotation. This multi-layer design provides redundancy at every stage, so a single point of failure never takes down your entire service.

Design Insight

Here's where things get interesting: what happens when a backend server starts responding slowly but still technically passes health checks? This is a subtle but critical scenario in real production systems. Most health checks are binary, returning success if the server responds within a threshold (often 5-10 seconds). However, a server can respond just fast enough to pass checks while still degrading user experience with 2-3 second response times.

Advanced load balancers handle this through adaptive load shedding and performance-based weighting. Instead of treating all healthy servers equally, the L7 balancer monitors actual response times and latency percentiles. Servers exceeding latency thresholds get fewer new connections, while faster servers absorb more traffic. Some systems implement circuit breaker patterns that temporarily reduce traffic to a degraded server, giving it breathing room to recover. The key insight is that health checking should evolve beyond availability into performance assessment, using metrics like response time, error rates, and resource utilization to make routing decisions. Tools like InfraSketch help visualize these complex decision flows, making it easier to communicate health strategy across your team.

Watch the Full Design Process

See how this architecture comes together in real-time:

Try It Yourself

Ready to design your own load balancing system? Head over to InfraSketch and describe your system in plain English. In seconds, you'll have a professional architecture diagram, complete with a design document. Whether you're optimizing for high throughput, low latency, or graceful degradation, you can iterate on your design instantly and share it with your team. This is Day 128 of our 365-day system design challenge, so keep building!