Every engineer has a mental model of what "hard problems" look like in large-scale systems. They think of recommendation engines, real-time video encoding, or distributed databases that span continents. Identity infrastructure rarely makes the list. And that's exactly why it's the hardest problem in the building — it's invisible until it breaks, and when it breaks, everything breaks.
I've built identity systems that serve billions of users across multiple products. Along the way I've learned that identity isn't a service you bolt on. It's the connective tissue of your entire platform. Get it right and nobody notices. Get it wrong and you're looking at cascading auth failures, account takeovers at scale, and the kind of trust erosion that takes years to repair.
This post is a practitioner's account of what actually works, what doesn't, and what I wish someone had told me when I started.
Why Identity Is Structurally Different from Other Infrastructure
Most backend systems can tolerate brief inconsistency. A social feed that's two seconds stale is fine. A search index that hasn't caught the last write is acceptable. Identity doesn't get that grace period.
When a user changes their password, revokes a session, or enables two-factor authentication, that change must propagate instantly and universally. If there's a ten-second window where an old session token still works after a password reset, you've created a ten-second window for an attacker who already has that token. Identity is one of the few domains where consistency isn't a nice-to-have — it's a security boundary.
At the same time, identity sits in the critical path of every single request. Every API call, every page load, every real-time message begins with "who is this user, and are they allowed to do this?" If your identity layer goes down, your entire platform goes down. That means you also need extreme availability.
This is the fundamental tension: you need strong consistency and extreme availability, which anyone who's read the CAP theorem knows is a painful place to live. The real engineering isn't choosing one over the other. It's designing around the tension so that the tradeoffs are invisible to users and safe for the platform.
The Core Design Challenges
Federated Identity Across Products
Single-product companies have it easy. One user table, one auth flow, one session model. But the moment you operate a family of products — each with its own history, its own user base, its own auth requirements — identity becomes a federation problem.
The naive approach is to build a central identity service and migrate everyone onto it. I've watched this approach fail more than once. Products have different trust levels, different session lifetimes, different regulatory environments. A messaging product handles identity differently than an e-commerce platform or a streaming service.
What works is a layered identity model: a core identity graph that establishes who a person is, with product-specific identity projections that handle how that person is represented and authenticated in each context. The core layer owns the canonical user record, credential storage, and cross-product linkage. The projection layer owns product-specific sessions, scopes, and consent surfaces.
This separation sounds clean on a whiteboard. In practice, the hard part is the boundary. Where does the core layer end and the product layer begin? The answer shifts over time, and you need to design for that migration path, not just the current state.
Token Lifecycle at Billions Scale
A typical large-scale platform might have tens of billions of active tokens at any given moment — access tokens, refresh tokens, device tokens, API tokens, OAuth grants. Each has its own lifetime, revocation semantics, and blast radius if compromised.
The mistakes I see most often:
Over-reliance on long-lived tokens. Engineers default to long expiration times because short-lived tokens mean more refresh traffic. But long-lived tokens are long-lived attack surfaces. The right pattern is short-lived access tokens (minutes, not hours) backed by longer-lived refresh tokens with rotation on every use. If a refresh token is used twice, you know it's been stolen — kill the entire token family.
Revocation as an afterthought. It's easy to issue tokens. It's incredibly hard to revoke them instantly across a globally distributed system. Maintaining a global revocation list that every service checks on every request doesn't scale. What does scale is a combination of short token lifetimes (so most revocations are "wait for expiry"), a lightweight real-time revocation channel for critical events (password changes, account compromises), and epoch-based invalidation where you can bump a user's "auth epoch" and instantly invalidate everything issued before it.
Treating all tokens as equal. A token that lets you read your own profile and a token that lets you modify account security settings should not have the same architecture. Step-up authentication — requiring a fresh, stronger auth challenge before high-risk operations — is essential. But it needs to be baked into the token model, not layered on top.
Zero Trust in a Multi-Service World
The old model was simple: authenticate at the edge, then trust everything inside the network perimeter. That model is dead, and it should be. Any system with hundreds of internal services has too large an internal attack surface to trust implicitly.
In practice, zero trust for identity means every service-to-service call carries user context and is independently verified. The user's identity and their authorization scopes travel with the request, and each service makes its own authorization decision based on that context.
The engineering challenge is doing this without destroying latency. If every service makes a fresh call to your identity service on every request, you've built a system where your identity service's p99 latency is your platform's p99 latency. The solution is cryptographic verification at the edge: signed, self-contained tokens (like JWTs, but carefully — see below) that services can verify locally without a network call, combined with an asynchronous revocation channel for the cases where you need to pull the rug out.
A word of caution on JWTs specifically: they've become the default answer, but they're not a free lunch. They can't be revoked without infrastructure to support it. They bloat quickly when you stuff too many claims in. And the ecosystem of JWT libraries has a terrible security track record — algorithm confusion attacks, none-algorithm acceptance, key confusion between signing and encryption. If you use JWTs, treat the library choice and configuration as a security-critical decision, not a dependency you pull in and forget about.
Patterns That Work (and Ones That Break)
Hierarchical identity models work. Rather than a flat user → session mapping, model identity as a tree: person → accounts → sessions → tokens. This gives you natural revocation cascades (disable an account, all its sessions die; kill a session, all its tokens die) and clean multi-account support.
Progressive authentication works. Not every action requires the same level of assurance. Let users browse with a session cookie, but require a fresh password or biometric before they change their email or download their data. This reduces friction for low-risk flows while maintaining security for high-risk ones. The key is building this into your identity model from the start — retrofitting progressive auth onto a system that treats all sessions as equal is painful.
Global session management works, eventually. Letting a user see and revoke all their active sessions across all devices sounds simple. Building it at scale, across multiple products, with real-time accuracy, is a multi-year infrastructure project. But it's table stakes for user trust and regulatory compliance (GDPR's right to withdrawal of consent, for instance). Start early.
What breaks: monolithic auth services. I've seen teams build a single auth service that handles registration, login, session management, OAuth, SAML, passwordless auth, and fraud detection. It becomes the most critical, most complex, and most feared service in the entire stack. No one wants to deploy to it. No one fully understands it. Break it up. Separate the credential verification path from the session management path from the token issuance path. They have different scaling characteristics, different failure modes, and different change velocities.
What breaks: treating identity migration as a one-time project. Identity infrastructure is never done. Credential standards change (passwords → OTP → push notifications → passkeys). Regulatory requirements shift. New products join the platform. Design for continuous migration, not a big bang cutover.
What Most Engineers Get Wrong
The biggest mistake is treating identity as an application feature rather than a platform primitive. When identity is owned by a product team, you get auth flows that are optimized for that product's onboarding funnel but impossible to extend to new products. When identity is a platform, you get consistent security properties everywhere, even in products that haven't been built yet.
The second mistake is underinvesting in the unglamorous parts. Credential rotation, token hygiene, session cleanup, audit logging — these aren't exciting. They're also the things that determine whether you survive a security incident or end up in the news.
The third is ignoring the human side. Identity systems are ultimately about people, and people do unpredictable things. They share accounts. They forget passwords on devices they no longer own. They pass away, and someone needs to access their account. They're minors who age into adulthood and need their data handling to change. Every identity system I've built has eventually needed to handle cases that no spec anticipated, and the systems that handled them gracefully were the ones designed with extensibility and human judgment baked in.
Where Identity Infrastructure Is Heading
Passkeys are the most significant shift in consumer authentication in a decade. They eliminate passwords entirely, replacing them with device-bound cryptographic credentials that are phishing-resistant by design. The migration is messy — you need to support passwords and passkeys simultaneously for years — but the security improvement is step-function, not incremental. If you're building identity infrastructure today and you're not planning for passkeys, you're building for the past.
Decentralized identity (DIDs, verifiable credentials) is interesting but not ready for mainstream consumer platforms. The standards are maturing, but the user experience is still poor, and the trust model requires users to manage cryptographic material, which history says they won't do reliably. Watch this space, but don't bet your architecture on it yet.
AI-driven adaptive authentication is already here in practice, even if it's not always labeled that way. Risk-based authentication — adjusting the auth challenge based on device fingerprint, location, behavioral signals — has been standard at scale for years. What's changing is the sophistication of the signals and the models. Expect authentication to become increasingly invisible for legitimate users and increasingly hostile for attackers, with the identity system making real-time judgments about how much friction to introduce.
The constant across all of these trends is that identity infrastructure keeps getting more complex, more critical, and more underappreciated. If you're an engineer working in this space, know that what you're building is the foundation everything else stands on. And if you're not in this space, the next time you log in seamlessly across three products on two devices without thinking about it — someone built that, and it was harder than it looked.
Top comments (0)