Getting Out of the Password Business

#security #authentication #java #architecture

Our old authentication system logged users in with the OAuth2 client_credentials grant.

If you know OAuth, you just winced. client_credentials exists so that a client application can obtain a token for itself, rather than on behalf of a user. It has no concept of a human being at all. But it takes two values, an ID and a secret, and if you squint, an email address and a password are also two values.

So that's what it did. Each user effectively became an OAuth client: client_id was their email address, and the registered client secret was their BCrypt password hash. The browser sent an HTTP Basic header, the server matched the credentials against that per-user registered client, and out came a token.

It worked for years. That is the recurring theme of this post.

What we actually had

The platform is a multi-tenant billing system that ingests orders and payments from e-commerce marketplaces and produces accounting documents. Authentication was entirely in-house: a Spring Authorization Server configuration of about 380 lines, issuing and signing its own tokens. Access tokens lived 24 hours, refresh tokens 48.

Issued tokens went into an oauth2_authorization table, with a startup job that deleted rows older than a couple of days. Other services didn't parse tokens at all — they called back to the auth service to introspect each one as an opaque string. The oldest component in the system sat on the hot path of every authenticated request in every other service.

Password hashes lived in a password column on the user table, and registration hashed into it directly.

None of this was incompetent. It was written when the platform was a single application, and it kept working while the platform stopped being one. The failure mode of home-grown auth is rarely a dramatic breach; it's that the thing quietly becomes load-bearing infrastructure that nobody signed up to own.

Three forces finally moved it: security standards set by our parent company across the group, enterprise customers asking to sign in with their existing Google Workspace and Microsoft Entra ID accounts, and the accumulated cost of maintaining a bespoke identity stack.

The cut-over

We moved to Google Identity Platform, primarily because our infrastructure already runs on Google Cloud — identity became native rather than another integration, with one tenant per environment.

Verification ended up taking two forms, which I'd call a mild inconsistency rather than a mistake. The auth service verifies ID tokens through the Firebase Admin SDK in a servlet filter. Every other service uses the stock Spring resource server pointed at the provider's JWKS endpoint, so token validation there is ordinary configuration rather than code we maintain. Domain-specific facts — the selected organisation, the user's internal identifier, whether the caller is a service account — ride along as custom claims minted per token.

Now the part I'd flag to anyone planning this: we cut over in a single day. Five backend repositories and the frontend all merged on the same date. There was no dual-auth period, no compatibility endpoint that let old tokens be exchanged for new ones, and no feature flag to route a percentage of traffic. The frontend switched by pointing its login route at a new screen and leaving the old one unrouted.

That was deliberate, and I'd defend it. The obvious alternative is a transition period where every service accepts both the old tokens and the new ones. But that doubles the authentication surface you have to reason about and test, and interim compatibility layers have a habit of outliving their purpose — six months later you still have two ways to authenticate and nobody wants to be the one who deletes the old path. With a single team and a fixed set of clients we controlled, one coordinated release with everybody present and watching struck me as the lower risk.

The commit history immediately afterwards shows what that costs, honestly: a week of small fires — configuration corrections, test fixtures to repair, a handful of things that only surface once real traffic arrives. That's the trade. You concentrate the pain into a window where the whole team is paying attention, instead of spreading it across a quarter where they aren't.

The most instructive failure arrived within the hour, and it wasn't in any of the code we'd just rewritten.

Five thousand accounts, zero password resets

One constraint was non-negotiable: nobody gets a "please reset your password" email. That email, sent without a breach behind it, spends trust you can't easily earn back.

The provider supports importing users along with their existing hashes, if you tell it the hash algorithm and parameters. So the migration exporter read the existing users, imported them in batches of a thousand with BCrypt declared as the hash scheme, and wrote the resulting provider identifiers back onto our own user rows.

The hashes moved across intact. Everyone's existing password kept working the next morning, verified by the provider instead of by us. Nobody had to do anything and, which was the whole point, users simply carried on.

That's the real end state. We are not in the business of storing credentials anymore. Our application does not verify a password, does not compare a hash, and would have nothing useful to leak on that front if the database were compromised tomorrow.

What surprised me was how much arrived with it. Multi-factor authentication was the obvious one: TOTP, the familiar QR-code flow used by Google Authenticator and its equivalents. Enrolment, verification, recovery codes and the enforcement rules around them are weeks of careful work to build and a permanent obligation to maintain. We had never built it, and — being honest about our priorities — probably never would have. It came as configuration, as did federated sign-in for the identity providers our customers actually use. And the same foundation is what makes the passwordless work we're doing now — passkeys — tractable at all.

That's the underrated part of delegating identity. It isn't only that you stop carrying a liability. It's that a whole category of security work stops being a project you have to justify and becomes a feature you switch on.

The account that wasn't there

Scheduled jobs — billing runs, marketplace imports — authenticate as a technical user rather than as a person. Under the old system that was a real row in the database with a token persisted alongside it, updated by hand when it expired. Under the new one it's a service-account identity that mints and exchanges its own short-lived token.

On release night, every machine-to-machine call started returning 401.

The cause was not in the code. The technical user simply did not exist in the production database. It existed in development, where we'd built and tested everything, and had never been created in production. Human logins worked fine, because humans were covered by the account import. The one identity that wasn't a real person had fallen outside it.

We caught it the same evening and fixed it, because the entire team was watching a release we knew was high-stakes. That is the argument for concentrating a migration into one window: the failure surfaced while everyone was still at their desks, rather than at 3am on a Tuesday three weeks later when nobody was.

The lesson I took isn't about tokens at all. When you migrate identity, you naturally think about users — the people with passwords, the ones who will notice and complain. Service accounts have none of those properties. Nobody advocates for them, they don't file support tickets, and they fall outside the mental model you're using when you plan the work. They're also the accounts that run everything important while you're asleep.

Environment parity gets discussed as a configuration problem — same versions, same flags, same infrastructure. This was parity of data: an identity that existed in one environment and not the other, in a system whose entire job is deciding who exists.

What we quietly lost

Migrations remove things, and it's worth being honest about which.

The old system locked an account after three failed logins and emailed the user when a login arrived from an unusual location. Both were custom, both had been written by someone who cared, and both went away with the code that contained them. The provider brings its own protections against credential stuffing, so the net security position improved — but those specific behaviours weren't reimplemented, and nobody made an explicit decision to drop them. They simply weren't in scope, which is how features die.

We also left ourselves some scaffolding. Migration code tends to arrive with a comment promising it will be deleted afterwards, and "afterwards" is not a date. Cleanup work that has no deadline doesn't get one, and the only reliable fix I know is to put the removal in the plan as a task with an owner, before the migration ships and everyone's attention moves on.

The passkey work mentioned above is in progress — written and working on a branch, not yet merged.

What I'd take from it

Home-grown authentication doesn't fail loudly, it just accumulates. Ours cost us SSO deals and put a decade-old component on the hot path of every request. The bill arrives in instalments.
A big-bang cut-over is a legitimate choice, not a failure of nerve. Dual-authentication periods double the surface you have to test and tend to become permanent. Concentrating the risk into one window, with the whole team watching, can be the safer option — provided you actually watch.
Service accounts fall through the gaps in an identity migration. You plan around users, because users are the ones who notice. The accounts that run your overnight work have nobody to speak for them and are absent from every checklist you write.
Environment parity includes data, not just configuration. Same versions and same flags mean nothing if an identity exists in one environment and not the other.
Write down what you're dropping. Account lockout and location alerts were real features that disappeared without anyone explicitly deciding they should. Removal by omission is the easiest kind to miss.
Delegating identity buys more than it removes. Losing the credential liability was the goal; getting MFA and federated sign-in as configuration, and a solid foundation to build passwordless on, turned out to be worth more.
For most applications, storing credentials has become an optional responsibility rather than a necessity. It's still the right call for some. It just isn't the default any more, and it's worth asking whether you're holding it on purpose.