DEV Community

Ender Yentar
Ender Yentar

Posted on Originally published at mailflat.net

We show your API key once, and here is the option we turned down

We shipped a change that takes something away. You used to be able to open an inbox in the dashboard and reveal your API key whenever you wanted. Now the full key appears once, at the moment it is created, and after that you only ever see mf_sk_••••PeH0.

Nobody asked for that. We did it on purpose, and the reasoning is more interesting than the change.

A schema task with a product decision inside it

The task on the board said: drop two plaintext columns. It read like housekeeping. Migration, tests, done.

Before touching it we went looking for everything that reads those columns. One of them fed a button. The inbox modal had a "Show" control that revealed the whole key, and a "Copy" that copied the whole key. Both of them read the plaintext column directly.

So dropping the column was not housekeeping. It answered a question nobody had written down:

can a user see their key again after they create it?

That is a product decision. It was sitting inside a schema ticket, unlabelled, and it would have been decided by whoever happened to run the migration.

What we actually store

Before choosing, we wrote down what is already true, because two of the three options only make sense in contrast to it:

What How Reversible
API keys HMAC-SHA256 with a pepper no
Passwords scrypt 32768:8:1 with a salt no
Messages RSA-OAEP-2048 plus AES-256-GCM yes, but the private key lives in your browser

One note, because it comes up every time: base64 is not encryption. It is an envelope, not a lock. If a key is base64 in your database, it is plaintext with extra steps.

Three options

A. Show it once. The key is displayed at creation, with a warning, and never again. Lost key means revoke and create a new one.

B. Encrypt instead of hash. Store the key encrypted with a server-side key, decrypt on request, show it whenever the user asks.

C. Encrypt with the user's own key. We already run end-to-end encryption for messages: the public key encrypts, the private key stays in the browser. The API key could be stored the same way and decrypted client side, with the server still blind.

We chose A.

Why B was rejected

B is the option that keeps every feature. It is also the one that quietly cancels the reason the work existed.

The whole point of hashing is a sentence we get to say: if our database leaks, the keys in it are not usable. Option B replaces that with if our database leaks, the keys are encrypted, and so is the key that decrypts them, which our application can reach.

An attacker with database access is usually one step from application access. B does not add a wall, it adds a drawer next to the wall.

The way we phrased it internally: A lets us say "we cannot see this key either". B turns that into "we cannot see it, but we do keep it". Those are not the same promise, and only one of them survives a bad day.

Why C is on a shelf and not in the bin

C is genuinely good. The infrastructure exists, it is running in production for message bodies, and it would give users their key back without the server ever reading it.

It has one real cost: it forks the system.

Users can turn end-to-end encryption off, and keys created through the API have no browser to decrypt in. So C is not "the same thing, better". It is a second branch that has to be built, tested and explained alongside the first. Two paths where there is currently one.

So we wrote C up as its own plan, and then wrote down what would have to be measured before we build it:

  1. Users regularly landing in a revoke-and-recreate loop, visible as short-lived key rows on the same inbox.
  2. Real support requests along the lines of "I cannot see my key".

Both are currently zero. Not "we think they are low". Zero, with a query behind it.

That last part is the bit worth stealing. "We might do this later" is a way of avoiding a decision. "We will do this when this number stops being zero" is a decision, and it has an owner and a trigger.

Shipping it

Four code paths read or wrote the plaintext column: the "does this inbox already have a key" check, the write on creation, the clear on revoke, and the list endpoint, which was returning the full key in its response body.

Authentication was not one of them. It had been moved to the hash a few weeks earlier, which is the only reason this change did not touch the login path at all. Being able to say that, with a line number, was worth more than any amount of confidence.

The revoke button stayed exactly where it was. If you lose your key, that is your exit, and removing the reveal makes revoke more important rather than less.

The proof that counted

Production verification was five measurements on a throwaway inbox:

  1. Inbox creation returns no key
  2. Key generation returns the plaintext once
  3. Both plaintext columns are null afterwards
  4. The list endpoint returns a masked string, not the key
  5. A live HTTP request using that key returns 200

The first four say nothing leaks. The fifth says it still works. Either one alone is a trap. A system that leaks nothing because it is broken passes 1 through 4 perfectly.

We are still running with the old columns in place. They will be dropped in a separate step, after a week of the new path being live, with a separate approval. There is no reason to combine an irreversible migration with a behaviour change on the same day.

What we would tell ourselves a week earlier

Before you run a migration, grep for what reads the column. Not to check the code compiles, but because a column that feeds a button is not a storage detail, it is a promise to a user, and someone should decide about it on purpose.

We build MailFlat this way throughout: the server holds ciphertext it cannot read, and the parts we cannot recover for you are the parts we are proudest of.

Top comments (6)

Collapse
 
peterbuildssecure profile image
Peter •

The five-measurement proof is the right shape — especially pairing "nothing leaks" with "it still works," since a broken system passes the first four for free.

One path worth grepping for alongside the four you found: the masked display, mf_sk_••••PeH0. Showing the last 4 characters is standard practice, but it requires those characters to exist somewhere in a form the read path can reach at display time. If that suffix is derived by storing a few plaintext characters separately (a common shortcut when you've moved everything else to a one-way hash), that's a fifth code path reading key material outside the four you listed — and it's exactly the kind of thing that survives a review because it feels like display logic, not storage. If the mask is instead reconstructed from the HMAC or generated at creation time and cached only in the once-shown response, this doesn't apply — but worth confirming which one it is and writing down why, the same way you wrote down HMAC-with-pepper for the full key.

Collapse
 
enderyentar profile image
Ender Yentar •

Good catch, and you guessed the shortcut correctly. The last four characters are stored in their own column, key_last4, next to a key_prefix column that only ever holds the fixed type prefix (mf_sk_ or mf_live_). Both are written once, at creation, while the plaintext is still in hand. The read path builds the mask from those two columns and never touches key material, so there is no fifth reader of the key itself. What it does mean is that 4 characters of every key sit in the database in plain text. An mf_sk_ key carries 256 random bits and those 4 characters give away 24 of them, which we are comfortable with, but you are right that it deserved a written line next to the HMAC one instead of being left to look like display logic. Adding it to our notes.

Collapse
 
peterbuildssecure profile image
Peter •

The 24-bit number is right if key_last4 is a literal slice of the raw random bytes, but it's worth stating the assumption instead of just the bit count, because it breaks quietly if the encoding changes. Hex gives 4 bits/char; base64 or base58 gives more, so the same 4-character mask leaks more than the hex math suggests. Worth pinning the assumption ('N bits per char, this alphabet') next to the HMAC note, not just the headline bit count.

Collapse
 
raknaos profile image
Raknaos •

The line "it would have been decided by whoever happened to run the migration" is the real finding here.

I ran into the mirror image of this maintaining a local relay that holds long-lived browser sessions: a field I treated as plumbing turned out to be load-bearing for a UI button I hadn't even known about. The schema ticket was trivial; the archaeology underneath it was the actual work.

Your table of what's stored how is also the part most writeups skip. Showing HMAC-with-pepper next to scrypt next to encrypt-and-reveal makes it obvious why "can the user see it again" has a different answer per column — it's not one policy, it's a decision per data class. Most teams flatten that into "encrypt everything" and end up with keys in logs anyway.

One question from the ops side: what happened to users who legitimately lost their key after the change? If the only recovery is rotate-and-reissue, that's fine for API consumers, but worth naming in the changelog so people don't file it as a bug.

Collapse
 
enderyentar profile image
Ender Yentar •

Thanks. "The archaeology underneath it was the actual work" is a better summary than anything in the post. On lost keys: getting a new key is the only path, on purpose, and it is one step away. For account keys there is a Rotate button: one click, the old key stops working immediately, and the new one is shown once with the same scopes. For inbox keys it is revoke and create, from the same modal. The creation screen says "Copy it now. This is the only time it is shown" at the moment the key appears, and the modal tells you to revoke and create a new one if you lose it. Good point on the changelog, noted.

Some comments may only be visible to logged-in visitors. Sign in to view all comments.