DEV Community: Email Digit

Invisible Unicode Characters in Email: How to Find the Ghosts in Your Templates

Email Digit — Wed, 11 Nov 2026 20:40:35 +0000

Picture a Tuesday deploy. Marketing pastes the new welcome email copy from a shared doc into your template, you ship it, and everything looks perfect in the preview. Two days later a support ticket arrives: a customer's email client showed one line of the message in a strange order, and your keyword-based content filter in staging never flagged the word it was built to catch. You diff the copy. The diff is empty. Nothing is wrong. Except something is.

Welcome to invisible Unicode characters in email: characters that take up a position in your string, draw nothing (or rearrange what's around them), and survive every copy-paste with the stealth of a ninja who also happens to be a lawyer.

The two families of invisible characters

Most text never contains these, which is exactly why they surprise people.

Family	Examples	What it does
Direction controls	Right-to-left override (U+202E), left-to-right embedding (U+202A), isolates (U+2066 to U+2069)	Tell the renderer to lay out the following characters in a different direction
Zero-width characters	Zero-width space (U+200B), word joiner (U+2060)	Occupy a position in the string but render as nothing

Both have honest jobs in typesetting. The trouble starts when the text a reader sees and the text your code stores stop being the same string.

How they disguise a line

The classic demonstration is a file name. Put a right-to-left override in front of gpj.exe at the end of invoice_, and the name displays as invoice_exe.jpg. It looks like a picture. It's a program.

In 2021, security researchers showed the same trick making source code read one way to a reviewer and compile another way. That research, tracked as CVE-2021-42574, is why the technique is now widely called "Trojan Source".

Zero-width characters are quieter, but they cause similar headaches:

Breaking up words. A zero-width space inside a word means a filter looking for that word no longer matches, while a human sees nothing unusual.
Making two strings look identical. A link label with an invisible character inside looks the same on screen as the real thing, but === disagrees.
Tagging copies. Patterns of zero-width characters can watermark text so a leaked copy can be traced.

How they end up in ordinary email

Almost nobody puts them there on purpose. They arrive by paste: from a word processor, a shared doc, a web page, a chat message, or an old template that was itself pasted from somewhere else. Nothing on screen shows them, so they hitchhike from campaign to campaign like a burr on a sock.

If you build or send email from code, that means your templates, your CMS fields and your copy pipeline are all possible entry points.

Find them: a one-line regex

Most editors and every language with a regex engine can search for them. This pattern matches the direction controls and the common zero-width characters:

[\u200B\u2060\u202A-\u202E\u2066-\u2069]

A quick Node check over a template string:

const INVISIBLE = /[\u200B\u2060\u202A-\u202E\u2066-\u2069]/g;

function findInvisible(text) {
  return [...text.matchAll(INVISIBLE)].map(m => ({
    index: m.index,
    codepoint: 'U+' + m[0].codePointAt(0).toString(16).toUpperCase().padStart(4, '0'),
  }));
}

console.log(findInvisible('Welcome\u200B aboard'));
// [ { index: 7, codepoint: 'U+200B' } ]

A hit in text that's meant to be plain English is almost always safe to delete. Run it in CI against your email templates and you'll catch the next paste before a customer does.

Don't strip everything: the characters real languages need

Here's where the "just delete all of it" instinct goes wrong. Two invisible characters are load-bearing in real writing:

Zero-width non-joiner (U+200C): Persian and Arabic spelling use it to stop letters connecting where they shouldn't.
Zero-width joiner (U+200D): Indic scripts use it to form the right conjuncts.

That's why the regex above deliberately leaves out U+200C and U+200D. Strip them and you haven't cleaned your Persian newsletter, you've misspelled it.

And real right-to-left text (Arabic, Hebrew) doesn't need override characters at all. The standard HTML dir attribute on a block, with embedded left-to-right runs such as prices and URLs isolated, gets direction right without any invisible controls in the text.

What a sending platform can (and can't) do for you

This is the approach Email Digit takes, described in its guide to invisible Unicode characters in email: when an email is rendered for sending, the text of paragraphs, quotes and table cells is cleaned. Direction controls, the zero-width space, the word joiner and a few other invisible formatting codes such as the soft hyphen are removed, while the joiner and non-joiner are kept.

The guide is also upfront about the limits, which are worth copying into your own mental model whatever tool you use:

Not every field is cleaned. Subject lines, headings, list items, button labels and raw HTML blocks aren't passed through it today, so check those yourself if the copy came from somewhere you don't trust. (Subject lines are the sneaky one; a subject line tester helps you look at them before sending.)
Invisible isn't the same as lookalike. A URL that swaps a Latin "a" for a Cyrillic one is made of visible characters. No invisible-character filter will catch it. Read link destinations, not just link text.
It isn't a spam filter. Cleaning hidden characters stops your own email carrying them by accident. It doesn't judge what the copy says.

A short checklist for your email pipeline

Add the regex check to CI for every template and stored snippet.
Normalize pasted copy at the input boundary (CMS field, admin form), not only at send time.
Whitelist U+200C and U+200D so Persian, Arabic and Indic text survive.
Use dir="rtl" on blocks instead of override characters.
Hand-check subject lines and button labels, which many cleaners skip.

If a character is invisible and changes what a line means, it doesn't belong in an email. Your diff tool can't see it. Now your CI can.

I work on Email Digit, a deliverability-first email platform. Sources: Email Digit's guide on invisible Unicode characters (read November 12, 2026) and the Trojan Source CVE record.

This post was drafted with AI assistance and checked against the sources above.

Idempotency Keys for Transactional Email: How to Retry Without Sending Duplicates

Email Digit — Wed, 11 Nov 2026 01:58:13 +0000

It's 2:14 a.m. A user can't log in, so they hit "Forgot password". Your API calls the email provider. The provider queues the email. And then, somewhere between their data center and yours, the response quietly dies. Timeout. Dropped connection. A deploy that picked exactly this moment to restart the worker.

Your code does the responsible thing and retries. The provider, having no idea it has met this request before, does the other responsible thing and sends it again.

The user now has two reset emails with two different links, and a growing suspicion that one of them is a phishing attempt. They click the older one. It's expired. They open a support ticket with the subject line "your app is broken???" (three question marks, which is how you know it's serious).

You can't fix this by removing retries; that just trades duplicate emails for lost ones, which is worse. The real fix is to make the retry recognisable. That is exactly what an idempotency key for transactional email does, and it's one of those tiny changes that saves you from a whole category of 2 a.m. bugs.

This post covers how to pick a key that actually works, what your code should do with each response, and the sneaky mistakes that break it without telling you.

What an idempotency key does

An idempotency key is a string you send with the request that names the intent: "the reset email for reset request 8412". The provider remembers keys it has seen:

New key: the request is processed and the email is sent.
Key seen before, same payload: nothing new is sent; you get the original result back.
Key seen before, different payload: the request should be refused, because that is almost always a bug in the caller.

The whole pattern rests on one rule: every retry of the same logical email must carry the same key. Break that rule and you've built a very elaborate way of sending duplicates.

Choosing a key: derive it from your data, not the HTTP call

The key has to be decided before the first attempt and never regenerated. That means it should come from the event that caused the email.

Key	Safe?	Why
`pwreset-8412` (reset request id)	Yes	Every retry of that reset carries the same key
`receipt-order-5531`	Yes	One receipt per order, however many times the job runs
`crypto.randomUUID()` at call time	No	A retry generates a new UUID, so it looks like a new email
The recipient's address	No	Their legitimate second reset next month looks like a repeat

A random UUID is fine if you generate it once, store it with the event, and read it back on retry. The failure mode is generating it inside the function that makes the HTTP call. That code looks perfectly reasonable in review, passes every test, and then sends two receipts the first time production has a bad day.

A simple convention that scales: <email-type>-<source-entity-id>, for example welcome-user-1207, invoice-inv-2026-0042, shipping-order-5531-parcel-2.

A retry loop that is safe to run twice

Here is the shape in JavaScript. The important part is that idempotencyKey is computed once, outside the loop.

async function sendResetEmail(resetRequest) {
  const idempotencyKey = `pwreset-${resetRequest.id}`; // decided once

  for (let attempt = 1; attempt <= 5; attempt++) {
    try {
      const res = await fetch(EMAIL_API_URL, {
        method: "POST",
        headers: {
          Authorization: `Bearer ${process.env.EMAIL_API_KEY}`,
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          trigger_key: "password_reset",
          to: resetRequest.email,
          variables: { action_url: resetRequest.url },
          idempotency_key: idempotencyKey,
        }),
        signal: AbortSignal.timeout(10_000),
      });

      if (res.status >= 500) throw new Error(`retryable ${res.status}`);
      return await handleResponse(res); // 2xx and 4xx are final
    } catch (err) {
      if (attempt === 5) throw err;
      await sleep(2 ** attempt * 250); // backoff, same key next time
    }
  }
}

If the first attempt actually succeeded and only the response was lost, attempt two returns the original result instead of a second email.

Handle the answers, not just the happy path

Retrying is only correct when the outcome is unknown. Most non-2xx answers mean "fix something", not "try again". This is the table Email Digit publishes for its transactional endpoint, and it is a good template for any provider:

Response	Meaning	Retry?
202, status `duplicate`	Same key, same payload: original message ids returned	No, record it as success
202, status `suppressed`	Address hard bounced or complained before; nothing sent	No
409	Key reused with a different payload, or the journey isn't active	No, fix the cause
404	No journey has that trigger key	No, fix the key
422	Missing recipient, or sending domain not verified	No
403	API key lacks the send scope	No
Timeout or 5xx	Outcome unknown	Yes, same key

That 409 on "same key, different payload" is worth wiring to an alert. It usually means someone built the key from the user id instead of the event id, so two different emails collided. It's the API equivalent of a coworker saying "hey, quick question": never as quick as advertised, always worth answering.

Mistakes that break idempotency quietly

Regenerating the key per attempt. The most common one, and the most humbling. If your logs show a different key on each retry, you have no protection, just extra steps.
Keying on the recipient. Two genuine resets a minute apart should send two emails. They need two keys.
Reusing a key for a different message. Treat keys as permanent. Don't assume a provider forgets them after 24 hours unless it documents that.
Retrying 4xx errors. A 422 for an unverified sending domain will fail the same way forever. Retrying just burns queue time.
Sharing keys between test and live. Check your provider keeps them separate, or prefix keys with the environment.

Idempotency is not deduplication of events

An idempotency key stops the same request from sending twice. It does not decide whether two different events should both send. If your app emits two order.paid events for one order, the email API will dutifully send two receipts, because from its point of view you asked twice. You need dedupe upstream, at the event or job level, before you ever reach the email API.

Idempotency keys won't fix your event bus. They will, however, let you sleep through the next network blip, and your support inbox will be noticeably less dramatic.