DEV Community

Daniel Pertu
Daniel Pertu

Posted on

A user replied "thank you very much" and we received our own email back underneath it

#webdev #typescript #node #email

Our support tickets are created in the app, and every ticket email we send sets
reply-to to our support address with the ticket ref in the subject:

Re: [TCK-63D41C9A] Question: General enquiry
Enter fullscreen mode Exit fullscreen mode

Which means users do the obvious thing: they answer from their inbox. For a
while, that reply arrived at my end as a raw forward of the whole email. The
case that made me fix it was a Gmail user who wrote three words, "thank you very
much", and the forward I received put our own reply underneath them with
exactly the same prominence. I read the quote first and could not work out what
they were asking me.

Threading those replies back onto the ticket turned out to be two small problems
that are both more interesting than they look.

Problem one: which ticket is this

The only thing carrying the identity is the subject line, and the ref in it is
deliberately short because humans quote it out loud:

/**
 * A short, human-quotable reference for a ticket, derived from its UUID id.
 * Deterministic (no extra column needed) and stable for the life of the ticket.
 */
export function ticketRef(id: string): string {
  return `TCK-${id.replace(/-/g, '').slice(0, 8).toUpperCase()}`;
}
Enter fullscreen mode Exit fullscreen mode

Eight hex digits, derived rather than stored, which is what makes it free. It is
also the first group of the UUID, which is what makes the reverse lookup a
prefix match:

const idPrefix = ref.replace(/^TCK-/i, '').toLowerCase();
if (!/^[0-9a-f]{8}$/.test(idPrefix)) return null;

const rows = await db
  .select({ id: supportTicketsTable.id, userId: supportTicketsTable.user_id })
  .from(supportTicketsTable)
  .where(
    and(
      like(supportTicketsTable.id, `${idPrefix}-%`),
      sql`lower(${supportTicketsTable.email}) = ${senderAddress.toLowerCase()}`
    )
  )
  .limit(2);
return rows.length === 1 ? { ticketId: rows[0].id, userId: rows[0].userId } : null;
Enter fullscreen mode Exit fullscreen mode

Three things are load-bearing in that query.

The regex on idPrefix runs before the query, not as validation theatre. The
value goes into a LIKE pattern, and _ and % are wildcards in a LIKE
pattern. Drizzle parameterises the string so there is no injection, but a
pattern of % would still happily match every ticket in the table. Asserting
eight hex characters means the pattern cannot contain a metacharacter at all.

The sender has to be the ticket's own address. Eight hex digits is 32 bits.
That is a fine ref for a human to read back to you and a bad key to look up
alone, because anyone who can guess a ref should not be able to post into
somebody else's conversation by putting it in a subject line. Pairing the prefix
with the address means an attacker would have to control the email account the
ticket belongs to, at which point they can read the thread anyway.

And .limit(2) with rows.length === 1. Two rows means the prefix is ambiguous
for that sender, which should be vanishingly rare and is not impossible. The
code does not pick one. It returns null, and the caller falls back to
forwarding the email to me as it always did. "I cannot tell which of your two
tickets this is" is a situation for a human, and the fallback already exists, so
using it costs nothing.

Problem two: what did they actually write

Mail clients append the message you are replying to, and every one of them marks
the boundary differently. The module that finds that boundary is 84 lines and
most of them are describing somebody else's format.

Gmail writes an attribution line:

Enter fullscreen mode Exit fullscreen mode

Outlook writes a header block, sometimes under a row of underscores:

Enter fullscreen mode Exit fullscreen mode

iOS Mail quotes with > and puts the signature above it, which has to stay.
Older clients write ----- Original Message -----.

The naive cut is "everything above the first line starting with >", and it
fails on the plain-text part of a Gmail reply, where the quote is often not
marked with > at all. So the detector has four separate signals:

function quoteStart(lines: string[]): number {
  for (let i = 0; i < lines.length; i++) {
    const line = lines[i].trim();
    if (line.startsWith('>')) return i;
    if (SEPARATOR.some((re) => re.test(line))) return i;
    if (OUTLOOK_FROM.test(line) && OUTLOOK_SENT.test(lines[i + 1]?.trim() ?? '')) return i;
    if (/[:：]$/.test(line)) {
      const start = attributionStart(lines, i);
      if (start !== -1) return start;
    }
  }
  return -1;
}
Enter fullscreen mode Exit fullscreen mode

The Outlook signal requires two lines to agree, a From: immediately followed
by a Sent: or Date:. A lone "From:" is something a person writes.

The attribution line is the hard one

English clients end the attribution with "wrote:". That is useless for everyone
else, and our users are not all writing in English. German Gmail produces:

Enter fullscreen mode Exit fullscreen mode

Translating "wrote" into thirty languages is the obvious trap. The thing every
client's attribution has in common is not a verb, it is our own address in
angle brackets, plus a date, plus a trailing colon:

const ATTRIBUTION = [
  /^On\s.*\d.*\swrote:$/i,
  /^(?=.*\d)\S.*<\s*[^<>\s]+@cogniprep\.app>.*[:：]$/i,
];
Enter fullscreen mode Exit fullscreen mode

The second pattern is language independent. The lookahead requires a digit
somewhere, because every client prints a date; the <...@cogniprep.app> is the
part that makes it an attribution rather than a sentence; the trailing colon
includes the fullwidth variant, because a CJK client will write one.

Two refinements came out of writing tests for it.

Gmail wraps a long attribution across lines in the plain-text part, splitting it
mid-address. So the matcher joins up to three lines and tries the shortest match
first. But a multi-line match is only allowed when a blank line sits above it,
so the sentence a user typed directly above the attribution is never eaten:

if (n > 1 && start > 0 && lines[start - 1].trim() !== '') continue;
Enter fullscreen mode Exit fullscreen mode

And a line ending in a colon is a completely normal thing for a user to write,
which is why the colon alone is only a trigger for trying the patterns, never a
cut. This is my favourite test in the file:

it('does not treat a reply line ending in a colon as an attribution', () => {
  const text = 'On my results page I see this:\nScore 0';
  expect(stripQuotedReply(text)).toBe(text);
});
Enter fullscreen mode Exit fullscreen mode

A line that starts with "On", ends with a colon, and is immediately followed by
the thing they are describing. Lose that and you have thrown away the only part
of the message that mattered.

Returning empty is a feature

A bottom-posted reply, where the user types underneath the quote, produces no
new text above the boundary. The function returns '' rather than guessing:

it('returns nothing for a bottom-posted reply', () => {
  expect(stripQuotedReply(`${QUOTED_TICKET_EMAIL}\n\nThanks`)).toBe('');
});
Enter fullscreen mode Exit fullscreen mode

And the webhook treats that as "not a reply I can place":

const reply = stripQuotedReply(parsed.text ?? '').slice(0, SUPPORT_LIMITS.messageMax);
if (reply.length < SUPPORT_LIMITS.messageMin) return false;

const match = await findTicketForInboundReply(ref, sender);
if (!match) return false;
Enter fullscreen mode Exit fullscreen mode

Every return false in that function means the email gets forwarded to me whole,
exactly as before. The feature is strictly additive: when it is confident it
threads the reply onto the ticket and sends me the same notification an in-app
reply sends, and when it is not confident the old behaviour is still there. No
email can be lost by this code being wrong, only by it being too clever, which
is why none of the signals guess.

One ordering detail, because this runs in a webhook that can be retried:

// Nothing after addUserReply can throw, so a retried delivery never records
// the reply twice.
Enter fullscreen mode Exit fullscreen mode

The write happens last among the things that can fail. The notification email
swallows its own errors, so a mail provider blip cannot turn a successfully
recorded reply into a 500 that invites a redelivery, which would post the reply
a second time.

Attachments needed a different sender

The ticket thread stores text, so a screenshot pasted into an emailed reply has
nowhere to live in the database. It still has to reach me, which meant the
notification could no longer go through our normal templated send and had to go
raw, carrying the files:

// No content IDs: the notification's HTML references none of them, so an image
// pasted into the reply arrives as a plain attachment rather than vanishing.
const attachments = parsed.attachments.map((attachment) => ({
  filename: attachment.filename,
  content: attachment.content,
  contentType: attachment.contentType,
}));
Enter fullscreen mode Exit fullscreen mode

Dropping the content IDs is the point. An inline image in an email carries a
Content-ID so the HTML can reference it with cid:. Our notification's HTML
references none of them, and an attachment marked as inline that nothing
references is one a mail client is entitled to hide. Stripping the ID downgrades
it to an ordinary attachment, which is visible.

The quietest kind of feature

Nobody will ever notice this working. The only visible evidence is the absence
of a thing: I no longer read a forwarded email twice to work out which half the
human wrote.

If you want to see the surface this hangs off, the
help page is where a ticket starts, and there is a
Help link in the footer of every page on the site, including the
practice test library.

Top comments (0)