DEV Community

BeGoodTool.com
BeGoodTool.com

Posted on

Why splitting text into sentences by punctuation breaks on "Dr. Smith" (I found out building a case converter)

I added a "Sentence case" mode to a text case converter I was building — the kind of tool that turns pasted text into UPPERCASE, lowercase, Title Case, or "just capitalize the start of each sentence." The first three are one-liners. The fourth one, sentence case, is the one that quietly ate an afternoon, because "find the start of a sentence" turns out to be a much fuzzier problem than "find a period."

The .split() trap I started with

The file I was working in still has my first attempt sitting there, commented out, as a reminder of how I got it wrong:

// const capitalizeSentences = () => {
//   outputArea.value = inputArea.value.replace(
//     /.+?[\.\?\!](\s|$)/g,
//     function (txt) {
//       return txt.charAt(0).toUpperCase() + txt.substr(1).toLowerCase();
//     }
//   );
// };
Enter fullscreen mode Exit fullscreen mode

It looks reasonable: lazily match everything up to a ., ?, or ! followed by whitespace (or end of string), then capitalize the first letter and lowercase everything else. The bug is in that last part — .toLowerCase() on the rest of the "sentence." Paste in "I love NASA missions. The ISS orbits every 90 minutes." and this version happily lowercases NASA and ISS along with everything else, because it doesn't know the difference between "the rest of a sentence" and "a proper noun or acronym that happens to not be first." It also had no concept of line breaks, so multi-paragraph text got flattened into a single blob-replace pass.

What actually shipped

The version that replaced it splits the problem into two passes instead of one regex:

function capitalizeSentences() {
  if (!inputArea.value) return "";

  const trimmedText = inputArea.value.trim();
  let group = trimmedText.split(/(?<=\n)/);
  group = group.map((item, index) => {
    const sentences = item.split(/(?<=[.?!])\s+/);
    const capitalizedSentences = sentences.map((sentence) => {
      const trimmedSentence = sentence.trim();
      const firstChar = trimmedSentence.charAt(0).toUpperCase();
      const restOfSentence = trimmedSentence.slice(1);
      return `${firstChar}${restOfSentence}`;
    });
    return capitalizedSentences.join(" ");
  });

  outputArea.value = group.join("\n");
}
Enter fullscreen mode Exit fullscreen mode

Two things changed on purpose here. First, it splits on lines before splitting on sentences, using a lookbehind (?<=\n) so the newline stays attached to the chunk before it — that keeps paragraph structure intact instead of merging everything into one line. Second, and more importantly, sentence-splitting uses (?<=[.?!])\s+ instead of a consuming match. A lookbehind checks that a period/question mark/exclamation point came right before the split point without actually consuming it, so the punctuation stays glued to the end of the sentence it belongs to instead of getting lost or duplicated.

The other deliberate choice: it only touches firstChar. restOfSentence is used exactly as typed, no .toLowerCase() anywhere. That's what fixes the NASA/ISS problem from the naive version — if a word was already capitalized correctly, sentence case just leaves it alone instead of "helpfully" re-lowercasing it. It's a much dumber piece of code than the first attempt, and that's exactly why it behaves better.

Title Case has a completely different bug

Title Case (capitalize every word) doesn't need sentence boundaries at all — it just needs "word" boundaries, so it uses a much simpler regex:

const capitalizeWords = () => {
  outputArea.value = inputArea.value.replace(/\w\S*/g, function (txt) {
    return txt.charAt(0).toUpperCase() + txt.substr(1).toLowerCase();
  });
};
Enter fullscreen mode Exit fullscreen mode

\w\S* means "a word character, followed by any run of non-whitespace." That last part is the trap: \S* doesn't stop at hyphens, apostrophes, or slashes, so "well-known" is matched as a single token, not two words. Capitalizing the first letter and lowercasing the rest turns it into "Well-known" instead of "Well-Known." Same story for anything with an internal capital that isn't at a word boundary a human would recognize — "NASA," for instance, becomes "Nasa," because the regex only sees one long non-whitespace run and blindly lowercases everything after its first character.

Where this still gets it wrong

Even the shipped sentence-case version has real gaps, because it has no idea what an abbreviation is:

  • Abbreviations fool it completely. "He is 6 ft. tall and strong." splits into "He is 6 ft." and "tall and strong." because a period followed by a space is treated as a sentence end every time — there's no list of "Dr.", "Mr.", "etc.", "e.g." that get skipped. The output capitalizes "Tall" as if it started a new sentence, because as far as the regex is concerned, it did.
  • Manual line wraps get treated as sentence boundaries too. Since each line is processed independently before being rejoined with \n, a sentence that was soft-wrapped across two lines without ending punctuation still gets its second line's first letter capitalized — even though grammatically it's the middle of a sentence, not the start of one.
  • One detail I noticed while writing localized copy for this tool: a couple of the translated marketing blurbs claim the sentence-case mode "recognizes common abbreviations (Dr., Mr., etc.) that don't end sentences." That line isn't true of the actual code above — there's no abbreviation list anywhere in it. It's a good reminder to go check the source before believing your own product description.

None of this makes it useless — it still saves you from manually re-capitalizing a pasted paragraph — but it's pattern-matching on punctuation, not parsing grammar, and it's worth knowing exactly where that line is.

I cleaned this up into a small free tool if you want to see the four modes side by side: Online Case Converter. No sign-up, just paste and click.


Available in other languages

Top comments (0)