DEV Community

Cover image for Typos don't break LLM prompts. One missing quote mark does.
vadim albarov
vadim albarov

Posted on

Typos don't break LLM prompts. One missing quote mark does.

Everyone who types prompts has wondered the same thing. Does the model care that I wrote "teh" or forgot a comma? Should I clean up my prompt before hitting enter?

I ran a test to find out. The short answer: stop fixing your spelling. Start checking your quotes and colons.

What I tested

I wrote a set of tasks with short, checkable answers. Arithmetic, extraction from a passage, list filtering with "except" and "not", a Python snippet's output, counting words inside quotes, and so on. Then I broke the prompts in controlled ways and sent every version to every model, several times each, in a fresh session every time.

Two prompt sets:

  • Easy set. 12 one-line tasks, each hand-written at five error levels, from clean to "heavy typos plus broken punctuation".
  • Hard set. 9 long prompts (200 to 450 words) where the question depends on a detail buried in the text. Here I separated the two kinds of damage:
    • Spelling noise: a script misspells 35% or 70% of the words, but never touches the words the answer depends on.
    • Structural break: the spelling is perfect, but exactly one punctuation mark that carries meaning is wrong. A missing closing quote. A colon dropped before a list. A comma moved around "except". 9.45 instead of 9:45. resign instead of re-sign.
    • Two realistic profiles on top: non-native grammar (articles and verb forms off, spelling fine) and voice-to-text (no punctuation, no capitals, numbers spelled out, "resign" for "re-sign").

Models: Claude Opus 5, Opus 5.5, Sonnet 5, Haiku 4.5, Fable 5.1 (all at low and medium effort), plus seven local models through Ollama: gemma4, qwen3 14B, phi4 14B, granite 4.2, ornith-1.5 9B, llama 3.1 8B, mistral 7B.

Total: about 4,900 sessions. An answer counts as correct if the ideal answer appears in the reply, even with an explanation attached.

Result 1: spelling and grammar cost nothing

Here is the hard set. Each cell is the percent of correct answers.

model clean 70% typos non-native grammar voice-to-text one punctuation break
Opus 5 100 100 100 100 88
Opus 5.5 100 98 100 100 88
Fable 5.1 100 100 100 98 92
Haiku 4.5 100 100 100 98 88
Sonnet 5 100 100 100 89 77
granite4.2 93 67 89 56 88
phi4 14B 74 74 85 78 62
gemma4 78 78 81 44 79
qwen3 14B 67 74 85 56 62
ornith-1.5 9B 74 48 70 52 42
llama 3.1 8B 44 56 63 33 38
mistral 7B 52 26 41 56 38

Read the Claude rows first. With 70% of the words misspelled, every one of them scored the same as on clean text. Non-native grammar: 100% across the board. This is the prompt that scored 100%:

shop sellin pens 3dolar each an notbooks 5 dolar,each tom buys 4pen an 2notbook.how much he pay totaly,anser numbr only

Now read the last column. One punctuation mark, with perfect spelling everywhere else, and the same models lose 8 to 23 points.

Result 2: which punctuation mark breaks which model

Not every break matters. Most were recovered from context by every Claude model. Two were not.

break Opus 5 Opus 5.5 Fable Haiku Sonnet
missing closing quote 0 33 33 0 17
colon dropped before a list 100 67 100 100 0
comma moved around "except" 100 100 100 100 100
European decimals (1.500 at 2,50) 100 100 100 100 100
9.45 instead of 9:45 100 100 100 100 100
managers' → manager's 100 100 100 100 100
resign instead of re-sign 100 100 100 100 100
comma dropped in "not closed, and assigned to Lee" 100 100 100 100 100

The missing quote. The task: count how many times "report" appears inside a quoted paragraph, not in the text after it. Remove the closing quote and every model counts the three extra "report"s after the paragraph. Answer 8 instead of 5. Opus 5.5 even wrote "the closing quote is missing, so I counted all" and still gave the wrong number. It noticed the problem and followed the broken structure anyway.

The dropped colon. "Give the plural form of each word below keeping the order mouse child foot analysis reply with the plurals only comma separated". Sonnet answered mice, children, feet, analyses, replies. Every local model did the same. The word boundary between instruction and data was gone, so "reply" became data.

Everything else was recovered, including the genuinely ambiguous one. "List the tickets that are not closed and assigned to Lee" can mean two things once the comma is gone. All five Claude models picked the intended reading every time, because the sentence started with "For Lee's standup".

Result 3: dictation is a different story

The voice-to-text column is where the small models fall apart. gemma4 goes from 78 to 44. granite from 93 to 56. Two things do the damage: spelled-out numbers ("three hundred forty pallets across twenty two trailers") and, again, the missing colon before a list. Every local model scored 0% on the dictated list task.

The frontier models mostly shrug it off, with one honest exception. "fifteen hundred units at two fifty each": Sonnet read "two fifty" as $250 in six trials out of six and answered 375,000 instead of 3,750. Fable did it once. Haiku turned "extension two zero one" into 2001 once. These are not model bugs. Spoken numbers are ambiguous, and the model has to guess.

Things that did not help

  • Effort level. Low versus medium made no difference for any model on any task. Structural breaks are not a reasoning problem, so more thinking does not fix them.
  • Warning the model. I prepended "the message below may contain spelling, grammar, and punctuation mistakes, answer according to the writer's likely intent". Zero effect on Opus 5, Haiku, and Sonnet. A few points on Opus 5.5 and Fable. Mixed or negative on the local models.

What degrades first: the format, not the answer

Sloppy prompts get slightly sloppier replies before they get wrong ones. Asked for "the number only", Fable and Opus 5.5 add a line of working in about 20% of runs on any prompt. Haiku, on the easy set, drifted into "Yes. If all bloops are razzies..." explanations as the grammar got worse, and at the heaviest level refused four times out of eight because "bloops and razzies are not real words". With clean grammar it had answered the same nonsense-word question fine.

What to actually do

  1. Don't fix spelling or grammar. The model does not care, and neither should you.
  2. Do check the punctuation that carries structure: closing quotes, the colon before a list, commas around "except" and "not", and anything separating your instructions from your data.
  3. If you dictate prompts, re-read the numbers and the list boundaries before sending.
  4. Small local models are much less forgiving. If you run a 7B to 14B model, treat every delimiter as load-bearing and avoid dictation.

Caveats

Twelve short tasks and nine long ones, three trials per cell, one random seed for the typo generator. The voice-to-text prompts are hand-simulated, not real speech engine output. Claude models were called through the Claude Code CLI with a minimal system prompt and tools disabled. Local models ran at Ollama defaults with thinking off. The harness and every prompt and reply are in the repo linked below, so you can rerun it with your own tasks.

Repo with the harness, all prompts, and every raw reply: github.com/valbarov/prompt-noise-test

Top comments (2)

Collapse
 
blobdole profile image
Doug •

This is a bad sign for texting. Lazy texting still has great spelling because of autocomplete, but no punctuation. Someone needs to better train an LLM to respond to a distracted college kid texting a group chat while parked halfway into a crosswalk.

Some comments may only be visible to logged-in visitors. Sign in to view all comments.