Digit Tokenization: Why a Comma Changes Your LLM's Math Answer

I spent an afternoon debugging an invoice pipeline where the model summed line items. Someone had added a "normalization" step that stripped thousands separators before the prompt — 1,250,000 became 1250000. Cleaner input, obviously better. The sums got worse.

The model never saw the number. It saw a sequence of tokens, and stripping the commas changed which digits landed in which token. That's digit tokenization, and it is the most under-discussed failure mode in numeric LLM work — it silently degrades arithmetic, numeric extraction, and even retrieval over part numbers and IDs.

TL;DR

• Modern BPE tokenizers (cl100k_base, o200k_base, Llama 3) split runs of digits into groups of up to three, greedily from the left. 1234567 becomes 123|456|7 — the least-significant digits land in a ragged trailing group.
• Because grouping is left-anchored, two numbers of different lengths have completely different token structures. Place values do not line up in the token stream, which is exactly what carry-propagating arithmetic needs.
• Thousands separators force right-aligned 3-digit groups (1|,|234|,|567), restoring place-value alignment at a cost of ~2 extra tokens per number. This is why the "write numbers with commas" folklore actually works.
• Group tokens are atomic. The embedding for 456 has no compositional relationship to 4, 5, 6 — the model learns magnitude per token, not from digits.
• In production, don't fix arithmetic with prompting. Route it to a tool. Use formatting for reading, comparing, and extracting numbers.

What is digit tokenization, and why does it break LLM arithmetic?

Digit tokenization is the pre-tokenizer rule that decides where a run of digits gets cut before BPE merges run. In the tiktoken family (cl100k_base for GPT-4-era models, o200k_base for later OpenAI models) the regex contains an alternative equivalent to \p{N}{1,3}, matched greedily left to right. Llama 3's tiktoken-based pre-tokenizer uses a similar up-to-3-digit rule. Llama 2 and Gemma went the other way and split digits individually.

The consequence: a 7-digit number is three tokens, and the boundaries depend on the total length.

Text	Pieces (cl100k/o200k family)
917	917
4821	482 1
1234567	123 456 7
12345678	123 456 78
1,234,567	1 , 234 , 567

Look at 4821. The ones digit — the digit every addition starts from — is alone in a second token, while 917's ones digit is buried inside a single token alongside its hundreds and tens. There is no positional handle the model can use to align them. It has to learn that relationship in the weights, per length combination.

Then there's the vocabulary problem. 456 is one atomic token whose embedding is learned independently of 4, 5, and 6. Arithmetic over 3-digit chunks is base-1000 arithmetic with a thousand-entry lookup and no compositional structure — plus a carry rule between chunks whose boundaries move depending on operand length. That the models do this as well as they do is the surprising part.

Why does adding commas change the answer?

Because commas convert left-anchored grouping into right-anchored grouping, and place value is anchored on the right.

Compare adding 4821 + 917:

raw:      482|1     +  917
comma'd:  4|,|821   +  917

In the comma'd form, the last digit-token of each operand covers the ones through hundreds place. The groups line up by magnitude. In the raw form they don't line up at all, and the alignment changes again if you add one more digit to either operand.

This is why "format numbers with thousands separators" keeps showing up as folk advice, and why it is not superstition. It costs about two extra tokens per number and buys you a token layout that mirrors the decimal place structure the task actually depends on.

A second trick works for the same reason: forcing single-digit tokens. If you tell the model to write out 1 2 3 4 5 6 7 with spaces during a chain of thought, each digit becomes its own token and every digit gets its own position. Slower, more tokens, more reliable on the digits themselves.

Neither trick makes arithmetic reliable. They make it less unreliable. Keep that ordering straight when you decide what to ship.

How do I inspect what my tokenizer does to numbers?

Run it. For OpenAI-family encodings, tiktoken gives exact boundaries:

import tiktoken

enc = tiktoken.get_encoding("o200k_base")

samples = [
    "917", "4821", "1234567", "12345678",
    "1,234,567", "1 234 567", "$1,234,567.00",
    "2026", "20260809", "SKU-0004821",
]

for s in samples:
    ids = enc.encode(s)
    pieces = [enc.decode([i]) for i in ids]
    print(f"{s:<16} {len(ids):>2} tokens  {pieces}")

Anthropic doesn't publish a tokenizer, so you can't get boundaries directly for Claude — but you can get exact counts from the API, and count deltas tell you most of what you need:

from anthropic import Anthropic

client = Anthropic()

def ntok(text: str) -> int:
    return client.messages.count_tokens(
        model="claude-sonnet-4-5",
        messages=[{"role": "user", "content": text}],
    ).input_tokens

base = ntok("")  # message scaffolding overhead
for s in ["1234567", "1,234,567", "1 2 3 4 5 6 7"]:
    print(f"{s:<16} {ntok(s) - base}")

Do not estimate Claude token counts with tiktoken. It's OpenAI's tokenizer; on numeric and code-heavy text the error is large enough to blow through a context budget you thought you had headroom on.

Why does this break RAG retrieval and not just math?

Because embedding models inherit the same pre-tokenizer, and because numbers carry meaning in their magnitude that cosine similarity cannot see.

Three failure modes I've hit in production:

Numeric near-duplicates collapse. $1.2M and $12M share almost all their tokens and land close together in embedding space. Cosine similarity has no notion that one is ten times the other. If your query has a numeric constraint ("deals over $10M"), vector search will not enforce it — push numeric predicates into metadata filters or SQL and use the vector index only for the semantic part.

Identifiers tokenize inconsistently. A query for invoice 4821 against a document containing Invoice #04821 differs by a leading zero, and that zero reshuffles every group boundary (482|1 vs 048|21). Same for RTX3080 vs RTX 3080. Normalize identifiers on both the index side and the query side, in code, before either touches the model.

Chunk boundaries split numbers. Character-based chunkers happily cut 12,345,678 in half. Now you have a chunk ending in 12, and one starting with 345,678, both indexed, both wrong, and neither will ever match the query. Chunk on token or sentence boundaries, or at minimum add a regex guard that refuses to split inside a digit run.

What should I do in production?

1. Route exact arithmetic to a tool. This is the whole answer for anything where being off by one matters. The tool description does real work here — state the threshold explicitly, because three digits is where a number stops being a single token:

import anthropic

client = anthropic.Anthropic()

TOOLS = [{
    "name": "evaluate",
    "description": (
        "Evaluate an exact arithmetic expression. Use this for ANY addition, "
        "subtraction, multiplication, division, or comparison involving numbers "
        "with more than 3 digits. Do not compute those in your head."
    ),
    "input_schema": {
        "type": "object",
        "properties": {
            "expression": {
                "type": "string",
                "description": (
                    "Arithmetic in Python syntax. Digits only — no thousands "
                    "separators, no currency symbols. e.g. '1250000 + 48210'"
                ),
            }
        },
        "required": ["expression"],
        "additionalProperties": False,
    },
    "strict": True,
}]

response = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=4096,
    tools=TOOLS,
    messages=[{"role": "user", "content": "Line items: 1,250,000 / 48,210 / 9,905. Total?"}],
)

Evaluate with an AST-based evaluator over a whitelist of node types, never bare eval. For money, use decimal.Decimal — you did not escape float rounding just because an LLM is upstream.

2. Keep numbers verbatim through extraction; normalize afterward, in code. The model's job is to locate the number and copy it. Reformatting is a deterministic post-step. Ask for numbers as strings in your schema so a JSON parser can't silently coerce 007 to 7 or a 20-digit ID to a float.

3. Format for the model when the task is reading, not computing. Thousands separators when comparing magnitudes or reading tables. Digit-by-digit with spaces when the model must transcribe or manipulate a long literal.

4. Validate cheaply. Regex-check that the extracted digit count matches the source. Re-sum in Python. These catch the class of error where a group boundary swallowed a digit, which is exactly the error a human reviewer skims past.

When is prompting enough?

Numbers of three digits or fewer are single tokens, and the model handles them roughly like vocabulary — small-number arithmetic is mostly fine. Comparisons between numbers of the same length are usually fine, because the groups align by construction. Anything past that — multi-digit multiplication, long sums, percentage-of-total across a table, date arithmetic on YYYYMMDD strings — is where the token layout stops cooperating and you should be calling a tool.

The short answer

A comma changes your LLM's math answer because digit tokenization chunks runs of digits into groups of up to three, greedily from the left, so 1234567 becomes 123|456|7 while 1,234,567 becomes 1|,|234|,|567. The comma'd form aligns token boundaries with decimal place value; the raw form leaves the ones digit in a ragged trailing group whose position shifts with the number's length. Since the model must do carry propagation across atomic, non-compositional group tokens, that alignment is load-bearing. Formatting numbers with separators measurably helps — but the durable fix is to stop asking the model to do exact arithmetic at all and hand it a calculator instead.