Write for the machine

Write for the machine, not the expert

The effect of abbreviation, syntax, redundancy, and filler words on structured information extraction

Does how we write matter when feeding unstructured text to an LLM? Experiment 1 isolates this question using a deliberately simple test case: a single flat payroll agreement containing 11 fields.

---

Experimental Setup

• Dataset: 1,000 rows of text total (the same payroll agreement written in 5 distinct styles across 200 unique workers).
• Model: Qwen2.5-1.5B-Instruct in GGUF format (tested across a wide range of quantizations from Q2 to FP16 in a CPU-only environment).
• Inference Pipeline: To prevent malformed JSON, we utilized grammar-constrained decoding via LlamaGrammar.from_json_schema(), restricting token generation to the target structure.
• Metric: Accuracy was measured using field-level extraction quality:

Accuracy = True Positives / Total Unique Fields

---

Here is the exact JSON used as the target baseline for a test case:

{
            "first_name": "Layla",
            "last_name": "Silva",
            "worker_type": "contractor",
            "class_name": "Boxing",
            "pay_type": "per_class",
            "class_rate": 40.0,
            "bonus_amount": 4.0,
            "bonus_threshold": 8,
            "head_count_basis": "check_ins_late_cancels",
            "holiday_rate_multiplier": 1.5,
            "cap_per_session": 100.0
 }

S1
Label
Shorthand
Description
Abbreviations, symbols, no verbs
Example
Layla Silva [C]: Boxing $40/class. $4/head >8 att (CI+LC). Holiday 1.5x. Cap $100/session.

S2
Label
Spelled-out
Description
Abbreviations replaced with full words, colons as separators
Example
Layla Silva [contractor]. Rate: $40/class — Boxing. $4/head when attendance exceeds 8 (check-ins and late cancels). Holiday rate: 1.5x standard. Session cap: $100.

S3
Label
Explicit prose
Description
Full sentences, one clean statement per field
Example
Layla Silva operates as an independent contractor. She is paid $40 for each Boxing class she teaches. For every attendee above 8, she receives $4 extra, counting check-ins and late cancels. Public holidays attract a 1.5x multiplier on her $40 base rate. Session earnings are capped at $100.

S4
Label
Explicit redundant prose
Description
Same as S3 but key values repeated across sentences
Example
Layla Silva is hired as a contractor. Her Boxing rate is $40 per class. Each Boxing class she teaches pays $40. For Boxing classes, a per-head bonus of $4 applies for each attendee above 8, counting check-ins and late cancels. The $4 per-head threshold is 8. The Boxing rate of $40 increases to $60.00 on holidays (1.5x the $40 base). Per-session Boxing earnings are limited to $100 in total.

S5
Label
Explicit prose with filler
Description
Same as S3 but legal boilerplate text introduced.
Example
Layla Silva is contracted as a self-employed instructor. Her contractual rate for Boxing is $40 per class. For the purposes of this arrangement, she is entitled to a per-head bonus of $4 for each attendee above 8, counting check-ins and late cancels. As per the agreed terms, work on designated public holidays is paid at 1.5 times the standard rate. Total per-session earnings are capped at $100.

---

Key Findings

The results highlight a distinct hierarchy in how language style impacts extraction accuracy:

• Spelling out words is paramount: Replacing cryptic abbreviations with full words (S1 to S2) yielded the largest performance jump, nearly 11 percentage points.
• Syntax matters less than semantics: Moving from spelled-out shorthand to full grammatical sentences (S2 to S3) made no meaningful difference. If the vocabulary is explicit, the model extracts efficiently without prose.
• Redundancy degrades accuracy: Repeating key values across sentences (S4) underperforms. Instead of providing helpful "anchors," duplication introduces token ambiguity that misleads the model.
• Filler text introduces noise: Adding filler text like “subject to the terms of the agreement” also had a negative impact.