Forem

Cover image for What We Learned From Analyzing 28,000 Production AI System Prompts
Kristofer Jussmann
Kristofer Jussmann

Posted on • Originally published at blog.kaelux.dev

What We Learned From Analyzing 28,000 Production AI System Prompts

Over the last few months developing PromptTriage, we've collected and analyzed over 28,000 production system prompts. Most are bloated, contradictory, and actively hurt reasoning quality.


📉 Anti-Pattern 1: The "Emotional Blackmail" Scaffold (14%)

Anti-Pattern Prevalence in 28,000 Production System Prompts
Caption: Anti-Pattern Prevalence in 28,000 Production System Prompts. (Full-width hero chart)

Over 14% still contain emotional appeals:

"Take a deep breath. If you miss a bug, the company will lose millions."

Why it fails: Modern RLHF has trained out the "anxiety" response. Emotional context distracts self-attention from the actual task.


🏗️ Anti-Pattern 2: The "Just in Case" Clause (62%)

62% of prompts over 300 words contained contradictory constraints. Our Study E data proved short prompts (<50 words, scoring 80.1/100) consistently outperform long ones (>300 words, 66.9/100).


🎭 Anti-Pattern 3: The "World Class Expert" Trap (80%)

Nearly 80% started with "Act as a world-class expert." Our Study C proved this provides zero lift on modern models (~78/100 with or without it).


🚀 The Fix: The 50-Word Rule

  1. State the role (10 words): "Extract data from SEC filings."
  2. State negatives (20 words): "Do not include pleasantries. Do not output markdown."
  3. Halt.

PromptTriage compresses 500-word prompts to the optimal 50-word framework.

Top comments (0)