I Built a Prompt Scoring Engine — Here's What 100,000 Prompts Taught Me

When I started building a prompt scoring engine, I expected the results to be noisy. Prompt quality is subjective, right? What counts as "good" depends on the use case.

I was wrong. After scoring over 100,000 prompts across dozens of categories — code generation, content writing, data analysis, creative work — the same five failure patterns appeared over and over again. Not occasionally. In roughly 85% of low-scoring prompts.

This article breaks down what those patterns are, how the scoring works, and gives you a practical checklist you can apply to your own prompts immediately.

How Prompt Scoring Works

Before diving into the failures, it helps to understand what "scoring" actually means in this context. The engine evaluates prompts across several dimensions:

Structural Completeness

Does the prompt contain the fundamental components needed for a clear instruction? At minimum, this means:

✅ A defined role or situation
✅ A specific task or action
✅ Relevant context or constraints
✅ Output format specification

A prompt missing two or more of these components scores poorly regardless of how well-written it is. You can write beautiful prose, but if you haven't told the model what format you want, you're rolling dice.

Specificity Score

This measures how precise the language is. Compare these two instructions:

Low specificity:  "Write something about our product"
High specificity: "Write a 200-word product description for our 
                   SaaS dashboard targeting engineering managers, 
                   emphasising time-saved metrics"

The scoring engine analyses word choice, looking for quantifiable values, named entities, and concrete nouns versus abstract ones. Prompts heavy on adjectives like "good," "nice," "better," and "interesting" score lower because these words carry almost zero signal for a language model.

Constraint Coverage

Constraints are the guardrails that prevent the model from going off-piste. The engine checks for:

• Positive constraints: what the output should include
• Negative constraints: what the output should avoid
• Boundary constraints: length limits, format requirements, scope limits

The most effective prompts typically have a ratio of roughly 3:1 positive to negative constraints. Too many negative constraints ("don't do this, don't do that") can actually confuse models, but having none almost always produces unfocused output.

Coherence and Readability

A prompt can have all the right components but still be poorly structured. The engine evaluates whether instructions are logically ordered, whether there are contradictions, and whether the prompt could be parsed unambiguously.

The 5 Most Common Prompt Failures

Here's where the data gets interesting. These five patterns account for the vast majority of low-scoring prompts.

Failure 1: Vague Context (Found in 72% of Low-Scoring Prompts)

This is the single most common failure. The prompt asks for something without establishing why, for whom, or in what situation.

Failing prompt:

Explain microservices architecture.

Fixed prompt:

Explain microservices architecture to a team of backend developers 
who currently maintain a Django monolith handling 50,000 daily 
active users. They're evaluating whether to decompose the 
authentication and billing modules. Focus on operational complexity 
trade-offs rather than theoretical benefits.

The second version gives the model a concrete scenario. It knows the audience's current stack, their scale, which modules they're considering, and what angle to take. The output will be dramatically more useful.

Failure 2: Missing Output Format (Found in 64% of Low-Scoring Prompts)

When you don't specify format, the model defaults to whatever it considers "standard" — usually a few paragraphs of prose. This is fine for casual questions but terrible for professional use.

Failing prompt:

Give me a code review of this pull request.

Fixed prompt:

Review this pull request and provide feedback in the following format:

## Summary
One paragraph overview of the changes.

## Issues (Critical)
Numbered list of bugs or security concerns that must be fixed.

## Issues (Minor)
Numbered list of style, naming, or optimisation suggestions.

## Positive Observations
What's done well — important for team morale.

Each issue should reference the specific file and line number.

The structured output format transforms a vague request into a repeatable, professional-grade code review template.

Failure 3: Ambiguous Task Scope (Found in 58% of Low-Scoring Prompts)

Many prompts contain what I call "scope creep by omission" — they don't define boundaries, so the model either does too much or too little.

Failing prompt:

Help me with my database.

What does "help" mean here? Schema design? Query optimisation? Migration planning? Debugging a specific error? The model will guess, and it'll probably guess wrong.

Fixed prompt:

I have a PostgreSQL 15 database with a `users` table (2.3M rows) 
and an `orders` table (18M rows) joined on user_id. The following 
query takes 12 seconds:

SELECT u.email, COUNT(o.id) 
FROM users u 
JOIN orders o ON u.id = o.user_id 
WHERE o.created_at > '2025-01-01' 
GROUP BY u.email 
ORDER BY COUNT(o.id) DESC 
LIMIT 100;

Diagnose why this is slow and suggest index or query changes. 
Show the EXPLAIN ANALYZE output I should expect after your fix.

Failure 4: No Persona or Role Definition (Found in 47% of Low-Scoring Prompts)

The model's "voice" shifts significantly based on who it thinks it's writing for. Without a persona, you get a neutral, encyclopaedic tone that's rarely what anyone actually wants.

This doesn't mean you need to write "You are an expert in X." That's the shallow version. Effective persona definition includes:

You are a staff engineer at a fintech company writing internal 
documentation. Your audience is new hires in their first week. 
The tone should be welcoming but technically precise. Avoid jargon 
that isn't defined in context. Use "we" and "our" to refer to 
company practices.

The difference between a generic explanation and one written as if by a specific person for a specific audience is the difference between a Wikipedia article and a useful internal doc.

Failure 5: Single-Turn Overloading (Found in 41% of Low-Scoring Prompts)

This is the temptation to cram everything into a single prompt when the task actually requires multiple steps.

Failing prompt:

Research the top 10 competitors in the UK project management SaaS 
market, analyse their pricing strategies, identify gaps in their 
feature sets, write a competitive analysis report, and create a 
go-to-market strategy for our new product launch.

This prompt asks for roughly a week's worth of analyst work in a single shot. The model will produce superficial coverage of everything rather than deep coverage of anything.

Better approach:

Step 1: "List the top 10 UK-based project management SaaS companies 
by market share. For each, provide: company name, estimated ARR, 
primary target segment, and pricing model."

Step 2: [Using Step 1's output] "For the top 5 by market share, 
analyse their pricing pages. Create a comparison table with columns: 
free tier limits, per-seat price, enterprise pricing model, and 
which features are gated behind higher tiers."

Step 3: [Using Steps 1-2] "Based on this competitive data, identify 
3 underserved segments or feature gaps..."

Breaking complex tasks into a chain of focused prompts produces significantly better results than single-turn overloading.

A Practical Scoring Checklist

You don't need a scoring engine to evaluate your prompts. Here's a manual checklist distilled from the data:

## Prompt Quality Checklist

### Structure (0-25 points)
- [ ] Contains a clear situation/role      (5 pts)
- [ ] Has a specific, bounded task         (10 pts)
- [ ] Includes relevant context            (5 pts)
- [ ] Specifies output format              (5 pts)

### Specificity (0-25 points)
- [ ] Uses concrete nouns over abstractions (5 pts)
- [ ] Includes quantifiable values          (5 pts)
- [ ] Names specific technologies/tools     (5 pts)
- [ ] Defines scope boundaries              (5 pts)
- [ ] Avoids vague adjectives               (5 pts)

### Constraints (0-25 points)
- [ ] Has at least one positive constraint  (5 pts)
- [ ] Has at least one negative constraint  (5 pts)
- [ ] Defines length or depth               (5 pts)
- [ ] Specifies language/regional prefs     (5 pts)
- [ ] Sets quality benchmarks               (5 pts)

### Coherence (0-25 points)
- [ ] Instructions are logically ordered    (5 pts)
- [ ] No contradictions                     (5 pts)
- [ ] Single clear objective per prompt     (5 pts)
- [ ] Could be parsed by a stranger         (5 pts)
- [ ] Appropriate complexity for one turn   (5 pts)

Scoring guide:

• 0-40: Your prompt needs significant restructuring
• 41-60: Passable but leaving quality on the table
• 61-80: Solid prompt that will produce good results
• 81-100: Excellent — professional-grade prompt engineering

What the Data Actually Shows

The most surprising finding wasn't about what's in the prompt — it was about prompt length versus quality.

There's a widespread assumption that longer prompts are better. The data doesn't support this. The sweet spot sits at roughly 75-200 words for most tasks. Below 75 words, prompts tend to lack crucial context. Above 200 words, they often contain redundancy or contradictions that confuse the model.

The correlation between score and output quality was strongest in the constraint coverage dimension. Prompts with well-defined constraints outperformed prompts with better situation-setting but no constraints. In other words: telling the model what NOT to do is more impactful than telling it who you are.

If you want to experiment with automated scoring, AI Prompt Architect has a free prompt scorer that evaluates across these dimensions in real time as you type. It's useful for building intuition about what the engine looks for — you start to internalise the patterns after scoring a dozen or so prompts.

Key Takeaways

1. Vague context is the #1 killer. Always establish who, why, and for whom.
2. Specify your output format. Never let the model guess at structure.
3. Bound your scope. If you can't state the task in one sentence, split the prompt.
4. Constraints beat adjectives. "Maximum 300 words, no bullet points, British English" beats "make it good."
5. 75-200 words is the sweet spot for most single-turn prompts.

The gap between average and excellent prompt engineering isn't talent — it's discipline. These patterns are learnable, repeatable, and measurable.

---

This article was originally published on AI Prompt Architect. AI Prompt Architect is a free prompt engineering platform with the STCO Framework, multi-model testing, and 500+ templates.