Hook
Most self-assessment tools score you the same way: answer questions, multiply by weights, sum it up. Higher total = more ready.
That model has a structural flaw. A team that doesn't use Git — but aces documentation, AI policy, and project fit — can score high. Yet without version control, every large AI-generated change is an irreversible overwrite. No amount of strength elsewhere compensates.
Call it the honor student problem: additive scoring rewards averages when what actually matters is the weakest link. The article walks through how I designed a scoring model that caps the total score when a fatal precondition is missing — and why the whole mechanism collapses into a single Math.min().
Target Audience
Developers and tech leads who:
- Are evaluating (or building) readiness / maturity assessments for AI-assisted development
- Design scoring, ranking, or evaluation logic and care about the failure modes of weighted averages
- Enjoy "small design decision, big consequence" write-ups grounded in a real open-source implementation
Key Sections
1. The Honor Student Problem
- How typical readiness assessments work: weighted average across categories
- Failure mode: catastrophic weakness in one area is diluted by strength in others
- Concrete example: no Git + perfect everything else = high score under additive scoring
- Why this matters more for AI-driven development: AI multiplies change volume, so missing safety rails are amplified, not averaged away
2. Bottleneck Scoring: Average for Progress, Cap for Preconditions
- Keep the weighted average — it's good at expressing continuous improvement
- Add a second layer: fatal preconditions that cap the total score when absent
- The entire mechanism reduces to
Math.min(baseScore, cap) - Analogy: Liebig's law of the minimum (the barrel with one short stave)
3. The Six Preconditions and Their Caps
- No version control (Git) → capped at 49
- Almost no written specs → capped at 49
- No task/ticket management → capped at 59
- No human review / production approval → capped at 59
- No automated tests and no change checklist → capped at 59
- No rules on what must not be fed to AI → capped at 69
- When multiple fire, the lowest cap wins (rate-limited by the worst bottleneck)
4. Why 49, 59, 69 — Caps as Level Ceilings
- 100-point scale, 5 axes: Documentation 25 / Process 25 / Quality Assurance 20 / AI Usage 15 / Project Fit 15
- 5 levels (Lv1: 0–29 … Lv5: 85–100)
- Caps sit deliberately just below level boundaries (50, 70): a cap encodes "the highest level you can reach while carrying this gap"
5. Proving It Works: Test Cases as Design Documentation
- All answers max → 100, Lv5
- No Git, everything else perfect → forced to 49 (Lv2)
- Multiple caps at once → the minimum (49) is applied
- Deterministic scoring (same input, same output) guaranteed by Vitest unit tests
6. Design Decisions That Followed From the Same Principle
- "I don't know" scores 0.2, not 0 — not knowing is different from not having
- Solo developers: team-only questions (e.g., PR review practice) get 0.5× weight
- Recommendations prioritized by impact (axis points / questions per axis × unmet degree), organized into a now / 1-month / 3-month roadmap
- Strength selection enforces axis diversity
- Of 8 development phases evaluated for AI fitness, documentation is the only one that can never be "not recommended"
7. The Tool Itself (Briefly)
- Fully client-side static app: TypeScript, React 19, Vite 8, Tailwind CSS 4, shadcn-ui; GitHub Pages
- Zero external transmission — answers stay in LocalStorage/IndexedDB, verified automatically with Playwright E2E tests
- Scoring logic versioned (SCHEMA_VERSION); old results get an "outdated version" badge
- 5 languages, MIT license, just shipped — no usage numbers to brag about, and don't pretend otherwise
Estimated Length
2,000–2,400 words (design deep-dive / engineering decision write-up)
Tone Notes
- This is a "one design decision, examined honestly" piece — the star is the scoring model, not the tool. Keep the tool as the implementation vehicle, mentioned mostly at the end.
- Lead with the honor student problem as a relatable trap; most readers have built or used an additive rubric.
- Use a before/after code-style diagram (additive score vs capped score for the no-Git team) to make the flaw visceral.
- Be candid that the implementation is almost embarrassingly simple (
Math.min) — the value is in choosing the caps, not the code. - Do not overstate the project: it's a personal OSS tool published in July 2026 with no track record yet. "A project I just shipped" is the right register.
- End with a question inviting readers to name the bottleneck their own team is averaging away.
SEO / Discoverability
- Primary keywords: "AI readiness assessment", "scoring system design", "weighted average problems"
- Secondary: "maturity model scoring", "bottleneck scoring", "self-assessment tool design"
- The generalizable lesson (cap vs average in any evaluation system) makes this shareable beyond the AI niche — frame the opening so rubric designers of any kind feel addressed
Top comments (0)