DEV Community

Cover image for Harness Score 1.0: measure AI harness maturity across Cursor, Claude Code, Windsurf, and more
Fernando Paladini
Fernando Paladini

Posted on

Harness Score 1.0: measure AI harness maturity across Cursor, Claude Code, Windsurf, and more

#ai #cursor #opensource #devtools

Two engineers. Same model. Same task.

One repo has AGENTS.md, scoped rules, tests in CI, and hooks that block rm -rf before the agent runs it. The other has a .cursorrules file from March and vibes.

Wildly different outcomes — and the model was never the variable.

In early 2026, Martin Fowler's site gave this practice a name: harness engineering — wrapping a coding agent with the right context, rules, tests, and guardrails so its output is reliable, not lucky. LangChain's engineering blog showed the same lesson quantitatively: improve the harness, not the model, and benchmark scores move.

The practical question for every team in 2026:

How harnessed is my repository — when Cursor, Claude Code, and Windsurf all read different config trees?

That's what harness-score answers. And today it hit 1.0.0.

TL;DR

  • npx harness-score — free, open-source, MIT, zero runtime dependencies
  • Scores any repo L0–L4 across six dimensions (108 points, 36 checks)
  • Multi-harness: Cursor, Claude Code, Windsurf, Cline, Continue, Codex, Copilot, and more — OR semantics, one scan
  • 100% deterministic: no LLM, no network, no telemetry — CI-gateable
  • 1.0 = stable API under semver (check IDs, JSON shape, CLI flags)
  • Ships as CLI + full guide + GitHub Action + README badges

What 1.0 actually means

The major bump is not a breaking change. If you were on v0.6.0, upgrading to 1.0.0 produces identical scores and output.

What changed is the contract:

Surface Stability promise
Check IDs (CTX-01 … HYG-08) Permanent identifiers — never repurposed
Report JSON (--json, typed API) Fields only added in minors
CLI flags & exit codes 0 pass · 1 gate fail · 2 usage error
Determinism Zero LLM · zero network · zero runtime deps — forever

Maturity model evolution (new checks, point totals) ships in minor versions and is flagged by --diff's maturityModelChanged. You can gate CI without fear of silent schema breaks.

One harness, not three: multi-tool OR semantics

When I first shipped harness-score, it was Cursor-first — Cursor exposed the richest harness surface (.mdc rules, hooks, skills, subagents, MCP). That was the right starting point.

But real teams don't pick one tool. Someone uses Cursor. Someone uses Claude Code. Someone is on Windsurf. Same repo, three config trees.

Since v0.4.0, the scanner uses OR semantics: each check asks "does any recognized tool provide this?" — not "does Cursor provide this?".

Examples:

Check dimension Any of these counts
Scoped rules .cursor/rules/*.mdc · .windsurf/rules/*.md · .clinerules/*.md · nested CLAUDE.md
Hooks .cursor/hooks.json · .claude/settings.json (hooks key)
Skills .cursor/skills/*/SKILL.md · .claude/skills/*/SKILL.md
Context guide root AGENTS.md · CLAUDE.md · GEMINI.md
Copilot instructions .github/instructions/*.instructions.md

You build one harness. Every tool inherits the parts it understands.

And since v0.5.0: adding a second tool can never lower your score. When multiple hooks configs exist, the one with the most registered events wins.

The report tells you which tools it detected:

  harness-score v1.0.0  ~/my-app

  Maturity: L2 · Guided   Score: 70/108 (65%)
  Detected: Cursor, Claude Code

  Context & Guides     ████████████████░░░░  80%  16/20 pts
  Skills & Commands    █████████████░░░░░░░  65%  11/17 pts
  Hooks & Guardrails   ░░░░░░░░░░░░░░░░░░░░   0%   0/14 pts
  Sensors & Feedback   ████████████████░░░░  80%  16/20 pts
  CI Feedback          ██████████████░░░░░░  71%  10/14 pts
  Hygiene & Safety     ███████████████░░░░░  74%  17/23 pts

  To reach L3: sensors ≥ 60%; ci ≥ 50%
Enter fullscreen mode Exit fullscreen mode

Same repo. Two tools. One score. One gap list.

The maturity ladder

Level Name What it means
L0 Unharnessed No persistent agent context
L1 Documented AGENTS.md (or equivalent) orients the agent
L2 Guided Rules, skills, hygiene in place
L3 Sensing Tests, types, CI verify the work
L4 Self-correcting Hooks close the loop before damage

Levels gate on dimension thresholds, not just total percentage. Beautiful docs with zero tests can't accidentally look mature.

Every failed check names what's missing and links to a remediation recipe in the guide.

Try it in 10 seconds 

npx harness-score
Enter fullscreen mode Exit fullscreen mode

Scan any path:

npx harness-score ./path/to/repo
Enter fullscreen mode Exit fullscreen mode

Machine-readable output:

harness-score --json > report.json
Enter fullscreen mode Exit fullscreen mode

Markdown report:

harness-score --md harness-report.md
Enter fullscreen mode Exit fullscreen mode

README badge:

harness-score --badge harness-badge.svg
Enter fullscreen mode Exit fullscreen mode

CI gate (exit 1 below L3):

harness-score --min-level 3
Enter fullscreen mode Exit fullscreen mode

Track progress over time:

harness-score --json > baseline.json
# ... improve the harness ...
harness-score --diff baseline.json
Enter fullscreen mode Exit fullscreen mode

Gate CI so maturity only ratchets up

Minimal workflow:

# .github/workflows/harness.yml
name: Harness Score
on: [push, pull_request]

jobs:
  audit:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: paladini/harness-score/action@main
        with:
          min-level: '3'
          badge: harness-badge.svg
Enter fullscreen mode Exit fullscreen mode

With PR comments (opt-in — needs pull-requests: write):

      - uses: paladini/harness-score/action@main
        with:
          min-level: '3'
          badge: harness-badge.svg
          comment: 'true'
Enter fullscreen mode Exit fullscreen mode

The action posts (and updates) a sticky comment showing score deltas — L2 → L3 — right on the PR.

No AI in the scanner (on purpose)

The scanner reads your filesystem and parses config. That's it.

No LLM calls. No network requests. No telemetry. Same repo, same commit, same score — on your laptop or in CI, forever.

An AI-powered "does this repo look well-configured?" tool might feel smarter, but it would be non-reproducible. You couldn't gate a pipeline on it, diff two runs, or trust that 80% today means the same thing next month.

Harness maturity belongs in the bucket Fowler describes: computational checks you can put in a pipeline.

The project dogfoods its own model: the harness-score repo scores L4 · 108/108 and gates that level in CI.

What's in the box

Piece What Link
CLI npx harness-score — scan, JSON, markdown, badge, diff, gate npm · JSR
Guide Harness engineering, maturity model, check catalog, multi-harness chapter paladini.github.io/harness-score
GitHub Action Scan on push, badge, gate, optional PR comments action/
Badges Branded SVG pill (harness · L4) and share cards Embed snippets
Plugins /harness-audit for Cursor & Claude Code (in repo; marketplace pending) plugins/

Published on npm, GitHub Packages (@paladini/harness-score), and JSR — all from the v1.0.0 release.

FAQ

Do I need Cursor?
No. Since v0.4.0, equivalent artifacts across Cursor, Claude Code, Windsurf, Cline, Continue, Codex, Copilot, and more all count. Universal artifacts (tests, CI, types, .gitignore, lockfiles) score the same regardless of tool.

Does it judge code quality?
No. It measures harness infrastructure — the files and configs that steer and verify an agent's work. A stale rule scores like a fresh one; that's the honest ceiling of any deterministic scanner.

Is it free?
Yes. MIT licensed. Runs on your machine or in your CI.

Can I contribute checks for another ecosystem?
Please — issues and PRs welcome. Check IDs are stable public API; new checks ship in minors.

Run it and tell me your level 

npx harness-score
Enter fullscreen mode Exit fullscreen mode

Drop your level in the comments — curious how many L0 repos are out there pretending to be "AI-native."

Links:

Top comments (0)