SpecJudge: which AI model is right-sized for your project? Ask your specs

Joaquín Ruiz — Tue, 21 Jul 2026 12:42:30 +0000

When you finish planning a project and it's time to actually build it, there's a small decision that quietly costs money: which model do you use?

Pick one that's too powerful for the job and you pay for capability the project never touches. Pick one that's too weak and it can't deliver — so you pay and get nothing.

I got tired of guessing, so I built a tool that answers it. It's called SpecJudge, it's open source, and it runs entirely on your machine.

The idea

If you work with Spec-Driven Development (SDD), you define your project first — constitution, spec, tasks — and only then start implementing. That moment, right before you write the first AI-assisted line, is when you have the richest possible description of the work and have spent exactly nothing on it.

SpecJudge uses that description to answer the model question at the cheapest possible moment.

It reads your SDD artifacts, and a local model running through Ollama estimates how demanding the project actually is. That demand is crossed against a catalog of models, and you get a ranked podium of what fits best — each with its price.

The part I care about most: right-sizing

SpecJudge does not recommend the cheapest model. It doesn't recommend the most powerful one either. It recommends the one that's right-sized for your project.

The podium ranks by fit. Price is shown for every model, but it only breaks ties between models that fit equally well. A cheaper model never wins over one that does the job better — because recommending something that can't do the job is the most expensive mistake of all.

Every model gets a rating on a fixed scale:

good Capable enough — the sweet spot
overkill Capable, but you're paying for headroom you won't use
fair Falls somewhat short
poor Not capable enough

Local by design

The judge runs on your machine. Your specs — your business logic, your design decisions — never touch a third-party service, and figuring out which model to buy costs you nothing in API calls. The optional browser report (--open) is a self-contained HTML file that loads nothing from the network.

This isn't a nice-to-have bolted on afterwards; it's the first principle of the project.

Quick start

You'll need Python 3.11+ and Ollama with at least one local model.

ollama pull llama3.1:8b        # a judge to evaluate your project
pip install specjudge
specjudge /path/to/your/project

On first run it lists your local models, asks which one to use as the judge, and remembers your choice. Add --open for a visual matrix in your browser, filterable by model family and by open source.

specjudge . --open

It's honest about what it doesn't know

A spending recommendation built on thin data is worse than none. So SpecJudge is explicit about how much you can trust it:

Sufficient — constitution, spec and tasks present: reliable recommendation.
Scarce — artifacts missing or thin: recommendation issued, with a warning.
Insufficient — nothing to evaluate: no recommendation at all.

And if no model in the catalog is capable enough for your project, it says so instead of recommending one anyway. Data states and exit codes are distinguishable, so you can wire it into CI.

Open source, and built to be maintained by the community

Models and prices change every few weeks. That's the whole reason this project needs contributors.

The most valuable contribution requires zero Python — the catalog lives in plain YAML, deliberately separate from the code. Adding a model is one block:

yaml
- id: my-new-model
  name: My New Model
  family: MyFamily
  open_source: false
  provider: SomeAI
  capabilities:
    reasoning: high        # low | medium | high | top
    size: high
    domain_specialization: medium
  price:
    input_per_million: 1.50
    output_per_million: 6.00
    currency: USD
    pricing_date: 2026-07-20   # required — makes freshness verifiable

Disagree with a rating? That's a PR, not a complaint — the catalog is opinion made inspectable.

It's MIT-licensed. Repo here:

JoaquinRuiz / SpecJudge

Local-first CLI that reads your Spec-Driven Development artifacts and recommends the best value-for-money AI model to implement them. Powered by Ollama — your specs never leave your machine.

🧑‍⚖️ SpecJudge

Stop guessing which AI model to use. Ask your specs.

SpecJudge reads your Spec-Driven Development artifacts and tells you which AI model actually fits the job — before you spend a single token implementing it.

$ specjudge .

             Model comparison (SpecJudge) - judge: devstral-small-2
┏━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃    ┃ Model                    ┃ Rating ┃ Price                  ┃ Priced on  ┃
┡━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ 🥇 │ Gemma 4 31B (local)      │ good   │ open-source/free       │ 2026-07-20 │
│ 🥈 │ Gemma 4 31B              │ good   │ 0.35 out / 0.12 in     │ 2026-07-20 │
│ 🥉 │ Qwen 3.6-35B-A3B (local) │ good   │ open-source/free       │ 2026-07-20 │
│    │ GLM-5.1                  │ good   │ 3.04 out / 0.97 in     │ 2026-07-20 │
│    │ Claude Opus 4.8          │overkill│ 25.00 out / 5.00 in    │ 2026-07-20 │
└────┴──────────────────────────┴────────┴────────────────────────┴────────────┘

🥇 Gold: Gemma 4 31B (local)
   Right-sized: capability matches demand exactly in every dimension.

Add --open for a…

View on GitHub

PyPI: https://pypi.org/project/specjudge/

If SpecJudge saves you from overpaying for a model, a ⭐ on GitHub helps it reach more people. And I'd genuinely love feedback on where the judging feels off — that's the part most worth getting right.

DEV Community: Joaquín Ruiz