Evaluating LLM outputs is one of those problems that sounds simple until you actually try to do it at scale.
Most approaches fall into three buckets:
Run the code — works for coding tasks but requires a sandbox, is slow, and breaks on edge cases constantly.
GPT-4 as a judge — surprisingly effective but costs money, adds latency, and feels wrong to use one LLM to evaluate another.
Manual review — fine for 50 samples. Completely falls apart at 1,500+.
I wanted something different. Fast, local, no API calls, no execution environment, no cost per eval. So I built LLM Judge.
How it works
The core insight is simple: semantic similarity between a model output and a reference answer is a surprisingly strong signal for correctness — especially for coding questions where there's usually one right approach.
Here's the pipeline:
Model output + Reference answer
↓
Sentence Transformers (all-MiniLM-L6-v2)
↓
Cosine similarity score
↓
Logistic regression classifier
↓
Correct / Partially correct / Incorrect + calibrated confidence score
I trained the logistic regression on a merged Hugging Face coding Q&A dataset, validated on 1,500+ labeled submissions.
Results
~75% accuracy on held-out coding Q&A split
~58% agreement with human judges on 1,500+ labeled submissions
Runs entirely locally — no API calls, no sandbox
Fast enough for batch evaluation at scale
The 58% human agreement number is the interesting one. Human judges disagree with each other more than you'd expect on coding questions — so 58% agreement with humans is actually competitive with inter-human agreement on ambiguous cases.
What I built
Three interfaces for different use cases:
CLI batch scoring — point it at a CSV of model outputs and reference answers, get scores back:
bash
python evaluate_csv.py --input outputs.csv --output scores.csv
Training your own judge:
bash
python train_judge.py --dataset your_dataset.csv
Streamlit UI — visual interface with calibrated scores and 3-tier feedback for each eval:
bash
streamlit run app.py
Stack
Python
Sentence Transformers
Scikit-Learn
Streamlit
Pandas
What's next
The open source version handles local evaluation. I'm building a hosted API version next — upload your dataset, get scores back via API, no setup required. If that's something you'd use, let me know in the comments.
Try it
GitHub: https://github.com/Zoh007/RAG_PRAC
Would love feedback from anyone doing LLM evaluation at scale — especially curious:
Is 75% accuracy good enough for production eval pipelines?
What edge cases do you run into that simple similarity scoring misses?
Would a hosted API be worth paying for or would you always self-host?
Top comments (0)