DEV Community

AI Verification Series' Articles

Back to John Green's Series
I Gave My LLM an Exam. The Exam Author Lost 5 Times.
ramses203 profile

I Gave My LLM an Exam. The Exam Author Lost 5 Times.

#ai #llm #testing #programming
3
Comments 3
5 min read
A Good LLM Exam Is 90% Traps
ramses203 profile

A Good LLM Exam Is 90% Traps

#ai #llm #testing #programming
1
Comments 1
3 min read
Grade Your LLM Pass/Fail and You Will Ship a Disaster
ramses203 profile

Grade Your LLM Pass/Fail and You Will Ship a Disaster

#ai #llm #testing #programming
1
Comments 2
3 min read
I Made an LLM Re-Grade My Exam. It Found Two Bugs in My Grader.
ramses203 profile

I Made an LLM Re-Grade My Exam. It Found Two Bugs in My Grader.

#ai #llm #testing #programming
1
Comments
5 min read
A Reader Caught My Answer Key Drifting Toward the Model
ramses203 profile

A Reader Caught My Answer Key Drifting Toward the Model

#ai #llm #testing #programming
1
Comments
4 min read
The Model That Costs 3x More Won by Exactly One Question
ramses203 profile

The Model That Costs 3x More Won by Exactly One Question

#ai #llm #testing #programming
1
Comments 7
3 min read
It Passed Every Test. That's Why It Can't Ship Yet.
ramses203 profile

It Passed Every Test. That's Why It Can't Ship Yet.

#ai #llm #testing #programming
3
Comments
3 min read
I Scraped 20,000 YouTube Comments. The Videos and the Comments Were Having Two Different Conversations.

Divergence between tutorials and user pain
ramses203 profile

I Scraped 20,000 YouTube Comments. The Videos and the Comments Were Having Two Different Conversations.

#ai #llm #testing #programming
9
Comments 9
3 min read
Steal This Exam. Here's How to Port It to Your Own Pipeline.
ramses203 profile

Steal This Exam. Here's How to Port It to Your Own Pipeline.

#ai #llm #testing #programming
3
Comments
5 min read
The Question That Picked My Model Didn't Survive Five Reruns

Reader feedback triggers a costly reality check
ramses203 profile

The Question That Picked My Model Didn't Survive Five Reruns

#ai #llm #testing #programming
6
Comments 6
5 min read
The AI Exam Author Was Never Wrong. I Still Can't Use Its Exam.
ramses203 profile

The AI Exam Author Was Never Wrong. I Still Can't Use Its Exam.

#ai #llm #testing #programming
3
Comments 2
4 min read
I Stole My Own Exam. It Failed the Tool Behind My Own Numbers.
ramses203 profile

I Stole My Own Exam. It Failed the Tool Behind My Own Numbers.

#ai #llm #testing #programming
2
Comments
4 min read
I Gave a Regex and an LLM the Same Exam. Fatal 3 vs Fatal 0.
ramses203 profile

I Gave a Regex and an LLM the Same Exam. Fatal 3 vs Fatal 0.

#ai #llm #testing #programming
3
Comments
4 min read
The Textbook Says Grade Binary. I Grade in Four. Was I Wrong the Whole Time?
ramses203 profile

The Textbook Says Grade Binary. I Grade in Four. Was I Wrong the Whole Time?

#ai #llm #testing #programming
Comments
4 min read
I Rewrote One Exam Question Fifty Ways. First, the Answer Key Was Wrong.
ramses203 profile

I Rewrote One Exam Question Fifty Ways. First, the Answer Key Was Wrong.

#ai #llm #testing #programming
3
Comments
6 min read