DEV Community

Kaggle Benchmarking Challenge

This is the official tag for submissions and announcements related to the Kaggle Benchmarking Challenge.

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Phish or Legit: Do LLMs Know When NOT to Cry Wolf?

Kaggle Benchmarking Challenge Submission

Phish or Legit: Do LLMs Know When NOT to Cry Wolf?

Comments 1
3 min read
MyanmarChemCalc-Bench: Do LLMs Do Chemistry Better in English Than in Burmese?

Kaggle Benchmarking Challenge Submission

MyanmarChemCalc-Bench: Do LLMs Do Chemistry Better in English Than in Burmese?

Comments
3 min read
Reward Evidence: an AI can track the money and still misread the opportunity

Kaggle Benchmarking Challenge Submission

Reward Evidence: an AI can track the money and still misread the opportunity

Comments
6 min read
'No peanuts' became include_ingredients: ["peanuts"]: a benchmark for tool calls a validator cannot catch

Kaggle Benchmarking Challenge Submission

'No peanuts' became include_ingredients: ["peanuts"]: a benchmark for tool calls a validator cannot catch

Comments 1
11 min read
Same Score, Different Failure: Fechamento BR

Kaggle Benchmarking Challenge Submission

Same Score, Different Failure: Fechamento BR

Comments
8 min read
Day 3: The benchmark caught me too.

Day 3: The benchmark caught me too.

Comments
6 min read
I asked AI to undo its work. One model broke 7 of 12 already-correct records.

Kaggle Benchmarking Challenge Submission

I asked AI to undo its work. One model broke 7 of 12 already-correct records.

Comments
7 min read
FrameFlip48: four models, one frame switch, two kinds of failure

Kaggle Benchmarking Challenge Submission

FrameFlip48: four models, one frame switch, two kinds of failure

Comments
5 min read
I graded 8 AI models the way I grade new contact centre agents

Kaggle Benchmarking Challenge Submission

I graded 8 AI models the way I grade new contact centre agents

Comments
6 min read
Day 2: The model I want is the one that's boring everywhere

Day 2: The model I want is the one that's boring everywhere

Comments
10 min read
I benchmarked what frontier models actually know about 2026 — most of it, they don't

Kaggle Benchmarking Challenge Submission

I benchmarked what frontier models actually know about 2026 — most of it, they don't

Comments
3 min read
When a Fact Is Retracted, Can an AI Keep What Still Holds?

Kaggle Benchmarking Challenge Submission

When a Fact Is Retracted, Can an AI Keep What Still Holds?

Comments
18 min read
Evidence Boundary: when an AI benchmark measures its own grading rules

Kaggle Benchmarking Challenge Submission

Evidence Boundary: when an AI benchmark measures its own grading rules

Comments
8 min read
Do LLMs Have a Spine? I Benchmarked Sycophancy Across 7 Frontier Models

Kaggle Benchmarking Challenge Submission

Do LLMs Have a Spine? I Benchmarked Sycophancy Across 7 Frontier Models

1
Comments 2
6 min read
To Retry or Not to Retry? That Is the Question.

Kaggle Benchmarking Challenge Submission

To Retry or Not to Retry? That Is the Question.

95
Picked as gem Comments 97
9 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.