DEV Community

Kaggle Benchmarking Challenge

This is the official tag for submissions and announcements related to the Kaggle Benchmarking Challenge.

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
I set filmmaker traps for AI "directors." The models fell for one. My rubric fell for three.

I set filmmaker traps for AI "directors." The models fell for one. My rubric fell for three.

Comments 1
4 min read
Who Reviews the Reviewers? Benchmarking AI Agents on PR Audits

Who Reviews the Reviewers? Benchmarking AI Agents on PR Audits

Comments
2 min read
The hard part of agent memory is knowing when to ask

Kaggle Benchmarking Challenge Submission

The hard part of agent memory is knowing when to ask

Comments 1
10 min read
Six of seven models contradicted themselves on the same deprecation question

Kaggle Benchmarking Challenge Submission

Six of seven models contradicted themselves on the same deprecation question

Comments
7 min read
I Built a World With Rules. Then I Changed Them.

Kaggle Benchmarking Challenge Submission

I Built a World With Rules. Then I Changed Them.

Comments
6 min read
I Tested Whether Frontier AI Can Do a Texas Real Estate Agent's Desk Work — Here's What 3 Models Taught Me (and What the 4th Taught Me About Benchmarks)

Kaggle Benchmarking Challenge Submission

I Tested Whether Frontier AI Can Do a Texas Real Estate Agent's Desk Work — Here's What 3 Models Taught Me (and What the 4th Taught Me About Benchmarks)

Comments
6 min read
Soul QA Bot: Smarter AI That Works Offline

Kaggle Benchmarking Challenge Submission

Soul QA Bot: Smarter AI That Works Offline

Comments
2 min read
Can an AI Count a Music Loop? 24 Audio Arithmetic Checks

Kaggle Benchmarking Challenge Submission

Can an AI Count a Music Loop? 24 Audio Arithmetic Checks

Comments
6 min read
The header says 10%. The table says 8%. Most frontier models quietly went with the table.

Kaggle Benchmarking Challenge Submission

The header says 10%. The table says 8%. Most frontier models quietly went with the table.

Comments 1
9 min read
My Benchmark Caught Me Lying Before It Caught Any Model: Do Coding Agents Report "Verified" Honestly?

Kaggle Benchmarking Challenge Submission

My Benchmark Caught Me Lying Before It Caught Any Model: Do Coding Agents Report "Verified" Honestly?

Comments
6 min read
"Can I Eat This?" — Benchmarking Whether AI Models Know Where Their Foraging Knowledge Ends

Kaggle Benchmarking Challenge Submission

"Can I Eat This?" — Benchmarking Whether AI Models Know Where Their Foraging Knowledge Ends

Comments
7 min read
My stock scanner's AI was a momentum echo. Are today's models?

Kaggle Benchmarking Challenge Submission

My stock scanner's AI was a momentum echo. Are today's models?

Comments 1
28 min read
Mega AI Battle: Benchmarking 6 Top LLMs with Advanced Bangla Logic Riddles

Kaggle Benchmarking Challenge Submission

Mega AI Battle: Benchmarking 6 Top LLMs with Advanced Bangla Logic Riddles

Comments
2 min read
RTL Receipt Test: Fluent Arabic is not the same as faithful evidence

Kaggle Benchmarking Challenge Submission

RTL Receipt Test: Fluent Arabic is not the same as faithful evidence

Comments
7 min read
ButterflyBench: I Changed One Instruction. What Else Did the AI Change?

Kaggle Benchmarking Challenge Submission

ButterflyBench: I Changed One Instruction. What Else Did the AI Change?

Comments
8 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.