This is a submission for the Kaggle Benchmarking Challenge
What I Benchmarked
I benchmarked p28-zero-diff, a minimal benchmark that formalizes the case where task difficulty is zero,
The capability I wanted to measure is whether a model can produce a valid zero-difficulty proof statement when prompted directly. I chose p28 because it is a known trivial case and a good sanity check for benchmark design.
Models Tested
This benchmark is model-agnostic and designed for any text-generation model.
For this submission I did not run a full model comparison yet - the notebook is a minimal template that proves the task structure itself (zero-difficulty check). The Kaggle Benchmark framework allows it to be run against any LLM lineup later.
Findings
My results:
- The p28-zero-diff benchmark runs successfully on Kaggle and returns a score of 1.0
- I verified that a zero-difficulty proof task is scorable without any solver or external API
- Notebook link proves end-to-end execution
Main insights:
- Not every benchmark needs to be hard - zero-difficulty tasks are useful as baseline sanity checks
- The Kaggle Benchmark framework works even for minimal single-prompt tasks
What surprised me:
- How trivial p28 becomes once formalized
What I'd measure next:
- Run this template across p1-p50 to find all zero-difficulty cases
- Add a second scorer for proof quality (not just generation success)
My Benchmark
https://www.kaggle.com/benchmarks/tasks/enrique6831/perfect-partition-28-zero-diff
Top comments (0)