This is a submission for the Kaggle Benchmarking Challenge
Stop or Ship: Can AI Make Safe Software Decisions?
Modern AI models are getting remarkably good at writing code.
But writing code is only one part of software engineering.
A much less discussed question is:
Can an AI correctly decide when it should change code, when it should stop, and when it should ask a human?
That question motivated my benchmark.
Instead of measuring whether a model can produce a syntactically correct solution, I wanted to measure something closer to what happens in a real software repository: incomplete requirements, ambiguous bug reports, risky changes, missing context, and situations where confidently doing the wrong thing is worse than doing nothing.
What I Benchmarked
I built Stop-or-Ship, a benchmark for AI decision-making in realistic software-maintenance scenarios.
Each task gives a model a small software-engineering situation, such as:
- a bug report with incomplete information
- a proposed code change
- a pull request with a subtle regression
- a feature request whose requirements are ambiguous
- a dependency update with a potential compatibility problem
- a change that looks correct locally but violates an existing constraint
The model must choose what should happen next.
The core actions are:
SHIP — the change is sufficiently safe and should proceed.
CHANGE — the proposed implementation should be modified before proceeding.
ASK — there is not enough information to safely make the decision.
STOP — the change introduces a meaningful risk or violates an important constraint.
This is intentionally different from a normal coding benchmark.
A model can write beautiful code and still make a terrible engineering decision.
What I wanted to measure
I focused on four dimensions:
Decision accuracy
Did the model choose the correct action?
Risk sensitivity
Could it identify changes where a seemingly reasonable implementation introduces a hidden problem?
Abstention quality
Did it know when it should ask for more information instead of pretending to know?
Calibration
When the model was uncertain, did its confidence match reality?
That last dimension is especially important for agentic systems. A wrong answer with low confidence can trigger a human review. A wrong answer delivered with extreme confidence can silently become production code.
Models Tested
I selected a mix of general-purpose frontier models and decision-oriented models so that the benchmark would not simply become a comparison between models from the same family.
The initial lineup includes:
- Jev — included because it is specifically designed around decision-making rather than long-form text generation.
- OpenAI GPT-family model — a strong general-purpose reasoning baseline.
- Anthropic Claude-family model — included to test another strong reasoning-oriented approach.
- Google Gemini-family model — included as a different model family with strong coding and multimodal capabilities.
- [Additional model]
- [Additional model]
The exact versions and inference configuration are recorded in the Kaggle benchmark so that the comparison remains reproducible.
I deliberately did not want to answer "which AI writes the best code?"
I wanted to answer a narrower question:
Which models are better at making the decision that should happen before code gets merged or deployed?
Findings
The most interesting part of this benchmark is not a single leaderboard number.
It is the difference between being capable of solving a problem and being capable of deciding whether a problem is safe to act on.
1. Getting the answer right is not enough
[INSERT ACTUAL RESULT]
The benchmark showed that raw decision accuracy does not tell the entire story.
Two models can achieve similar accuracy while behaving very differently:
One may confidently choose an action on almost every task.
Another may recognize uncertainty and ask for additional information.
For software agents, those behaviours have very different consequences.
2. Ambiguity is its own failure mode
[INSERT ACTUAL RESULT]
The "ASK" category produced some of the most interesting examples.
Traditional benchmarks often reward a model for producing an answer.
Real software engineering sometimes rewards the opposite behaviour.
When requirements are contradictory, tests are missing, or repository context is incomplete, not acting can be the correct action.
This benchmark makes that behaviour measurable instead of treating it as a failure to answer.
3. Confidence matters as much as correctness
[INSERT ACTUAL CALIBRATION RESULT]
I also tracked whether confidence was aligned with correctness.
That creates a more useful distinction:
- Correct + confident → ideal
- Correct + uncertain → potentially acceptable
- Wrong + uncertain → human review can still catch it
- Wrong + confident → dangerous automation failure
This is one reason I think benchmark design should move beyond a single accuracy number.
4. Decision models vs. language models is an interesting boundary
The benchmark also let me explore a broader question:
Does a model optimized for making decisions behave differently from a model optimized primarily for generating language?
Jev was particularly interesting to include because its design targets calibrated decisions rather than conventional chat completion.
[INSERT ACTUAL COMPARISON FROM KAGGLE RESULTS]
I don't think this benchmark proves that one model architecture is universally better.
Instead, it suggests that model choice should depend on the role an AI is playing.
An AI that writes documentation has different requirements from an AI deciding whether an automated deployment should continue.
What surprised me
The biggest surprise was how often the difficult cases were not about coding at all.
They were about knowing what the model did not know.
A task could contain a perfectly plausible implementation while still being impossible to approve safely because one critical piece of context was missing.
That made "ASK" one of the most important actions in the benchmark.
It also changed how I think about AI evaluation.
A model should not only be rewarded for:
"Can you solve this?"
It should also be evaluated on:
"Should you act on this?"
What I Would Measure Next
The next version of Stop-or-Ship could go beyond static prompts.
I would like to test:
Multi-turn repository context
Give the model the ability to inspect files, tests, commit history, and previous discussions before making its decision.
Tool-use decisions
Measure whether the model requests the right tool before making a high-risk decision.
Cost-aware decisions
Test whether a model can balance correctness against API calls, latency, and engineering cost.
Long-horizon agent behaviour
Instead of one decision, evaluate an entire sequence of actions and measure whether the model eventually recovers from a bad assumption.
Adversarial tasks
Introduce realistic distractions such as misleading comments, plausible-but-wrong tests, stale documentation, and incomplete bug reports.
That would turn Stop-or-Ship from a simple classification benchmark into a more realistic evaluation of autonomous software agents.
My Benchmark
Kaggle Benchmark: [INSERT YOUR KAGGLE BENCHMARK LINK]
The benchmark contains the task definitions, evaluation methodology, model runs, and leaderboard.
All model comparisons should be interpreted within the exact dataset, prompts, and configurations used in the benchmark rather than as a universal ranking of AI systems.
Why I Built This
AI coding benchmarks usually ask whether a model can produce the right code.
I think the next generation of benchmarks should also ask whether the model knows when it should be allowed to act.
That's the difference between an AI that can generate code and an AI that can safely participate in software engineering.
Stop or Ship is my attempt to measure that gap.
Top comments (0)