DEV Community: ty215

A Small Prototype for Cost-Aware Bug Investigation

ty215 — Fri, 10 Jul 2026 00:10:24 +0000

The question that started it

A weak point in AI-assisted coding is not always broken syntax or a failing test suite. The original concern behind this project was about tests too: if the AI-assisted workflow helps with testing, is that set of tests enough?

If the testing pattern is biased, a bug may sit where the tests do not look. The early idea was adversarial and game-theoretic, even equilibrium-like: treat bugs or failure conditions as if they were trying to stay hidden, then ask what testing or investigation policy would still hold up.

That is not what v0.1.0 proves. The current repository is not a formal game-theoretic debugger, and it does not claim real-world debugging accuracy.

That is the frame of bug-cause-inference-game, a small Python prototype for cost-aware bug-cause investigation. The version discussed here is tag v0.1.0, which points to commit 9e30c93f246602d840c875e975c362e6ab1e7747.

This version is a preparation step. P1a asks how a cost-aware policy chooses the next investigation action on synthetic cases. P1b compares metadata-derived evidence with execution-grounded evidence in a small injected-bug scaffold. P1c looks beyond averages to worst-case and difficult buckets.

The conclusion is narrow: the prototype shows the design value and limits of cost-aware investigation and worst-case reporting, not a finished debugger.

The boundary matters

The repository is careful about what it does not claim. It is not a production fault-localization engine, an automated repair tool, an LLM debugging benchmark, a fuzzing framework, or a formal game-theoretic debugger.

That boundary is part of the design. The project is closest to probabilistic debugging, Bayesian fault diagnosis, and active diagnosis: rank likely causes from observations, then choose the next observation under cost.

For AI-assisted coding workflows, that modest scope is the point. If generated code can look plausible while missing the intended behavior, the next layer of tooling should help decide what evidence to collect, not just produce another confident explanation.

P1a: make the investigation action explicit

P1a is the smallest slice. It works on synthetic observed-bug cases, updates posterior probabilities over coarse cause categories, compares investigation policies, and emits a DecisionReport whose main output is recommended_next_action.

The synthetic dataset contains 50 cases, built from 5 cause categories and a fixed seed 20260627.

The 5 cause categories are:

boundary_condition
missing_null_handling
configuration_environment
race_order_dependence
specification_mismatch

Each case has 8 possible investigation actions:

inspect_error_log
run_boundary_tests
compare_environment
inspect_recent_diff
run_reproduction_matrix
add_instrumentation
check_spec_acceptance
run_concurrency_stress

That action list is the first useful abstraction. Instead of asking an agent to "debug this," the prototype asks it to choose among named actions with explicit costs.

The implemented P1a policies include random, fixed_checklist, posterior_greedy, cheapest_first, information_gain, information_gain_per_cost, and static_posterior. The main policy is information_gain_per_cost.

The stopping settings are also explicit: top probability threshold 0.75, margin threshold 0.15, budget limit 10, max steps 5, minimum expected information gain per cost 0.03, and failure cost 12.

That makes the prototype less magical. It cannot hide behind "the model decided." The design keeps the current belief state, the available actions, the action costs, and the next recommended action explicit.

What P1a showed

The current P1a dataset is not hard. Before any investigation action, the generated cases already have initial top-1 accuracy of 70% and initial top-2 accuracy of 100%, with 15 initially-wrong top-1 cases out of 50.

Even with that caveat, the policy comparison is useful. On the synthetic setup, information_gain_per_cost has mean cost_to_true_cause_top1 of 1.12, while fixed_checklist has 1.56. The recorded fixed-checklist cost reduction is about 28%, and the primary success rate within budget is 94%.

The same result also exposes a warning sign. The primary wrong-stop rate is about 13%, where wrong-stop means stopping on a high-confidence but incorrect cause hypothesis.

The initially-wrong subset makes the tradeoff sharper. On those cases, information_gain_per_cost has lower mean cost than fixed_checklist (3.73 versus 5.2), but fixed_checklist has the higher success rate (0.87 versus 0.8).

So the safe reading is narrow: cost-aware active investigation reduced average investigation cost in this synthetic setup, but it did not prove real-world debugging accuracy.

That is already a useful habit for AI-assisted coding tools. A good report should show where the policy saves cost, where it stops too early, and where a boring checklist still wins.

P1b: make the evidence less synthetic

P1b adds a small injected-bug checkout/pricing benchmark scaffold. It evaluates budget-aware policies on 20 buggy variants and 5 clean variants.

The P1b evaluator looks at failure discovery, function-level location ranking, coarse cause inference, and fix-intent prediction. It still does not generate patches.

P1b uses two observation modes. metadata_synth is the frozen Phase A/B baseline that synthesizes evidence from variant metadata. execution_grounded builds observations from checkout test results, exceptions, traced checkout functions, coverage-spectrum suspicion, and Phase C real-diff artifacts.

The P1b action set is larger than P1a's and closer to a debugging workflow. It includes test actions such as run_smoke_tests, run_boundary_tests, run_null_missing_tests, run_config_matrix_tests, and run_state_sequence_tests, plus inspection actions such as inspect_traceback, inspect_coverage_spectrum, inspect_recent_diff, and inspect_spec_clause.

The main P1b policy is expected_utility_per_cost.

This is still a small scaffold, but it creates a useful pressure test: does a policy that looks good with metadata-derived evidence still look good when the observations are tied to execution?

The metadata optimism gap

If this article had to be reduced to one lesson, it would be the P1b comparison. The metadata_synth baseline made the primary policy look better than execution_grounded evidence on several metrics.

For expected_utility_per_cost, the primary-policy comparison reports:

Metric	`metadata_synth`	`execution_grounded`
bug discovery within budget	0.55	0.40
cause top-1 accuracy	0.80	0.55
fix-intent top-1 accuracy	0.75	0.40
mean investigation cost	2.80	4.64

metadata_synth still has a role as a baseline. The point is that its optimism gap is visible.

For AI-assisted coding, this is the more general warning. Metadata-derived evidence can make a policy look better than it looks with execution-grounded evidence. Execution-grounded evidence is still small here, but the gap is visible.

P1c: stop hiding behind averages

In v0.1.0, P1c is an analysis-only report over the existing 25-variant P1b scaffold.

The first P1c goal is to ask whether existing investigation policies remain useful when the evidence is deliberately difficult, ambiguous, expensive, or misleading inside a small reproducible benchmark.

The planning notes name several candidate adversarial models: worst-case variant selection, evidence-ambiguity buckets, observation-cost stress, and observation dropout or delay.

The implemented label buckets are boundary_precision, missing_optional_input, config_normalization, state_sequence, spec_semantics, and clean_false_positive.

The important design choice is that P1c does not collapse the result into one score. Discovery, first-failure cost, localization, cause, fix-intent, false-positive, and mean-cost metrics can point to different weaknesses.

That choice fits the original concern better than an average-only report. If a bug behaves like a difficult opponent, the average case is often the least interesting case.

What the worst-case view exposed

For the current primary P1b/P1c policy, expected_utility_per_cost, the aggregate reading is a useful baseline: discovery 0.40, cause top-1 0.55, fix-intent top-1 0.40, and mean investigation cost 4.64.

Those averages need the bucket view next to them.

Bucket	What P1c exposed
`state_sequence`	The primary brittle bucket: discovery, location top-3, cause top-1, and fix-intent top-1 are all `0.0`; first-failure cost is `14`.
`config_normalization`	The next broad weak bucket: those four metrics are all `0.25`; first-failure cost is `10.75`.
`missing_optional_input`	Successful but expensive: the four success metrics are `1.0` and first-failure cost is `1`, but mean investigation cost is `9.75`.

The current clean variants do not trigger false positives in the selected-bucket report or current stress profiles, but that does not show false-positive risk is solved outside this scaffold.

This is why the P1c layer matters. It turns "the average looks acceptable" into "which class of case breaks the policy, and how?"

Why call it a game at all?

The word "game" should be handled carefully here. The current repository does not claim a formal player model, payoff model, minimax-optimal policy, or game-theoretic guarantee.

The game-like idea comes from the original motivation: if bugs may hide where tests do not look, treat hard bugs or failure conditions as if they are selected by an opponent that wants to stay hidden, then ask which investigation policy still performs under that selection pressure.

In v0.1.0, that idea is still bounded. P1c selects worst cases and stress conditions inside a small injected benchmark; it does not generate new bugs, repair code, or claim real-world debugging accuracy.

That boundary keeps the current result readable. Formal game-theoretic claims stay outside this version until the project has an explicit player model, action space, payoff, and evaluation result.

Takeaways

For an AI-assisted coding workflow, the design lesson is deliberately small:

Make investigation actions explicit.
Attach costs to those actions.
Compare policies under the same stopping rules and budget limits.
Separate metadata-like evidence from execution-grounded evidence.
Report worst-case buckets beside averages.
Keep non-claims visible, especially when the prototype is small.

That last point is easy to skip. It is also the part that makes the experiment reusable. A small benchmark can still teach something if it is honest about what it cannot show.

Source version: guriguri215-lang/bug-cause-inference-game at v0.1.0.

Disclosure: this article draft was prepared with AI assistance, and the project claims and numbers were checked against the linked repository version and DEV's AI-assisted article guidance.

The Less I Type, the Better the Output: A Context Framework I Had Claude Design for AI Coding

ty215 — Sun, 05 Jul 2026 15:18:30 +0000

Fable 5, a Claude model, became available to me for a limited window. I could have spent that window shipping features. Instead I had it build something that would outlast the model itself: a context framework for running AI coding across several models.

The reason is a thing I keep circling back to when I work with coding agents. The less a human does by hand, the better the quality and the speed. Not because people are bad at the work, but because every manual step is a place to forget a constraint or approve something on autopilot.

So this is a design write-up, not a benchmark post. I have not run this on a long project for long enough to put numbers on it. What I can tell you is that using it felt more efficient, and I can show you the exact failure modes it was built to kill. If you want a graph, this isn't that article. If you keep watching your agent walk off in the wrong direction and you keep rewriting the same prompt, keep reading.

The pain I kept hitting

Before any of this, my problem was simple and expensive. The agent would drift. It would go off on something that wasn't the point, burn tokens on it, and I would spend real time pulling it back on course.

A lot of that traced back to me. When I write a task by hand, I leave out the parts I think are obvious. The agent doesn't share my "obviously." It fills the gap with its own assumption, does something I didn't ask for, and now there's rework. Sometimes the thing runs away entirely.

The workaround I found almost by accident: don't write the prompt myself. Have a model write the prompt, then read it and check it before it runs. That was steadier. This framework is that habit turned into a system.

Watching the same thing happen over and over, I could name four failure modes I wanted gone:

the agent going off the rails
too many pull requests and re-reviews that never end
my own bias pulling the work along, so course-correction comes late
approvals turning into rubber stamps once things get automated

One thing up front: the framework's own docs treat these four as design targets, and they label their effect numbers as hypotheses, not measurements. I'm keeping that line here.

The core idea: the human doesn't write long prompts

The center of the whole thing is one sentence. The human does not write long prompts. A model writes the prompt for the next step into a file, and the human copies and pastes it.

Day to day, the human does three things:

Open NEXT_ACTION.md to see what's next.
Copy NEXT_PROMPT.md and paste it into the model it names.
Read only the "needs human confirmation" part, then approve or send it back.

Every model, when it finishes, is required to update NEXT_ACTION.md and NEXT_PROMPT.md. So the loop keeps going on its own. I'm not composing instructions each turn.

The files sit next to the repo, not inside it:

<project root>/
  AGENTS.md              # short pointer for whichever CLI starts here
  _ai_management/        # the framework files (kept out of git)
  repo/                  # the actual code you git init and push

That split matters: the code you publish stays clean, and the coordination files stay out of your commit history.

The four failure modes, and the design response

1. Going off the rails

This was my worst one. The agent drifts, and course-correction eats the afternoon.

The design response is boring on purpose. The spec is the single source of truth, and a model is not allowed to implement anything that depends on an open, undecided question. That's a hard stop, not a suggestion. Acceptance criteria can't be written as "it works correctly"; they have to name the input, the output, and the command that checks them. And each task is routed by how much damage a wrong move would cost, not by which model I happen to like. There are defined routes from light fixes up to research work, with a rule to pick the heavier route when you're unsure.

2. Too many PRs, reviews that never end

Push "be critical" too hard and review turns into a game. The count of comments starts to look like the output, so a reviewer manufactures harmless nitpicks and sends the work back again and again.

The response is a review discipline with a few hard rules. Every comment has to carry a concrete failure scenario: which input, in which state, produces which wrong result. If you can't write that, it goes in an "observations" box and does not block the decision. Only high and medium severity can send work back; low-severity-only means approve with notes. A re-review after a bounce is scoped to "did the previous points get fixed," plus new high-severity only, so a bounce can't be stretched into three. Two bounces, then the route changes instead of a third. A request to send work back also has to state repair cost against the damage of leaving it in; if the damage is smaller than the fix, it doesn't bounce. And the model that wrote the code doesn't get to be its final reviewer. The last review crosses model families.

3. My own bias pulling the work along

My input is a mix of two things: decisions I've actually made, and opinions I'm still thinking through. If the model treats a half-formed opinion as a decision, my under-baked idea gets built, and quality drops.

The response is a prefix convention. Anything that starts with 決定: (decision:) is a decision. It's executed without debate and logged. Anything else is an opinion, and the model has to answer with a technical judgment, agree, agree with conditions, or disagree, and give its grounds before it acts. Flattering agreement is banned; it can't open with "you're absolutely right." It's also not required to disagree. If my idea is the best one, it says so in a line and takes it. When it does disagree and I override with 決定:, it runs the thing but logs "human decision over model objection" with the reason, so later we can check who was right. Purpose and priorities stay my call; facts and implementation correctness get checked against evidence, whoever said them.

4. Rubber-stamp approvals

When a person approves machine output, a bias creeps in: "it was generated, it's probably fine." Approval goes hollow. A bare "please confirm" makes it worse.

The response is an approval gate format. No free-form "looks good?" A request has to be filled into a fixed block:

## Approval gate
- Approving: <one line>
- Risk: high / medium / low
- Human checks (max 3): <which file / where to look / what "OK" looks like>
- Unapproval conditions (don't approve if any one is true):
  - <listed in verifiable form; at least one checkable by a command>
- Unverified: <what the model couldn't verify; writing "none" needs grounds>
- Recommendation: approve / approve after scrutiny / don't approve

(The framework's labels are in Japanese; this is my English rendering of the same shape.) The unapproval conditions are the load-bearing part: the specific things that, if any one is true, mean do not approve, with at least one written so a command can verify it. If a model can't write those conditions, the task isn't ready, and it gets sent back on the spot. The final gate is written by the last model that reviewed the work, not the one that built it, because you go easy on your own output. And every so often, a low-risk "light approval" gets pulled at random for a full check, to see whether the light path has gone hollow.

If you read the three above closely, they're one problem wearing three faces. The docs put it this way: evaluation gets bent by relationship instead of evidence. The single rule under all of it is the same shape each time. Agreement needs grounds, objection needs a failure scenario, approval needs unapproval conditions.

What surprised me, and where this stops

The honest part. What surprised me wasn't a clever trick. I didn't expect a model to build something this large. It came back as a full folder of specs, logs, and templates.

I also left one thing out on purpose. There's a benchmark stage in the design, and I chose not to run it. The project keeps qualitative notes only, no metrics. That's a real gap, and it's why I keep writing "felt" instead of "measured." I haven't run it long enough to claim a number.

So read the four responses above as design intent that lined up with my experience, not as proven results. The failure modes are real, and I've hit every one of them. Whether this framework cuts them by some percentage, I can't tell you yet.

Takeaway

If I compress the whole thing to one line, it's what I'd say first if a friend asked: cut the amount the human does by hand, and both quality and speed go up. Fewer manual steps means fewer places to slip.

The other half is about what you keep for the human. Don't pull the human out of approval. Make approval cheap and honest instead: outputs the human can check quickly, a small amount to check, and an explanation attached. In this framework the human's approval concentrates at just three points: locking the spec, the final sign-off on a diff, and anything irreversible like a push.

Disclosure: the framework described here was designed by Claude (Fable 5). This article was written with AI assistance, then edited and fact-checked by me; the design choices and the experience are mine. Also tagged #abotwrotethis.

Before the Algorithm: Building the Input Layer for My Poker Analysis Tool

ty215 — Sun, 28 Jun 2026 13:41:42 +0000

In my first post about this project I described repeated-poker-analysis: a small, abstract toolkit for studying repeated poker spots as a commitment-analysis problem. It is a research and learning project, not a poker solver and not real-money advice.

That post ended on the analysis side, with a hand-built toy game where committing to a fixed strategy beats the one-shot baseline. I planned to spend the next stretch on the algorithm. In practice I spent most of it on input, validation, and saving — the layer the algorithm sits on. This post is about that part, and about the abstract model growing a bit at the same time.

This describes the project on the feat/single-hand-form-edit-cli branch at the time of writing, so details may move after it is merged.

What changed since the last post

Last time, scenarios were small fixed examples and helper functions inside the code. To analyze anything new, I edited Python.

Two things changed. First, a scenario is now a JSON file: the tool reads it, checks it, builds the abstract game, runs the existing analysis pipeline, and can export the result. Second, the abstract model itself widened — I added more scenario modes, so the JSON can describe slightly richer hand and action structure than the single toy spot from the first post.

So this wasn't a stretch with no model work. It was widening the abstract model and the input formats together, while keeping both small enough to check.

Why the input layer came first

Analysis output is only as trustworthy as its input. If a scenario is silently malformed — a probability that doesn't sum to one, a missing matrix cell — then any EV or deadline the tool reports is meaningless, and I might not notice.

So before making the analysis bigger, I wanted the input to fail loudly when it's wrong instead of quietly producing a plausible-looking number. That meant firming up the input and validation layer first, even though it isn't the part I was looking forward to.

What a JSON scenario is

A scenario is a JSON description of one small abstract river spot: the pot and bet sizes, the rake rule, how showdowns resolve, a baseline strategy, and the repeated-game parameters. JSON is the source of truth — every other representation in the tool is derived from it and checked back against it.

Reading one through the current loader looks like this:

JSON scenario
 -> detect mode
 -> parse + build the abstract game
 -> (analysis pipeline, if requested)

The JSON is parsed and built into the actual game object before anything trusts it, so a broken scenario is rejected at the door.

The five scenario modes

I added the modes in order, each one allowing a little more abstract structure than the last:

single-hand — one abstract hand bucket. Check/bet for one player, call/fold for the other. This is the mode the first post's toy game used.
Hero-range-only — several Hero buckets with weights; the opponent still acts on a single information set.
showdown-matrix — Hero and Villain buckets, with each matchup's result given directly as hero / villain / chop.
equity-matrix — the same grid, but each cell is a Hero pot share between 0 and 1 instead of a discrete result. These numbers are supplied in the JSON; the tool does not evaluate real cards or compute equity.
river betting-tree — one street with a few more actions: an IP stab after an OOP check, and a single IP raise line against an OOP bet. It's still a fixed one-street tree, with no re-raises, nested sizings, or street transitions.

The abstraction level is still high. There's no real card-range parsing, no real card evaluation, no external solver import, no solver-grade GTO calculation, no multiple streets, and no arbitrary nested betting trees.

I split this into five modes on purpose. The goal wasn't to make the model large quickly — it was to keep each step small enough that when something broke, I could tell which stage and which input shape caused it. So the model and input formats grew incrementally, but there's still groundwork to do before any of this becomes realistic analysis.

Why I built form models before the GUI

I want a GUI eventually, so editing a scenario doesn't mean hand-writing JSON. But before building any screens, I built a form model for each of the five modes.

A form model is a flat, edit-friendly representation of a scenario — the shape a GUI form would bind to. It doesn't replace the JSON; the JSON stays the source of truth. The form model just gives a future screen something convenient to read from and write to.

Two design choices mattered here:

The validator returns field-level messages instead of throwing. While editing, a form is often temporarily broken — a probability that doesn't sum to one yet, a half-typed value — and a GUI needs to show that next to the field rather than crash.
Invalid values are not quietly rounded to valid ones. If a field is wrong, it's surfaced as an error before saving, not silently "fixed." A scenario that passes validation is one that round-trips back through the parser and builder.

Building the form models for all five modes first let me check the data structures behind the screens before committing to any screen.

What the inspect / roundtrip / edit CLIs check

To confirm the form layer works without a GUI, there are three small developer-utility CLIs. They aren't user features; they're how I check the plumbing.

inspect_scenario_form.py reads a scenario through its form model, detects the mode, runs validation, and confirms it round-trips back through the parser and builder. It's a health check for the input layer.
roundtrip_scenario_form.py reads a scenario, runs it through the form model, and writes it back out — only if validation and the round-trip succeed. This is close to what a GUI "save" will eventually do.
edit_scenario_form.py reads a scenario, edits a few fields, validates, and saves. It's the minimal version of "change a field and save it," and for now it's single-hand only.

Inspecting the first post's toy game looks like this:

$ python scripts/inspect_scenario_form.py examples/scenarios/nuts_chop_steal_bet98.json
Scenario form inspection
scenario_id: nuts_chop_steal_bet98
mode: single-hand
form: SingleHandScenarioForm
validation: ok
round_trip_parse: ok
round_trip_build: ok

It's a boring output, which is what I wanted: it confirms the read → validate → round-trip path holds before I build a screen on top of it.

Working with AI as the codebase grew

I'm building this with AI assistance, and how I work with it changed as the code grew.

I went into this stretch expecting to get a lot done over a weekend. To keep quality up, though, I started having the assistant explain code back to me, adding more tests, and running a review pass before moving on. That slowed feature work down, and the stretch ended at the form layer rather than further along.

I don't think that was bad stalling. It's the scaffold I'll need to trust the analysis and the GUI later. The cost is that explaining, testing, and reviewing as I go uses far more tokens and takes longer than just generating code. It's still clearly faster than doing all of this alone from scratch — but on a project this size, the trade-off is something I now weigh.

Tests and verification

There are now over 1,000 tests. I want to be careful about what that means: it does not mean I'm verifying a large poker solver. It means I've accumulated checks on a small abstract model, its input/output paths, and boundary conditions — for example, that all five modes parse, that a CLI writes JSON and nothing else, that it won't overwrite a file unless told to, and that it writes nothing when validation fails.

That kind of check is most of the test count. It buys confidence in the plumbing, not in any real-world poker claim.

The harder question is still open: as the project grows, the amount I'd need to verify grows faster than I can read. I haven't solved that. I expect I'll have to validate from the outside — design the checks to be as exhaustive as I can and run many configurations — rather than reading every line.

Current limitations

At this stage the tool is still a small abstract toolkit. In particular, it does not have:

a real card-range parser or real hand evaluation,
any external solver import or solver-grade GTO output,
an arbitrary game-tree editor, multiple streets, or re-raise / multi-sizing / nested betting trees,
a GUI yet, or edit CLIs for the range / matrix / betting-tree modes (edit is single-hand only),
anything resembling real-money strategy advice, or any model of human learning speed or psychology.

The EV numbers it produces are outputs of a deliberately small model, not poker advice.

What I want to do next

The near-term goal is to make the tool easier to run and watch — to get from "abstract model that passes its checks" to "I can load a scenario, change it, and see the result" without editing Python.

Concretely, that means extending the edit flow beyond single-hand, and eventually a minimal GUI on top of the form models that are already in place. Alongside that, I still want to work out the verification approach for when the model grows, since that's the part I don't have a good answer for yet.

This stretch was mostly groundwork. It's not the part I was looking forward to, but the algorithm work I do want to do will be easier to trust because of it.

Why I Built a Tiny Repeated-Game Poker Analysis Tool

ty215 — Sat, 27 Jun 2026 00:23:00 +0000

Most poker solvers answer one question very well: given a single hand and a single decision tree, what is the equilibrium strategy? (Yes, there is subgame solving, node locking, and plenty more — but the default frame is still one hand, one equilibrium.)

I kept getting stuck on a different one. What if the same kind of spot shows up over and over, and a player can commit to a fixed strategy across those repetitions? In a few toy games I had a hunch, worked out by hand, that committing to a fixed strategy could change its value relative to the one-shot picture. I wanted a tool that could make that commitment value precise — to actually analyze it rather than just believe it. (Whether any of this rises to a repeated-game equilibrium is a much stronger claim, and one I am deliberately not making here.)

I'm still learning software engineering, so until recently I couldn't implement this — I was stuck reasoning about toy games on paper. AI tooling made the analysis feasible, so I finally started building it: repeated-poker-analysis.

It's a small research project: write one narrow model down, run small examples, and record what the model does and doesn't justify.

What `repeated-poker-analysis` is

It is an experimental Python toolkit for small abstract poker games. The current MVP covers:

fixed Hero commitment candidates,
exact Villain best-response diagnostics in small finite trees,
candidate generation and filtering,
T_deadline, an economic adaptation deadline,
local T_detect, an observable-distribution sensitivity estimate,
analysis reports and Markdown summaries.

It is small on purpose. It is not a full solver and it is not wired to real solver ranges. It starts from one toy game — a river spot — that is tiny enough to inspect and test by hand.

That toy spot is one where showdown always chops but rake still bites. In a single-hand view, putting more money into a raked pot can be locally unattractive. Across repeated occurrences the same spot raises a commitment question: if one player refuses to fold in a fixed pattern, how does the other respond, and how fast would that response have to come for the commitment to stop being worth it?

This is the question I wanted a tool to make precise — not a claim that any new equilibrium exists.

Why repeated poker is tempting — and where the trap is

Repeated games sound like a natural home for reputation, punishment, and adaptation, and poker has obvious repeated structure: similar river spots, similar blind-vs-blind situations, similar sizings, similar pools.

Here is the trap I had to respect. If the number of repetitions is known, the game is fully observed, each spot is independent, and both players are perfectly rational, then a finite repeated game often collapses back toward the one-shot equilibrium by backward induction. "This spot happens five times" is not by itself enough to claim a reputation equilibrium. That is the standard game-theory result, and it is the reason the project keeps the layers below separate.

So the project keeps several ideas apart that are easy to blur:

a one-hand baseline strategy,
a fixed Hero commitment candidate,
Villain's exact best response to that fixed Hero strategy,
an economic deadline for adaptation,
a local estimate of how visible the change is,
and the much stronger claim of a repeated-game equilibrium.

The MVP mostly lives in the commitment-analysis layer: if Hero is fixed to a candidate strategy in the supplied tree, what are Villain's exact best responses, and what happens to Hero EV under conservative tie handling?

What the MVP can do

(This describes the MVP on main at the time of writing. I'm still changing it, so details may move.)

It runs an end-to-end candidate-analysis pipeline on a small abstract game:

build tiny finite two-player game trees with rake,
evaluate fixed Hero and Villain mixed strategies,
enumerate exact Villain best responses for small trees,
report Hero EV under worst- and best-case Villain best-response tie rules,
generate simple Hero candidates from a baseline,
filter candidates before comparison,
compare candidate values against a baseline profile,
compute T_deadline and local T_detect,
render a Markdown summary.

In plain terms, the analysis loop is:

Start from a baseline profile: a fixed Hero strategy and a fixed Villain strategy on the supplied tree. These are action probabilities at each information set — not hand ranges; the tool does not model or import real solver ranges.
Generate Hero candidates by shifting probability between two actions at a single Hero information set (a blind, systematic enumeration of small shifts — not a search aimed at hurting Villain).
For each candidate, lock Hero to it and compute Villain's exact best response. That yields Hero's worst-case EV after Villain adapts.
Flag a candidate robustly_profitable only when that post-response worst-case Hero EV is strictly higher than Hero's EV in the baseline profile. The point is not "positive EV" — it is "still better than the one-shot baseline even after the opponent best-responds."
T_deadline / T_detect then add repeated-game timing on top of the candidates that survive.

The main entry point is run_candidate_analysis_pipeline.

python scripts/check_mvp.py

A simplified workflow:

from nuts_chop_river import build_nuts_chop_river, default_hero_strategy
from candidate_library import baseline_villain_strategy

from repeated_poker import (
    CandidateFilterConfig,
    CandidateGenerationConfig,
    run_candidate_analysis_pipeline,
)

tree = build_nuts_chop_river()
baseline_hero = default_hero_strategy()
baseline_villain = baseline_villain_strategy()

result = run_candidate_analysis_pipeline(
    tree,
    baseline_hero,
    baseline_villain,
    generation=CandidateGenerationConfig(shift_amounts=[0.1, 0.2]),
    horizon=5,
    profit_tolerance=-2.0,
    max_selection_l1_distance=0.3,
    detection_log_likelihood_threshold=3.0,
    detection_occurrence_probability_per_opportunity=0.5,
    filtering=CandidateFilterConfig(
        max_l1_distance=0.3,
        min_required_observations=5,
    ),
)

print(result.markdown_summary)

The output is a diagnostic report for the model you supplied, not a poker recommendation. Here is an excerpt from the actual output of examples/analysis_pipeline.py on the nuts-chop river toy game (I've trimmed the Configurations block and some columns; 8 candidates generated, 6 dropped by the filter, 2 compared):

generated=8 kept=2 excluded=6
compared=2

## Candidate Analysis Summary

### Summary Counts
- total: 2
- eligible: 2
- excluded: 0
- minimum_villain_ev: 1
- pareto_frontier: 2

### Candidate Rows

| candidate_id            | fixed_hero_ev | post_response_hero_ev_worst | robustly_profitable | t_detect | exclusion_reasons |
| ----------------------- | ------------- | --------------------------- | ------------------- | -------- | ----------------- |
| H1\|check->bet\|shift=0.1 | 0.625         | -0.850                      | no                  | 278      | -                 |
| H1\|bet->check\|shift=0.1 | 0.275         | -0.750                      | no                  | 294      | -                 |

The baseline Hero EV in this run is +0.45. The column that matters is robustly_profitable: it is yes only when post_response_hero_ev_worst exceeds that baseline. Here both candidates are no (-0.85 and -0.75 are below +0.45). A candidate that clears the baseline is rare and can exist in constructed cases — the tool's job is to search the candidates and find it when it does. The next section is a hand-built spot where one does.

A toy game where the commitment beats the baseline

I needed at least one example where the machinery clearly does what it is meant to: a known spot where committing to a fixed strategy leaves Hero better off than the one-shot baseline, even after the opponent best-responds. This nuts-chop steal is that example, and I wrote a dedicated test for it (tests/test_nuts_chop_steal_commitment.py). Treat it as a check that the tool can detect the effect at all — not as the end goal, and not as a claim about real games. Outside this constructed spot I do not know which situations, if any, are profitable to commit in.

The spot: a river where the board is already the nuts, so every showdown chops. There is no value betting — the only reason to bet (shove) is fold equity. Rake is below its cap, so a called pot just bleeds chips to the house. With a small starting pot and a big shove, a single hand looks like this:

initial commitment = 1, initial pot = 2, bet = 98, rake = 5%, cap = 4

| Line         | Hero/IP EV | Villain/OOP EV |
|--------------|-----------:|---------------:|
| check-check  |      -0.05 |          -0.05 |
| bet-fold     |      -1.00 |          +1.00 |
| bet-call     |      -2.00 |          -2.00 |

In one hand the caller folds: -1.00 (fold) beats -2.00 (call). So the one-shot subgame answer is OOP bets / IP folds — a pure steal, since the board is a chop and there is no value in betting.

Now lock IP to always call and ask the tool for OOP's exact best response. The steal's only profit source (fold equity) is gone, a called pot is -2.00 for OOP, so OOP's exact best response flips to check — and check-check is -0.05 for both. The test asserts exactly this: solve_exact_response returns {"OOP_river": "check"} once Hero is locked to call.

And crucially, this clears the baseline: Hero's EV goes from -1.00 (the one-shot steal baseline) to -0.05 after OOP adapts — still negative, but strictly better than the baseline, which is exactly the robustly_profitable condition. That is the whole point of the project stated in one example: the one-shot subgame answer (bet/fold) is not the answer under the fixed commitment I wanted to test (check/check). The commitment to call removes the opponent's only incentive to bet. (Whether this constitutes a repeated-game equilibrium is the stronger claim I am deliberately not making — this is a commitment-analysis result, not an equilibrium proof.)

The tool also puts a number on how long that commitment stays worth it. With baseline Hero EV = -1.00 (steal), pre-adaptation = -2.00 (locked call while OOP still bets), post-adaptation = -0.05 (OOP has switched to check), T_deadline comes out as floor(1 + 19N/39):

| N (horizon) | T_deadline |
|------------:|-----------:|
|          10 |          5 |
|          20 |         10 |
|          50 |         25 |
|         100 |         49 |

The honest caveat: this is a tiny, hand-built tree, and the EVs are ones I can check by hand — that is exactly why I trust this result more than anything else in the repo. It is not evidence about real games; it is evidence that the model and the code agree on one constructed example built to validate the effect.

Verification on my machine:

python -m pytest tests/test_nuts_chop_steal_commitment.py -v → 15 passed
python -m pytest -q → 500 passed
python scripts/check_mvp.py → passes
git diff --check → clean

How I worked with AI

I supplied the algorithm and the poker model. Codex wrote the implementation instructions and reviewed the results; Claude Code wrote the code. I checked Codex's prompts and corrected wrong premises, but I did not review the code line by line — I relied on the Codex/Claude review loop and the test suite (currently 500 passing tests).

Two things from that process are worth recording:

The assistant kept drifting toward the general case. For the commitment analysis I wanted Hero fully fixed and only Villain's exact best response computed, but it repeatedly tried to set up CFR — wasted machinery when Hero is fixed. Stopping it led to a side question I hadn't considered: CFR with one side frozen looks like a fixed-environment learning problem. I'm noting that as a question, not a result.
Explaining the toy game to the model was harder than explaining it to a person — it over-generalized and assumed things that don't apply (e.g. Villain value-bets on a board that is already the nuts). I ended up brushing each spec up in a chat first, then handing the cleaned version to the coding agent.

A note on terms the code keeps separate: T_deadline is economic (how late Villain can adapt while the locked policy still beats the baseline); T_detect is visibility (how many local observations before the candidate's action distribution looks distinguishable from baseline). They are different questions.

What I learned

Best-response ties matter. If Villain has several best responses with identical Villain EV, Hero's EV can still differ across them. Returning one arbitrary response would hide that risk, so the MVP reports both ev_h_worst and ev_h_best across the tie set. (Verified: BestResponseResult exposes both and the action variation across optimal pure strategies.)

Small examples are not a weakness. The nuts-chop river benchmark is tiny on purpose: easier to hand-check, harder to mistake for a real-money recommendation.

Current limitations

The main one: the code has not had an independent human code review. Tests pass, but I haven't read the implementation line by line and nobody else has either. Rather than rely on reading the code, I plan to validate it from the outside — design the verification to be as exhaustive as I can make it, run simulations across many configurations, and check that the results hold up. Whether static or property-based checking can give that coverage is something I'm still working out.

The narrower limits: it is not a full solver, does not import real solver ranges yet, does not solve large no-limit games, and does not do STT / ICM / preflop push-fold yet. The exact response engine enumerates Villain pure strategies, so it is meant for small abstract trees only — there is an explicit max_pure_strategies ceiling, default 100,000. Candidate generation is simple: finite shifts from a baseline, not a continuous strategy space.

Most importantly: positive EV inside this model does not guarantee profitable play. The model can be wrong if the abstraction, action tree, rake rule, ranges, or adaptation assumptions are wrong.

This is not gambling, bankroll, financial, or legal advice.

Next steps

The toy game confirmed the effect, so next I want to extend the tool: analyze with hand ranges rather than abstract action probabilities, and model the opponent adapting gradually (e.g. a Bayesian update of their response over repetitions) instead of switching to an exact best response in one step. Alongside that, I want to firm up the outside-in verification described above before trusting results on new spots.

Links

Repository: guriguri215-lang/repeated-poker-analysis
MVP walkthrough: docs/mvp_walkthrough.md
Assumptions and limitations: docs/assumptions_and_limitations.md
Publication policy: docs/publication_policy.md

Disclosure: I used AI assistance throughout this project and to draft this article. The division of labor was deliberate: I supplied the algorithm and the poker model, Codex handled instructions and review, and Claude Code wrote the code; I checked the prompts and relied on automated review and tests for the implementation. This article was also drafted with AI help and then rewritten to reflect my own decisions, mistakes, and open questions. Technical claims are marked where I have verified them against the code myself; where I say something is provisional or unreviewed, that is literally true.

DEV Community: ty215

A Small Prototype for Cost-Aware Bug Investigation

The question that started it

The boundary matters

P1a: make the investigation action explicit

What P1a showed

P1b: make the evidence less synthetic

The metadata optimism gap

P1c: stop hiding behind averages

What the worst-case view exposed

Why call it a game at all?

Takeaways

The Less I Type, the Better the Output: A Context Framework I Had Claude Design for AI Coding

The pain I kept hitting

The core idea: the human doesn't write long prompts

The four failure modes, and the design response

1. Going off the rails

2. Too many PRs, reviews that never end

3. My own bias pulling the work along

4. Rubber-stamp approvals

What surprised me, and where this stops

Takeaway

Before the Algorithm: Building the Input Layer for My Poker Analysis Tool

What changed since the last post

Why the input layer came first

What a JSON scenario is

The five scenario modes

Why I built form models before the GUI

What the inspect / roundtrip / edit CLIs check

Working with AI as the codebase grew

Tests and verification

Current limitations

What I want to do next

Links

Why I Built a Tiny Repeated-Game Poker Analysis Tool

What repeated-poker-analysis is

Why repeated poker is tempting — and where the trap is

What the MVP can do

A toy game where the commitment beats the baseline

How I worked with AI

What I learned

Current limitations

Next steps

Links

What `repeated-poker-analysis` is