DEV Community: ethantao14

Testing game rules with bots: Monte Carlo on a TypeScript reducer

ethantao14 — Sun, 04 Oct 2026 00:36:32 +0000

If your game logic is a pure reducer, (state, action) => newState, then a bot is just a function from state to action. That means you can measure your rules by playing thousands of games, not just test that they do not crash.

I did this for Game Mode Sandbox, a front-end-only Next.js and TypeScript site that I build with two teammates. It runs one stock-drafting game under six rule sets, with fake companies and generated prices. I own the engine. This post walks through the "Mode Lab" I added this week: five bots, a seeded runner, a terminal command, and a browser page. Players see nothing new. It is a developer tool. (I built it with Cline.)

1. A bot is a function

The engine exposes createGameReducer(market), which returns (state, action) => newState. An action the rules do not allow returns the same state object, unchanged, and never throws. Given that, a bot only needs this:

// A bot sees what a player sees and returns one action. The runner calls it until the game ends.
export type Strategy = {
  name: string;
  description: string;
  chooseAction(state: GameState, market: Market, rng: Rng): GameAction;
};

export const bigSpender: Strategy = {
  name: "Big spender",
  description: "Buys every round it can, always at the largest allowed amount. Never sells.",
  chooseAction(state, _market, rng) {
    return buyOrSkip(state, rng, largestAmount);
  },
};

I wrote five, each one a probe for a specific rule:

Random: sells now and then (25% chance when it can), skips 1 round in 5, and spends a random allowed amount.
Big spender: always buys at the largest allowed amount. Never sells.
Penny pincher: always buys at the smallest allowed amount. Never sells. Probes the minimum-spend rules.
Loyalist: only buys from the industry of the oldest stock it owns. Probes the industry limits.
Churner: sells everything the rules allow each round, then buys at the largest amount. Probes the selling rules.

2. The runner

playGame in lib/lab/run.ts starts a game with startGame(config, market, seed), then loops: ask the bot for an action, feed it to the reducer, count what happened. Here is a simplified version:

// Simplified from lib/lab/run.ts. Helper names are shortened.
const reduce = createGameReducer(market);
let state = startGame(config, market, seed);

while (!isOver(state)) {
  if (!hasLegalAction(state)) { stats.stuck++; break; }
  const action = bot.chooseAction(state, market, actionRng);
  const next = reduce(state, action);
  if (next === state) throw new Error(`${bot.name} made an illegal move`);
  record(stats, state, next);
  state = next;
}

Three choices are worth calling out.

Illegal moves fail loudly. The reducer returns the same object for an illegal move, so next === state means the bot broke the rules. Throwing there means a buggy bot cannot quietly skew the averages.

A stuck check. Before each move, the runner checks that the player has at least one legal action (buy, skip, or sell). An earlier version of the engine had a real stuck-game bug, so this is a regression guard.

Separate randomness. Actions use their own random stream derived from the game seed, so any single game can be replayed exactly.

Per game, the lab records buys, skips, rounds with nothing buyable, the share of cards blocked by rules, sales, auto-sells, idle cash, final value, and stuck games. The table shows per-game averages plus the 10th percentile, median, and 90th percentile of final value. From the terminal it runs with:

npm run lab -- --games=1000 --seed=2024

3. Fair comparisons: same seeds for every bot

Every bot plays the same seeds within a mode. Without that, one bot could simply draw better boards and you would be measuring luck. With it, differences between rows come from behavior.

A nice side effect is a built-in sanity check. In modes without selling, the Churner cannot sell, so its row should be identical to Big spender's. It is.

4. Watch out: count transitions, not steps

Before I opened the first PR, a review flagged that auto-sells were being double-counted.

Auto-sells happen as a new round starts, and the engine records them as events on that round's state. When the last round ends, the engine marks the game done but keeps that round's events on the final state. The lab counted auto-sell events after every move, so the last round's auto-sells got counted twice. Big spender in Active Trader showed 8.0 auto-sells per game instead of 7.0.

The check was arithmetic. Active Trader has 10 rounds and a 3-year hold, so stocks bought in rounds 1 to 7 reach their hold by round 10: 7 auto-sells.

The fix is to count only on transitions:

// Simplified: only count auto-sells when the round actually advances.
if (next.round !== state.round) {
  stats.autoSells += countAutoSells(next);
}

I wrote the test first. It failed on the old code (got 4, expected 3 in Sector Focus), then passed with the fix. All 347 existing tests had passed with the bug present.

Rule of thumb: when state carries events forward, count them when the state changes phase, not on every step.

5. Running it in the browser without freezing

The /dev/lab page runs the same lab in a React client component. You pick games per bot and a seed, press "Run the lab", and the tables fill in one mode and bot pair at a time. Between pairs, it yields so the page can repaint.

My first version yielded with setTimeout(0). A full run took 13.0 seconds in Chrome, even though the terminal does the same work in about a second. The tell: 100 games per bot took as long as 1,000.

Chrome throttles chained timers in pages it treats as hidden. In my test, 20 chained setTimeout(0) calls took 11.5 seconds, and 20 MessageChannel messages took 0ms. So I yield the way React's scheduler does:

const nextTick = () => new Promise<void>((resolve) => {
  const channel = new MessageChannel();
  channel.port1.onmessage = () => {
    channel.port1.close();
    resolve();
  };
  channel.port2.postMessage(null);
});

The run dropped from 13.0s to 4.2s in a background window. A Web Worker would move the work off the main thread entirely. This was the smaller change, and it was enough.

6. Testing the page

The page and the terminal script share one list of column definitions, so they always print the same numbers. A test renders the page, runs it, and compares the cells to the runner's output for the same seed.

That test passed on my laptop and failed on CI after 1,267ms. Testing Library's waitFor gives up after 1 second by default, and the CI runner was just slower. I gave that waitFor a 10-second timeout and the test 20 seconds. A test that does real work has a time budget, so set it on purpose.

7. Was it useful?

Here is Sector Swap at 1,000 games per bot, seed 2024 (per-game averages):

Bot	Buys	Sales	Idle cash	Median final value
Big spender	8.0	0.0	$0	$11,571
Penny pincher	8.0	0.0	$6,000	$10,402
Churner	8.0	7.0	$8,038	$10,601

Two findings from the full run:

In Sector Swap, selling every round caps you at one stock. A sale must be replaced from the same industry, and the replacement uses that round's pick. Two reasonable rules combine into a play style nobody designed.
In Classic Draft, the Loyalist makes 1 buy in 8 rounds and leaves $8,750 idle, because the one-per-industry rule blocks the rest of its industry.

Neither is a bug. Both are things we would not have known from reading the rules. The suite is at 355 tests, and the whole lab is a few hundred lines on top of an engine that was already pure.

Repo: github.com/AryamanGandhi/app-store

Three Bugs That Survived Every Automated Check (And How Line-by-Line Review Caught Them)

ethantao14 — Wed, 16 Sep 2026 20:45:30 +0000

Game Mode Sandbox is a new project our team is building alongside our stock market game: an app-store-style site where each app is a front-end-only version of the stock draft game running on fake data, one per rule variant, so we can playtest which rules feel best before locking anything into the real product. Kenny owns the store pages and data, Jane owns the gameplay screens, and I own the engine, the pure logic layer that decides what a player can do and what happens when they do it.

Design principles for the engine

Pure functions, no UI: every rule can be tested directly, no clicking through screens required
Seeded randomness: every random draw, like which stocks appear each round, comes from the game's seed, so the same seed replays the same game and bugs are reproducible
Invalid actions are no-ops: they leave the game state unchanged instead of crashing
Money is always rounded to whole cents
Market data is passed in, never hardcoded, so tests can supply their own

What shipped

Four pull requests merged this week, +1,703/-25 lines total:

Config validation (+792/-10): checks whether a rule setup is actually playable and explains any problem in plain English, pointing at the setting to fix
Board drawing (+456/-5): picks which stocks appear each round across all three board types, greys out unpickable stocks with a reason, and guarantees at least one pickable stock every round
An off-by-one fix (+30/-5), covered below
Spend validation (+425/-5): works out how much a player can spend per pick under four different money rules, holding back enough cash to cover remaining picks

140 tests pass on main, 122 of them for the engine, roughly two lines of test code for every line of engine code.

Three bugs, all caught after everything was green

Lint, typecheck, the full test suite, and the build all passed on every pull request. An automated code reviewer ran on two of them and reported no actionable defects. It also could not run the tests in its own sandbox. All three bugs below were found by reading the code line by line against the written spec, not by any of the automated tooling.

Bug 1: the shrinking board

Classic Draft is supposed to show 5 stocks every round. Instead it went 5, 5, 5, 5, 4, 3, 2, 1.

// Before
const unownedIndustries = industriesWithStock.filter(
  (industry) => !ownsIndustry(state, industry)
);

The one-per-industry board type was filtering out sectors the player already owned before drawing, a rule meant only for the same-industry board type. One committed test even asserted the shrinking 1-card board as correct behavior, since it was written from the buggy code instead of the spec. The fix draws from every industry with an available stock and greys out owned ones instead of removing them. Verified against 5,000 randomly generated game states across five rule sets: 0 violations.

Bug 2: the hold that never ends

// Before
holdYears > rounds
// After
holdYears >= rounds

A stock bought in round 1 of a 5-round game with a 5-year hold needs a 6th round to auto-sell, which never comes. The check only warned when the hold was strictly longer than the game, not equal to it, so this exact setup passed with no warning. The original tests asserted that this case should not warn, which is how it slipped through.

Bug 3: the lost cent

// Before
Math.floor((cash / picks) * 100) / 100
// After
Math.floor(Math.round(cash * 100) / picks) / 100

Floating point decimals like $0.57 are stored as something closer to 0.5699999, and flooring that drops a cent. Tested across every dollar amount from $1.00 to $10,000.00, split 2, 3, 5, and 8 ways: the old formula was wrong in 98,440 cases. The fix does the arithmetic in whole cents, which are exact integers. After the fix: 0 errors across 714,215 checked combinations.

Takeaway

A green checkmark tells you the code matches the tests. It does not tell you the code matches the spec, and a test written from the code instead of the spec can lock in a bug with just as much confidence as it confirms a fix.

Repo: https://github.com/AryamanGandhi/app-store

Two Cline Features That Actually Changed How I Work: Permissions and Diff Explanations

ethantao14 — Wed, 26 Aug 2026 19:19:49 +0000

I have been building the draft and portfolio builder side of a stock draft game using Cline, running on Sonnet through a Claude Console API key. Most of the writing about AI coding tools focuses on whether the code is good. That matters, but two other things ended up mattering just as much day to day: how much control I actually had over what the tool could do, and whether I could tell what it had just done. Here is a concrete look at both, pulled directly from this project.

Permissions: a settings menu, not a guessing game

Every agentic coding tool has to answer the same question. How much should it be allowed to do without asking first. Too restrictive and you spend your day approving trivial actions. Too permissive and one bad command becomes a real problem.

Cline handles this with a straightforward auto-approve menu rather than anything hidden or inferred from a conversation. Here is the exact configuration used throughout this project.

Cline's auto-approve settings menu.

Breaking down what each toggle actually does:

Read project files / Read all files, both on. The first scopes reads to files inside the current project. The second extends that to anywhere on disk. With this on, Cline never had to ask before opening a file for context.
Edit project files, on. Edit all files, off. Cline could freely modify anything inside the project without asking, but editing something outside the project's boundaries required explicit approval each time. This is the one I would flag as the most important default. It gives an agent real freedom inside the sandbox of your repo while keeping a hard stop at the edge of it.
Execute safe commands, on. Execute all commands, off. Cline distinguishes between commands it considers safe, things like running a linter or test suite, and anything it cannot classify that way. Safe commands ran without interruption. Anything else needed a manual yes.
Use the browser, off. Cline never had a reason to browse during this project, so this stayed off by default rather than being something to remember to disable.
Use MCP servers, on. This project used MCP tooling, so it was left enabled.

The practical effect: Cline moved quickly on everything that mattered for iteration speed, reading context, running lint and test commands, while the higher risk categories, editing arbitrary files outside the project, running arbitrary shell commands, browsing, stayed off unless deliberately turned on. None of this required trusting an opaque memory system or hoping a written instruction got remembered three sessions later. It was a set of checkboxes, visible at any time, adjustable in seconds.

If you are evaluating a coding agent for a real project, this is worth checking early. Can you see, in one place, exactly what it is allowed to do, or do you have to infer it from behavior?

Clarity: a diff explanation you actually get, not one you have to ask for twice

After Cline finishes a task, it shows two buttons next to its summary: View Changes and Explain Changes. View Changes is the raw diff. Explain Changes, when clicked, generates a plain-language explanation placed directly under the relevant code, file by file, with a reply box so you can ask a follow-up question right there, the same way you would comment on a pull request.

Here is a real example from this project. A review pass caught that four tickers in the shared sector data list had no matching historical price file, meaning a user could draft one and have it silently vanish from their results. The fix was a small, scoped helper function rather than editing the shared data file directly:

// lib/available-stocks.ts
import { STOCKS_BY_SECTOR } from "@/data/sectors";
import type { Sector, Stock } from "@/lib/types";

// A handful of tickers in the shared sector lists (data/sectors.ts) don't have
// a matching file in data/historical/, so picking them would silently show
// "No data" on the results page. Filtered out here, scoped to the draft flow,
// rather than editing the shared sectors list.
const TICKERS_WITHOUT_HISTORICAL_DATA = new Set(["HES", "PXD", "EMR", "ETN"]);

export function getAvailableStocks(sector: Sector): Stock[] {
  return STOCKS_BY_SECTOR[sector].filter(
    (stock) => !TICKERS_WITHOUT_HISTORICAL_DATA.has(stock.ticker),
  );
}

Clicking Explain Changes on the commit that wired this helper into the draft page produced this, directly under the diff:

Cline explaining why the getAvailableStocks wiring change mattered.

That explanation is not a changelog entry someone wrote after the fact. It is generated at the point of the change, describing not just what moved but why the one-line wiring change was the part that actually made the fix take effect in the running app. The reply box underneath means if that explanation is not enough, you do not have to open a new conversation and re-explain your own codebase to ask a follow-up. You ask it right there, with the diff still on screen.

The same pattern showed up when a review pass flagged that the homepage was still 100% unmodified create-next-app boilerplate, with zero link into the actual feature:

Cline explaining the homepage rewrite.

Across a project where a dozen or more files can change in a single task, this is the difference between being able to explain your own code to a teammate afterward and quietly hoping they do not ask too many questions about a diff you half remember.

Why both of these matter together

Permissions and clarity solve different problems, but they compound. A tool that moves fast without asking permission for every action is only comfortable to use if you also know, clearly and immediately, what it just did and why. Cline's auto-approve menu gives you control over the blast radius. Explain Changes gives you visibility into what happened inside that radius. Neither one alone would have been enough to make me comfortable handing over a real feature build. Together, they made it a normal part of the workflow rather than something to babysit.

Repo: github.com/ethantao14/cline-stock-market-game