DEV Community

Cover image for Measuring Puzzle Difficulty With Random Play (12 Mahjong Layouts)
Instant Games
Instant Games

Posted on

Measuring Puzzle Difficulty With Random Play (12 Mahjong Layouts)

How hard is a puzzle level? Tile counts are a poor answer. A 144-tile layout can be gentler than a 100-tile one if the shape is forgiving. When I added 12 layouts to Instant Mahjong Solitaire, I wanted a number that tells players and me something real.

This post is about one cheap trick: measuring difficulty with random play.

The layouts

The game has 12 layouts, from 60 to 144 tiles: Harmony, Garden, Phoenix, Pyramid, Pagoda, Crane, Lotus, Butterfly, Dragon, Turtle, Bridge and Mountain. They differ in height, symmetry and how many tiles start locked under others.

Every deal is generated so that a winning path exists. That removes "impossible" from the equation, which leaves a different question: how easy is it to find the path?

The random-play bot

The measuring tool is almost embarrassingly simple.

// simplified sketch
function randomPlay(board, rng) {
  while (true) {
    const moves = freePairs(board);
    if (moves.length === 0) return board.isEmpty();   // cleared or stuck
    board.remove(moves[Math.floor(rng() * moves.length)]);
  }
}

function clearRate(layout, runs = 2000) {
  let wins = 0;
  for (let i = 0; i < runs; i++) {
    if (randomPlay(deal(layout, seed(i)), mulberry(i))) wins++;
  }
  return wins / runs;
}
Enter fullscreen mode Exit fullscreen mode

The bot picks a random legal pair every turn, with no hints and no lookahead. If it clears a layout often, careless play is enough to win, so the layout is gentle. If it almost never clears, the player must think.

Reading the numbers

On the layouts I tested, clear rates ranged widely. Garden sat around 70%, and Mountain around 20%. The numbers are a rough guide, not a law, but they give three benefits:

  1. Ordering. I can sort layouts from calm to demanding based on data and not my own feeling.
  2. Surprises. A layout I thought was hard but that the bot clears easily is a sign the shape is too open.
  3. Regression checks. If I change the generator or the matching rules, I rerun the numbers and see whether difficulty drifted.

What random play does not capture

Be honest about limits:

  • A random bot does not use unlimited undo and hints, which human players have. So human clear rates are higher than the bot's.
  • The bot does not plan. A skilled human can clear boards the bot nearly never does.
  • It measures the chance of stumbling into a win, not the enjoyment of the layout.

So I treat the clear rate as a floor for difficulty, not the final word.

Using it for design

Once you have a number, you can aim for it. If I want a "medium" layout, I look for one in the middle of the range. If a new layout lands at 5%, it is probably too punishing for a mainstream audience and might be better as a late-game challenge.

Why this works for other puzzles

The same approach fits any puzzle with a legal-move generator: sliding puzzles, match-3 boards, block puzzles. Build a dumb bot, run it a few thousand times, and plot the results. It costs an afternoon and tells you things playtests miss.

Try the layouts

You can see how they feel in Instant Mahjong Solitaire, a free mahjong game online with no download.

Have you measured difficulty with a bot? What was your metric?

If you want mahjong solitaire tips, start with a smaller layout and work up. It is a free mahjong solitaire game with mahjong online access in any browser.

Top comments (0)