DEV Community: Jeff Lowery

The Transposition Trap and 3,600 "Fake" Chess Positions

Jeff Lowery — Tue, 14 Jul 2026 22:08:18 +0000

Part 1: The Graph, The Orphans, and the Synthetic Bridge

If you ask a casual chess player what an opening is, they’ll give you a simple answer: it’s a name, an ECO (Encyclopedia of Chess Openings) code, and a sequence of moves.

1. e4 e5 2. Nf3 Nc6 3. Bb5 is the Ruy Lopez. Clean. Elegant. Linear.

If you ask a software engineer who maintains a chess opening database how hard it could be to map these openings, they’ll likely utter the famous last words of every data engineer:

"It’s just a flat list of moves. How hard could it be?"

I know this because I am that engineer. I maintain @chess-openings/eco.json, an open-source data package on npm. And during a recent extensive rewrite of my data pipeline, I was violently reminded that in the world of chess data, linearity is an illusion, FEN strings are slippery shapes, and the data structure isn’t a list at all.

It’s a directed nightmare of a graph.

The Transposition Trap

Let’s start with the fundamental lie of chess data: that a position’s identity is tied to how you got there.

In chess, different move orders can lead to the exact same layout of pieces on the board. This is known as a transposition. Consider the Benoni Defense. You can reach a classic Benoni position via the canonical move order:

1. d4 Nf6 2. c4 g6 3. Nc3 Bg7 4. e4 d6

But what happens if Black plays a slightly different order to achieve the exact same strategic setup?

1. d4 Nf6 2. c4 c5 3. d5 d6 4. Nc3 g6

To the human eye, the board is identical. To a database relying strictly on Forsythe-Edwards Notation (FEN) as a strict unique identifier, they can look like entirely different entities. A standard FEN string doesn't just track where the pieces are; it tracks whose turn it is, castling rights, and potential en passant squares.

If White changes the order of their pawn pushes, castling rights or en passant availability might technically differ for a fleeting move, even if the piece layout converges. (Yes, I hear you, chess purists: technically, an active en passant option makes identical piece layouts mathematically distinct game states. But for the sake of navigation mapping, we must occasionally fly close to the sun).

If your data pipeline treats a full, standard FEN string as an absolute, immutable identity, you will instantly split your data universe in two.

The fix? I had to implement a strict position-only FEN matching fallback across the entire ingestion engine. In chess data engineering, you quickly learn that a full FEN is merely a suggestion; the raw piece placement layout is the true identity.

A Directed Nightmare: The `fromTo` Graph

Once you accept that positions transpose, your flat file of chess openings explodes into a massive directed graph. In my latest build of the pipeline, this graph ballooned to 17,500 edges connecting over 16,000 unique nodes.

[Position A] --(Move: c5)--> [Position B] --(Move: d5)--> [Position C]
    \                                                         /
     \-----------------(Alternative Path)--------------------/

Every single opening and variation in the database must know where it came from (from) and where it can go (to). If a single link breaks, that entire branch of chess history becomes unreachable noise in the application layer.

When I ran my newly rewritten pipeline for the very first time, it didn't gracefully compile. It screamed. The integrity report spat out:

45 disconnected interpolated openings (floating in the void)
87 duplicate edges (circular logic loops)
16 FEN collisions (two different opening names claiming the exact same board state)

Fixing this wasn’t a matter of tweaking a JSON file. It required rewriting four fundamental pipeline changes across three separate architecture files to enforce strict deduplication at the moment of generation, rather than trying to retrofit it after the fact.

The Interpolation Rabbit Hole

But the deepest psychological damage came from what I call "Orphan Openings."

When you scrape, pull, and normalize chess data from ten different online sources, you will inevitably find records for deeply specific sub-variations. For instance, a source might list a highly complex line twenty moves deep into the King's Indian Defense.

The problem? The source database gives you the start position and the end position, but it doesn't explicitly define the seventeen moves in between as distinct database entries. They are missing from the master list. In graph theory, this leaves your deep variation as an orphan—floating in space with no parent node to connect it back to the game's starting array.

To make the graph navigable, I had to build an interpolation engine. When the pipeline encounters an orphan, it is forced to go down a recursive rabbit hole. It triggers a function called lineOfDescent, which programmatically walks backward through the move sequence, move by move, generating synthetic bridge nodes until it finally collides with a known, recognized opening.

Think about the absurdity of this: I had to programmatically generate 3,625 synthetic chess positions solely so that real chess openings could find each other.

These interpolated entries don't have historical flavor text or specific human names; they exist purely as the digital mortar holding the bricks of the graph together. If the recursion logic is off by even a single character, it triggers a recursion explosion, spinning out infinitely as it tries to resolve transpositions that loop back on themselves.

The Joy of the Green Check

After days of wrestling with the graph topology, adjusting fallback FEN algorithms, and ensuring the lineOfDescent didn't eat my CPU, I finally ran the verification suite.

There is a unique, quiet satisfaction that only a developer knows when a wall of red terminal text finally turns into a clean, green monolithic block of passing checks:

✓ fen-unique: 12509 unique in ecoA-E, 3625 in interpolated, 0 overlaps
✓ eco-prefix: all entries match their category file
✓ interp-isolation: interpolated entries correctly separated
✓ fromto-ref: all 17565 transitions reference existing FENs
✓ fromto-dup: no duplicate transitions
✓ valid-src: all source identifiers are valid
✓ rootsrc: all 3625 interpolated entries have rootSrc
✓ interp-connect: all 3625 interpolated entries connected in fromTo graph

✗ 0 failure(s)

The graph was whole. The orphans were adopted. The synthetic bridges held.

But normalizing the graph topology was only half the battle. Once the architecture was solid, I had to plunge into the messy, politically charged, and deeply inconsistent world of human naming conventions—a data normalization war where there are no winners.

In the next article, we’ll dive into "Source Wars," dealing with the chaos of ten conflicting databases, and why "King's Indian Attack" and "Reti: KIA" are a data engineer's worst nightmare.

How to build an associative graph using React + p5

Jeff Lowery — Sun, 12 May 2024 19:21:23 +0000

The idea

I've been working on a chess opening database in my copious spare time, and have begun experimenting with visualizations of said openings. One of those is a relationship graph (see above) that displays origin square to destination square of each opening move.

The Farmer in the Dell

Relationships (a type of association) can be found everywhere. In this post, I'll graph the relationships as described in a nursery rhyme called "The Farmer in the Dell".

The farmer in the dell
The farmer in the dell
Hi-ho, the derry-o
The farmer in the dell

After establishing that there's a farmer that lives in a dell, a series of new characters are introduced via the verb "takes".

The farmer takes a wife
The wife takes the child
The child takes the nurse
The nurse takes the cow
The cow takes the dog
The dog takes the cat
The cat takes the mouse
The mouse takes the cheese
The cheese stands alone

The word "takes" is way overloaded here, but that can be addressed later. From this list of relationships we can see that the farmer and the cheese have a single relation (terminal nodes in the graph to come), but all other characters have two.

Graphing the characters

To display the relationships among the characters in the rhyme, we can start by arranging each in a circle.

I'm using the JavaScript graphics library p5 inside a react component, like so:

/* eslint-disable react/prop-types */
import { useRef, useEffect } from "react";
import p5 from "p5";

const characters = {
    farmer: {},
    wife: {},
    child: {},
    nurse: {},
    cow: {},
    dog: {},
    cat: {},
    mouse: {},
    cheese: {},
};

const Version1 = () => {
    const renderRef = useRef();

    useEffect(() => {
        let remove;

        new p5((p) => {
            remove = p.remove;
            p.setup = () => {
                const r = 250;
                const cast = Object.entries(characters);

                for (const [index, [key]] of Object.entries(cast)) {
                    const character = characters[key];
                    character.angle =
                        (p.TWO_PI / Object.keys(characters).length) * index;
                    character.location = [
                        r * p.sin(character.angle),
                        r * p.cos(character.angle),
                    ];
                }

                p.createCanvas(600, 600).parent(renderRef.current);
                p.background(150)
                //move 0,0 to the center of the canvas
                p.translate(p.width / 2, p.height / 2);
                p.ellipseMode(p.CENTER);
                p.textAlign(p.CENTER, p.CENTER);
                p.textFont("Georgia");

                for (const c in characters) {
                    const [x, y] = characters[c].location;
                    p.ellipse(x, y, 40);
                    p.text(c, x, y);
                }
            };
        });

        return remove;
    });

    return <div id="Version1" ref={renderRef}></div>;
};

Without delving too deep into p5, it's worth pointing out some features of this code.

How to render graphics inside a React component

const renderRef = useRef();
p.createCanvas(600, 600).parent(renderRef.current);
return <div id="Version1" ref={renderRef}></div>;

useEffect() cleanup function

    useEffect(() => {
        let remove;

        new p5((p) => {
            remove = p.remove;
        /* ... */
        });
        return remove;
   });

the p5 remove function 'cleans up' the previous render before useEffect is called again.

Setting up the character locations

         p.setup = () => {
                const r = 250;
                const cast = Object.entries(characters);

                for (const [index, [key]] of Object.entries(cast)) {

Note the two Object.entries() calls. This is done to get an index to calculate each character's angle from the center:

     character.angle =
        (p.TWO_PI / Object.keys(characters).length) * index;

Drawing the relationships

Now to draw a line from each taker to each taken character. First, I add a "takes" relation from farmer all the way down to cheese, then draw a line from the taker location to the taken location.

const doRelations = () => {
    // for this version, all relations are the same.
    const keys = Object.keys(characters);
    keys.forEach((key, i) => {
        if (i + 1 < keys.length) {
            characters[key].takes = keys[i + 1];
        }
    });
};

/* ... */
for (const c in characters) {
    const character = characters[c];
    const [x, y] = character.location;

    if (character.takes) {
        const taken = characters[character.takes];
        const [x2, y2] = taken.location;
        p.line(x, y, x2, y2);
    }

    p.ellipse(x, y, 40);
    p.text(c, x, y);
}

The result:

Not terribly interesting, but it's a start.

Requirements analysis

Let's reexamine the word "takes". It's usage is very ambiguous and often nonsensical. Here's one attempt to disambiguate the rhyme:

The farmer married a wife
The wife married a farmer
The wife adopts the child
The wife employs the nurse
The child needs the nurse
The nurse cares for the child
The farmer owns the cow
The nurse milks the cow
The farmer owns the dog
The dog guards the cow
The dog befriends the cat
The cat befriends the dog
The cat adopts the farmer
The cat hunts the mouse
The mouse eats the cheese

Let's see what the relationship graph looks like now:

Not bad, but now let's color-code the relationships and add a legend:

Drawing curves instead of lines

One problem with using lines is that in reciprocal relationships, such as "married", one line overwrites another, obscuring the former relationship. Bezier curves can be used instead of lines, and look better. Each bezier curve takes eight arguments, which are the (x,y) coordinates of:

the start point
the first control point
the second control point
the end point

What are these control points? Here's an illustration:

The shape of the curve can be adjusted by moving the control points. In this case, I'll set the first control point to be halfway between the start location and the center of the graph, and similarly for the end point.

    const cp1 = [x / 2, y / 2];
    const cp2 = [x2 / 2, y2 / 2];

    p.noFill();
    p.bezier(x, y, ...cp1, ...cp2, x2, y2);

The result is:

Though aesthetically better, the change didn't fix the overwrite problem. The control points can be adjusted for each character by adding a "fudge" factor to the control point equation:

    const fudge = (v) => ((v*i*3)/40)

    const nx = x + fudge(x)
    const ny = y + fudge(y)
    const nx2 = x2 + fudge(x2)
    const ny2 = y2 + fudge(y2)

    i++;

    const cp1 = [nx / 2, ny / 2];
    const cp2 = [nx2 / 2, ny2 / 2];

Here, the variable i is a counter that is increased as each character in the rhyme is rendered. The final output is:

Arrows? What about arrows?

One thing missing from the diagram is an indication of the direction of the relationship, e.g. "farmer owns dog" and not "dog owns farmer". However, there is no native support for arrowheads in p5, so it takes quite a bit of trigonometry to draw them...another post in itself. See links below for more info on the subject.

References

A complete program can be found here.

Useful links:

This example gets halfway to an arrow feature.

One thing is to have the tip of the arrow touch the edge of each character's circle, using the formula in the answer found here.