Syncing level data from seven games into a static Next.js site

Mobile Casual Games is a guide hub for puzzle games. It currently covers seven of them, and the level rows behind those guides add up to just over 8,000. New walkthroughs keep landing in the database, so the data moves constantly. The site itself is an output: "export" Next.js build: no runtime API, no database connection, nothing fetched when a visitor loads a page. Every level list, every range page, every redirect was decided at build time from JSON files that live in the repository.

That raises a practical question: when the data keeps changing, how does it get from wherever it was produced into a static bundle without anything breaking in between? This is how the pipeline works.

Where the rows come from

The source of truth is a Neon Postgres database with two tables. games holds store metadata: name, developer, icon URL, Play Store link, rating, install count. game_levels holds one row per level:

game_name | level | title | video_id | watch_url | img_url | created_at

video_id is a YouTube id, watch_url the matching watch link, img_url the cover thumbnail we uploaded to our own image host. Getting rows into that database is a separate job per game. For Food Hunt, our capture pipeline downloads each walkthrough video, extracts the cover frame, and uploads it to Cloudflare R2, so the site never hotlinks. For another game the source was someone's WordPress walkthrough site, which exposed the native REST API. A small Python script (scraper/scrape_game.py) walks the sitemap, filters slugs by a per-game prefix, and pulls clean JSON in batches of 50 slugs per request. It writes both JSON and CSV. The REST API turned out to be the right call there: the cached HTML pages had ad blocks injected into the body, and parsing around them would have been fragile.

Whatever the source, once the rows are in Postgres the web repository stops caring about them until it needs a fresh copy.

Two sync scripts, because history

There are two download scripts, which looks redundant until you see which files each one owns.

node scripts/download-data.mjs                       # games.json + levels.json
node scripts/download-levels.mjs rotaterings meowdoku # src/data/levels/<game>.json

download-data.mjs is the original. It was written when the hub was a single-game site for Food Hunt, and it still writes those two legacy files: every game row into games.json, and the full Food Hunt level list into levels.json, mapped to the shape the page components render.

download-levels.mjs is the multi-game sibling. You pass it one or more game names and it rewrites src/data/levels/<game>.json, one snapshot per game. Everything newer reads from these files: level counts, the 1-100 style range pages, the Level Guide in the navigation, the sitemaps, and, for the two games without a dedicated site, the level pages themselves.

Both scripts do the same three things beyond the query itself:

// Force IPv4 first - Neon is fronted by Cloudflare and returns both A and
// AAAA records; some sandboxes/serverless runtimes can't route IPv6.
dns.setDefaultResultOrder("ipv4first");

// env resolution: real environment first, then --env-file, then ./.env
const sql = postgres(rawUrl, { connect_timeout: 15 });

and they write the snapshot with a small envelope:

{
  "game": "rotaterings",
  "updatedAt": "2026-10-12T03:26:45.530Z",
  "count": 170,
  "levels": [
    {
      "id": 1,
      "name": "Level 1",
      "image": "https://img.mobilecasualgames.com/rotaterings/level-1.webp",
      "videoId": "...",
      "embedUrl": "https://www.youtube.com/embed/...",
      "watchUrl": "https://www.youtube.com/watch?v=...",
      "difficulty": "Beginner",
      "publishDate": "2026-10-12"
    }
  ]
}

updatedAt and count make the git diff readable. When you commit a refreshed snapshot, the diff header tells you which game was re-pulled and roughly how much moved, without opening the file. If a query returns zero rows the script warns and skips instead of writing an empty snapshot, so a typo in the game name cannot quietly erase 1,400 levels.

Reading the snapshots at build time

src/data/games.ts is the registry. Each snapshot is a static import, and each game is one entry with its display metadata, a theme color, and the level data attached:

import rotateringsLevels from "@/data/levels/rotaterings.json";

export const GAMES: GameEntry[] = [
  {
    game_name: "rotaterings",
    shortTitle: "Rotate Rings",
    themeColor: "#2eaae4",
    // no levelUrlPrefix - the hub hosts this game's level pages itself
    levels: (rotateringsLevels as { levels: StoredLevel[] }).levels,
  },
  // ...six more
];

The levelUrlPrefix field decides where a game's level detail pages live. Five of the seven games have their own guide site, so on the hub /<game>/<id> is a 301 to https://that-site/levels/<id> and the hub only renders lists. Smash Fest and Rotate Rings have no dedicated site, so the hub renders their pages directly.

Above the registry, src/data/levels.ts is a plain accessor module: fetchLevels, getLevel, getLevelIds, getMaxLevel, getFeaturedLevels. Every function takes a gameName as its first argument and resolves it through the registry, so the same code serves all seven games and nothing in the page components knows or cares which snapshot a number came from.

One detail worth copying: difficulty is recomputed on read, not trusted from the file.

function asLevel(l: StoredLevel): Level {
  return {
    ...
    difficulty: difficultyFor(l.id),  // thresholds live in src/lib/types.ts
  };
}

The snapshots were produced by different scripts at different times, and those scripts used different thresholds (the old single-game script bucketed at 100/250/400, the multi-game one at 100/500/1500). Deriving the value from the level number at render time makes the stored field meaningless, which means the snapshots no longer have to agree with anything. The same idea holds for the database: created_at is repurposed as the video publish date for structured data, because it is the only date the table carries.

When the database lags behind

Rows sometimes exist before they are inserted into Postgres, right after a fresh scraping run. There are two side doors for that case. import-levels-from-csv.mjs takes the same CSV the scraper produces and writes the identical <game>.json snapshot, using the same row-to-level mapping. build-data.py is the older door: it generated the original Food Hunt snapshots straight from the capture pipeline's output directory. Both exist so a snapshot can be regenerated from whatever artifact currently has the data, and the registry cannot tell the difference.

Derived files, regenerated on every build

The snapshots feed three derived outputs, all produced before next build runs.

Redirects. generate-redirects.mjs reads the levelUrlPrefix values straight out of games.ts and writes the per-game block in public/_redirects. One rule per game, not per level:

/foodhunt/1-100   /foodhunt/1-100   200
/foodhunt/*       https://food-hunt.org/levels/:splat   301

The splat is one rule because Cloudflare Pages caps a project at 2,100 redirects, and Pixel Flow alone has 2,089 levels. A per-level rule set would not fit, and it would need regenerating every time the game shipped a new batch. The self-rewrites for the block pages (/foodhunt/1-100 stays on the hub) are listed before the catch-all so the splat cannot swallow them. The same script handles the blog-to-level redirect for walkthrough articles, since a guide with level: 7 in its front matter renders on the level page, not at its old /blog/<slug> URL.

Guide badges. <LevelCard /> is a client component, and it needs to know which levels have a written guide. The markdown sources live server-side (content/blog/*.md front matter plus src/data/blog.ts, which imports node:fs), so generate-guide-levels.mjs scans them before each dev server or build and writes a tiny guide-levels.json. The client component reads that. Adding a guide stays a one-file change in content/blog/.

Sitemaps. generate-seo.mjs builds URLs from the registry and the snapshots. Since the five games with dedicated sites keep their level pages elsewhere, the main sitemap carries the hub's own pages plus the range blocks, and the two hub-hosted games get their own sitemap files (smashfestlevel.xml, rotateringslevel.xml) so a thousand-plus level URLs do not bury everything else.

Bundling limits

Static imports of 8,000 rows are server-side only, and the design keeps them there. toGameCard() strips every field except name, icon, rating, and levelCount: game.levels.length, which is all a card renders. The navigation's Level Guide receives a compact index per game (min, max, and the array of missing ids) instead of the id list. A browser that opens the level picker learns which numbers exist without downloading eight thousand objects, and the full snapshots never enter a client chunk.

Adding the eighth game

The whole point of the arrangement shows up here. A new game needs its rows in the database (or a CSV), one run of download-levels.mjs <game> to write the snapshot, one entry in games.ts, and nothing else. Range pages, counts, nav menu, redirects or hosted pages, sitemaps, and the build all follow from those two artifacts, and every step leaves a reviewable diff in git.

You can see the result at mobilecasualgames.com, and the pipeline described here is what keeps its seven level databases in one repository, refreshed with a single command and a commit.