DEV Community: ZHIWEI

What actually happens between “type a topic” and a lip-synced rap video

ZHIWEI — Tue, 03 Nov 2026 16:58:36 +0000

When we tell people we built a generator that turns a sentence — or a photo — into a 12-second rap performance, the follow-up question is always the same: "okay, but what is actually happening in there?"

This post is the honest walkthrough of the pipeline behind AI Rap Video (https://useairapvideo.com), step by step, with the failure modes we hit while building it.

Step 1 — Lyrics from a topic

Everything starts with text. The user gives a topic ("my coworker who microwaves fish"), optionally their own lyrics, and picks a stage template. A lyric model turns the topic into structured bars: rhyme pairs, a hook, and a length target that fits the video duration. The template matters here — a diss track needs punchlines, a birthday rap needs the recipient's name in the first two bars, so the prompt carries template constraints, not just the topic.

Step 2 — A rap voice performs them

The lyrics go to a rap-styled vocal model. This is not text-to-speech with a filter: delivery — where the flow sits relative to the beat, which syllables get stressed — is part of the generation. The same bars performed straight sound like a recital; performed with push-pull against the beat, they sound like a verse.

Step 3 — An original beat

The beat is composed per video and matched to the vocal take's tempo. We made one deliberate product decision here: no copyrighted audio, ever. No famous instrumentals, no "sounds like" beats. The audio is original by default, users can upload beats they own the rights to, and there is a DMCA takedown flow. It is more work than bundling known beats, but it is the only version of this product we were willing to ship.

Step 4 — Lip sync from a photo

If the user supplied a photo, a lip-sync model animates the face to the vocal: it extracts mouth-region landmarks, generates a mouth shape sequence aligned to phonemes, and composites it back. Front-facing, well-lit photos sync dramatically better than angled ones — the model needs to see the whole mouth. Two photos? The duo template maps each face to its own vocal part.

Step 5 — Render

The vocal, beat, and animated video are composited and rendered as a 12-second MP4 (480p or 720p). Each render consumes credits from a one-time pack — there is no subscription, which users mention unprompted more than we expected.

Watch it end to end

There are four full sample videos on the homepage at https://useairapvideo.com — diss track, birthday, Santa, and a two-person duo — if you want to see all five steps compressed into a dozen seconds each. The rap generator itself runs free 6-second previews, one per account, if you would rather poke at it directly.

Questions about any of the steps are welcome in the comments — especially on the lip-sync stage, which is where most of our iteration time went.

— AI Rap Video team

Grok TTS speech tags, tested: what , and [laugh] actually do

ZHIWEI — Sun, 01 Nov 2026 03:02:49 +0000

xAI's Grok text to speech API (POST https://api.x.ai/v1/tts) has 28 built-in voices, 20 languages and a set of speech tags you put straight into the text. The docs list the tags; they don't say how much each one changes. I measured a couple.

The request

const res = await fetch('https://api.x.ai/v1/tts', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.XAI_API_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    text: 'Stand back. <loud>This is not a drill.</loud>',
    voice_id: 'eve',
    language: 'en',
    output_format: { codec: 'mp3', sample_rate: 24000, bit_rate: 128000 },
  }),
});
const mp3 = Buffer.from(await res.arrayBuffer());

language also takes auto. Price is $15 per million characters (xAI pricing page, checked 2026-10-30).

Two kinds of tags

Inline tags insert a sound where they sit: [pause], [long-pause], [hum-tune], [laugh], [chuckle], [giggle], [cry], [tsk], [tongue-click], [lip-smack], [breath], [inhale], [exhale], [sigh].
Wrapping tags change how the enclosed phrase is read: <soft>, <whisper>, <loud>, <build-intensity>, <decrease-intensity>, <higher-pitch>, <lower-pitch>, <slow>, <fast>, <sing-song>, <singing>, <emphasis>.

What two of them measured

Same sentence, voice Eve, English, one clip with and one without the tag. One clip each, so this is a quick check rather than a benchmark.

Tag	Length	Loudness (RMS)	Median pitch
none → `<loud>`	5.5 s → 5.5 s	-20.7 → -20.9 dBFS	195 → 253 Hz
none → `<singing>`	6.5 s → 8.4 s	—	189 → 250 Hz

So <loud> is closer to "raised, urgent delivery" than a volume control: if you need a louder file, normalise it afterwards. <singing> stretches the words into a melody, which also makes the clip longer (and, on a per-character price, not more expensive).

Trying tags without writing code

If you want to hear the tags on all 28 voices before wiring up the API, I built a free front-end: Grok Text to Speech. The speech tag reference has a before/after sample for each group, and there are preset pages for whisper, shouting and singing. It's an independent project, not affiliated with xAI.

Restoring Japanese mojibake: brute-forcing encoding pairs with TextDecoder

ZHIWEI — Tue, 20 Oct 2026 05:48:57 +0000

Paste a Japanese email into the wrong decoder and こんにちは comes back as
縺薙ｓ縺ｫ縺｡縺ｯ. Save the same word to a file and open it in a Western
editor and you get æ—¥æœ¬èªž. Annoying? Yes. Hopeless? No — in the
overwhelming majority of cases the bytes are still there, untouched.
Mojibake is a decoding mistake, not corruption, and you can undo it with
nothing but the TextDecoder that is already built into your browser.

This post walks through the whole restoration pipeline: the encoding pairs
that actually matter, how to get the bytes back, what to do when
TextEncoder can't help you, how to rank the candidate readings, and the
two nasty edge cases — the lone 縺・ at the end of old emails, and text
that was garbled twice.

Bytes in, wrong interpretation out

Every case of classic mojibake follows the same pattern:

Text is saved as encoding A (bytes on disk / bytes in the email).
Something reads those bytes as encoding B.
The reader prints whatever characters those bytes mean in B.

Nothing destroyed the bytes. The UTF-8 bytes of こんにちは are
E3 81 93 E3 82 93 E3 81 AB E3 81 A1 E3 81 AF — fifteen bytes. Read as
Shift_JIS (1.5 bytes per kana character on average), they turn into eight
different, perfectly valid characters: 縺薙ｓ縺ｫ縺｡縺ｯ. The information
survived; only the interpretation is wrong.

So restoration is a search problem: recover the bytes, re-read them with
every plausible charset, and rank the readings by how much they look like
real Japanese.

The nine pairs that matter

Japan has more legacy encodings than anyone would like: UTF-8, Shift_JIS,
EUC-JP, UTF-16 — plus Windows-1252/Latin-1, the fallback every Western
tool reaches for. Going through who-saved-what and who-misread-it, nine
pairings cover practically everything you will meet in the wild:

saved as	misread as	typical scene
UTF-8	Shift_JIS	Excel or other Windows software opening a UTF-8 CSV
UTF-8	Windows-1252	non-Japanese-aware services and APIs
UTF-8	EUC-JP	old Unix environments, EUC-JP-era web pages
Shift_JIS	EUC-JP	Shift_JIS files rendered on EUC-JP systems
EUC-JP	Shift_JIS	old EUC-JP mail/pages viewed on Windows
Shift_JIS	Windows-1252	Western tools opening Shift_JIS files
EUC-JP	Windows-1252	same, for EUC-JP
UTF-16LE	Windows-1252	UTF-16 files read as 1-byte text
UTF-16BE	Windows-1252	same, byte-swapped

If you only ever memorize one row, make it the first one — a UTF-8 CSV
double-clicked open in Excel is probably the single most common way
Japanese text gets mangled on earth.

Step 1: getting the bytes back

When the text was misread through a single-byte charset such as
Windows-1252, the original bytes are trivially recoverable — every
character you see is one original byte:

const bytes = Uint8Array.from(
  [...garbled],
  (ch) => ch.charCodeAt(0) & 0xff,
);

Now throw every decoder at it. TextDecoder accepts all the labels we
need: utf-8, shift_jis, euc-jp, utf-16le, utf-16be, and
windows-1252:

const labels = [
  "utf-8", "shift_jis", "euc-jp",
  "utf-16le", "utf-16be", "windows-1252",
];
const candidates = labels.map((label) => {
  try {
    return new TextDecoder(label).decode(bytes);
  } catch {
    return ""; // invalid sequences for this encoding
  }
});

Step 2: the TextEncoder problem

The recipe above works when the reading was wrong. But in the Japanese
→ Japanese cases (saved as Shift_JIS, misread as EUC-JP — or the
reverse), restoration needs the opposite move: take the characters you
see and re-encode them back through the wrong charset to recover the
bytes.

And here the platform says no: TextEncoder only ever produces UTF-8.
There is no new TextEncoder("shift_jis").

The workaround is to build the reverse mapping yourself. Decode every
possible byte sequence with TextDecoder("shift_jis") once, and you
have a complete byte → character table for Shift_JIS; invert it and you
can map characters back to bytes. It sounds heavy — a 2-byte code point
has 65,536 combinations — but the table is built lazily and the whole
thing runs in milliseconds. This is also how you recover bytes from
Shift_JIS-style mojibake where several characters were merged, not just
re-mapped one-to-one.

Step 3: which candidate is right?

Brute force returns many "readable" results. The wrong ones tend to be
assembled from rare characters. 縺, 繧, 繝 — the first characters of
classic UTF-8-as-Shift_JIS garbage — are all JIS level-2 kanji,
characters that appear almost never in daily Japanese. The correct
reading, meanwhile, is made of level-1 characters and everyday kana.

That gives you a cheap, dependency-free ranking function: score each
candidate by how frequently its characters occur in normal Japanese
text, adding weight for common kanji and kana, subtracting for JIS
level-2 rarities. A frequency table built from a modest corpus is
enough; the right reading almost always floats to the top with a visible
margin.

The lone 縺・ at the end

Old Japanese mail software truncated messages at fixed byte lengths —
sometimes right in the middle of a multi-byte character. The victim is
always the last character: its leading byte survived, the continuation
bytes were cut. The result ends with something like 縺・ and every
decoder refuses it or prints a replacement mark.

These bytes can be resurrected. A cut multi-byte character leaves a
known fragment; enumerate the byte sequences that would continue it
(there are only a handful of legal continuations), re-decode, and pick
the completion that best fits the surrounding text. It is a guess in
the strict sense — but an extremely well-educated one, and in practice
it reads naturally in the overwhelming number of cases.

When text was garbled twice

Copy a mojibake sentence, save it in another tool, misread it again —
now you have double garbling, where the second wrong decoding was
applied to the output of the first. The fix is what you would expect:
run the whole pair search once, then run it again on the first
round's output. What you would not expect is the cost: the second
pass multiplies the work ninefold, so a sane implementation runs it
only on short texts — pasted sentences, never whole files.

Putting it together

The nine pairs, the byte-recovery tricks, naturalness ranking, the
trailing-character inference and the second pass all run in one page at
henkanya.com — free, no sign-up, and entirely
in your browser: the text you paste never leaves your machine. It also
handles the encodings that don't fit the brute-force model
(=?ISO-2022-JP?B?…?= mail headers, old JIS mail with lost ESC
sequences, Unicode escapes, HTML numeric references) and converts CSV
or text files to UTF-8 with BOM so that Excel opens them without
decorating your data with mojibake of its own.

If you want to see the pipeline fail — or better, save your afternoon —
paste your favorite garbage string and check whether the top candidate
is the sentence you remember.

How I Keep a Fan Guide in Sync With a Viral Browser Game Using a GitHub Actions Cron

ZHIWEI — Tue, 13 Oct 2026 05:54:04 +0000

A Nigerian browser game called Lagos Life launched at the start of October 2026 and went viral fast. I run an independent fan guide for it, and within a few days I hit the problem every guide writer for a live game hits: the game changes faster than I can edit pages.

Players elect a governor who sets a law that changes the economy. There's a new in-game headline every day, a daily gem hunt with a fresh clue, and the developers tweak job and business tables without notice. A hand-maintained page is stale within days.

So I stopped editing by hand. This post walks through the small pipeline that now does it for me: a GitHub Actions cron that reads what the official site already serves to every visitor, writes a few JSON files, and rebuilds a static site only when something actually changed.

The setup

The guide is a prerendered static site (TanStack Start) on Cloudflare Pages.
Pages read their data from JSON files committed in the repo (live-log.json, jobs.json, …) at build time.
One workflow runs every 3 hours:

fetch scripts ──▶ anything changed? ──no──▶ stop
                        │ yes
                        ▼
                 build ▶ deploy ▶ ping IndexNow ▶ commit the JSON

on:
  schedule:
    - cron: '17 */3 * * *'   # minute 17, not 0: avoid the top-of-hour rush
  workflow_dispatch:

concurrency:
  group: live-log
  cancel-in-progress: false  # never kill a run halfway through a deploy

Source 1: the public JSON the site already serves

The official site calls a few JSON endpoints for logged-out visitors: the current governor and law, the daily headline, and the gem hunt. Reading those is the easy part — a plain fetch with a timeout, a couple of retries, and a User-Agent that says who I am and where the data ends up:

const UA = 'my-guide live log (+https://example.com/updates/)';

async function get(path) {
  for (let i = 1; ; i++) {
    try {
      const res = await fetch(`${API}/${path}`, {
        headers: { 'User-Agent': UA, Accept: 'application/json' },
        signal: AbortSignal.timeout(20_000),
      });
      if (!res.ok) throw new Error(`HTTP ${res.status}`);
      return await res.json();
    } catch (e) {
      if (i >= 3) throw new Error(`${path}: ${e.message}`);
      await new Promise((r) => setTimeout(r, 5_000));
    }
  }
}

Two decisions mattered more than the code:

Store only what the page shows. The governor endpoint also returns election candidates and player-written announcements. I never write those to disk. The log keeps the law id, when it was set, the daily date/weather/headline/tip, and the hunt's clue — nothing else.
Append, don't overwrite. A new law creates a new entry with a firstSeen timestamp, so the page becomes a dated history instead of a snapshot. If only the term-end time moves, the last entry is updated in place.

The optional endpoint (the gem hunt) is wrapped in its own catch, so if it breaks, the law and headline log still runs.

Source 2: when there is no API — read the bundle

Job tables and pay rules aren't exposed as JSON; they live in the game's client JavaScript. The game is a Next.js app, so the homepage HTML references chunks under /_next/static/chunks/, and chunks reference more chunks.

The script does a small breadth-first crawl: start from the chunk names in the homepage, fetch six at a time, queue any new chunk names found inside, and stop as soon as every value it needs has been found (hard cap: 400 chunks).

Each value has its own finder, a function that returns null until it sees its pattern in some chunk:

const FINDERS = {
  careers:   (t) => (t.includes('"CAREERS",0,') ? true : null),
  starBonus: (t) => {
    const m = t.match(/"STAR_BONUS",0,([\d.]+)/);
    return m ? Number(m[1]) : null;
  },
  // ...one finder per rule constant
};

Minified variable names change on every build, so the patterns anchor on things that survive minification — string literals and object shapes like {id:"…",name:"…",levels:[{title:"…",pay:…}]} — never on short identifiers like e or a.

Guard rails: fail loudly, keep the last good data

A regex scraper against minified code will break one day. The goal is that when it breaks, the site stays correct and I get told.

Validate before writing. If fewer than 30 careers parse, or any career doesn't have five paid levels, or a workplace has no name, the script writes nothing and exits 1.
Isolate failures. Each fetch script is its own workflow step with continue-on-error: true, so a broken job-table parser never blocks the law log.
Two dates, not one. checkedAt moves on every good read; changedAt moves only when the table actually differs. The page shows both, so readers can tell "verified today" from "changed today".
Be polite. The bundle crawl runs at most once per Lagos day (WAT, UTC+1). If today's check already happened, it makes zero requests.

const WAT_MS = 60 * 60 * 1000;
const today = new Date(Date.now() + WAT_MS).toISOString().slice(0, 10);
if (data?.checkedAt === today) return; // already checked today

Commit only after a successful deploy

This one took me a while to get right. If the workflow committed the new JSON before deploying and the deploy then failed, the next run would see "no change" and the live site would stay stale forever.

So the order is: fetch → build → deploy → commit. The data file only lands in git once the site that shows it is live. A failed run leaves git untouched, and the next run three hours later simply tries again.

The build step also greps the generated sitemap.xml for the expected URLs before deploying — a cheap check that catches a build that "succeeded" but produced the wrong site.

Tell search engines only about pages that changed

After a deploy, the workflow pings IndexNow — but only for the pages whose content changed. A new check date alone rebuilds the jobs page without a ping; a new law pings only the updates page. Pinging unchanged URLs every three hours would just be noise.

paths=""
if [ "$LIVE" = true ]; then paths="/updates/"; fi
if [ "$JOBS_TABLE" = changed ]; then paths="${paths:+$paths,}/jobs/"; fi

What it produces

Since 8 October the log has been filling itself: each law with the date it was first seen, every daily headline, every gem-hunt clue (clues only — no answers). That's the updates page of the guide:

→ Lagos Life Updates: This Week's Law, Governor & Gem Hunt

The game itself is at lagoslife.app; my site is an independent fan guide, not affiliated with the developers, and everything it reads is what the official site already serves to any visitor.

Takeaways

For a fast-changing live game, a 3-hourly cron + static rebuild is plenty — no server, no database.
Anchor bundle-scraping patterns on string literals and shapes, and validate hard before writing.
Commit after deploy, so a failed run is automatically retried.
Store the minimum — especially anything player-written or personal.

If you've built something similar for another live game, I'd love to hear how you handle the parser breaking.