DEV Community: Peter's Lab

Fitting Translated Speech Back Into the Original Timeline: The Duration Problem in AI Dubbing

Peter's Lab — Fri, 06 Nov 2026 08:34:13 +0000

Everyone who builds an AI dubbing pipeline hits the same wall in week two. The ASR works, the translation reads fine, the TTS sounds like the speaker. Then you lay the synthesized German over the original video and it is still talking 1.3 seconds after the speaker’s mouth closed, and the next line starts on top of it.

This post is about that wall. Not the voice model, not the translation model, but the step between them that nobody writes about: taking a translated segment whose natural spoken duration is wrong and making it land where the original did. At AI Video Translator this step went through four rewrites before it stopped generating support tickets, and the final version is less clever than the first one.

https://ai-video-translator.com/

Why the durations are wrong in the first place

Translated text is a different length from the source. By character count, English to German typically expands 10 to 35 percent, Spanish 15 to 25, French 15 to 20, while Japanese and Chinese contract by a third or more. Speech rate then modulates this: Spanish and Japanese speakers produce around 7.8 syllables per second, English about 6.2, German about 6.0. So Spanish expansion partly cancels out, Japanese contraction almost fully cancels out, and German gets hit twice: more material and a slower rate.

The practical result: a 4.0 s English segment comes back from TTS as roughly 4.0 to 4.4 s in Japanese, 4.3 to 4.8 s in Spanish, and 4.8 to 5.6 s in German. Nothing about that is a defect. It is what the languages do.

The naive fix and why it fails

The first thing everyone does:

python
ratio = target_duration / synthesized_duration
audio = time_stretch(audio, ratio) # phase vocoder / WSOLA

Stretch every segment to its original slot. It works up to about a 10 percent compression. Between 10 and 15 percent listeners notice something is off even if they cannot say what. Beyond 15 percent the dub sounds like legal disclaimers at the end of a radio ad. German needs 20 to 40 percent on a bad line, so the naive fix produces unusable German while looking fine on Japanese test clips. If your test set is Japanese, you will ship this and find out from users.

What the final version does

The current approach is a budget-and-fallback chain. Each segment carries a timing budget derived from the original, and four strategies are tried in order of how much they damage the result.

Compute the real budget, not the segment duration

The original segment runs from t_start to t_end, but the speaker does not necessarily start talking again at t_end. There is usually silence after it: a breath, a cut, a pause before the next sentence. That silence is free time, as long as it belongs to the same speaker.

python
def budget(seg, next_seg, speaker_turn_end):
hard_end = min(next_seg.start, speaker_turn_end)
slack = hard_end - seg.end
return seg.duration + min(slack, MAX_SPILL) # MAX_SPILL ~ 0.6 s

Two details that mattered. First, cap the spill; a line that drifts a full second late feels wrong even if nothing collides with it, because the viewer sees the mouth stop. Second, the speaker-turn boundary is a hard wall. With one speaker you can borrow any pause. With two, the pause after speaker A’s line may be the lead-in to speaker B’s line, and borrowing it makes the two synthesized voices overlap where the originals never did.

Mild time stretch, bounded python MAX_STRETCH = 1.12 needed = synth_dur / budget if needed <= MAX_STRETCH: return time_stretch(audio, 1 / needed)

A 12 percent ceiling is conservative. It is also the point past which our own listening tests stopped agreeing on whether the voice was “fine”, so it is where the automatic path stops.

Re-translate against a length target

This is the strategy with the most headroom and the one most pipelines skip because it means the translation step and the timing step have to talk to each other.

When a segment exceeds the stretch ceiling, we call the translation model again with the same context plus a constraint: the target rendering should be at most N syllables (or characters, for languages where syllable counting is unreliable) and should preserve meaning over phrasing. N is derived from the budget and the language’s speech rate:

python
syllable_rate = {"de": 5.97, "en": 6.19, "es": 7.82, "fr": 7.18, "ja": 7.84}
max_syllables = int(budget * syllable_rate[lang] * 0.95)

The model is good at this. “We’re now going to take a look at how this works in practice” becomes “Now let’s see how this works.” The meaning survives and the line fits. We allow at most two re-translation attempts with tightening budgets; after that we stop, because the third attempt starts dropping information.

One subtlety: re-translating a single line in isolation can break consistency with the lines around it (a term rendered one way in the long version and another way in the short one). The glossary layer that pins fixed terms across the whole document also applies here, and the re-translation call gets the same surrounding-context window the first pass got.

Flag it for a human

If the line still does not fit after re-translation plus mild stretch, it gets marked in the review UI with the overrun amount, and the user can edit the text and re-voice just that line. The billing follows: re-voicing one line costs seconds of new speech, not a full render, because if fixing a line costs the same as the whole video nobody fixes lines.

The order is the point. Stretch is cheap and bounded. Re-translation is free in audio terms and has the most range. Human editing is the backstop. The first version of this pipeline did them in the wrong order (stretch as hard as needed, then give up) and the fourth version is the one above.

Measuring it

The metric that tracks user complaints best is not average overrun but the 95th percentile of synth_dur / budget per video, after all automatic strategies. A video with a p95 under 1.0 almost never gets a timing complaint. A video with p95 above 1.15 almost always does, even if the mean looks fine, because the viewer remembers the one line that spilled over the scene cut.

Per language on our content mix, after the full chain: Japanese and Mandarin need no re-translation on most segments; Spanish and French need it on roughly one segment in ten; German on roughly one in four. German remains the language where the human step gets used, and it is the language I would test first if I were evaluating anyone else’s dubbing tool.

Things I’d tell myself at the start
Build the budget from the next speech onset, not the segment end. Most of the free time is in the pauses.
Treat speaker boundaries as walls from day one, even if you only support one speaker today.
Length-constrained re-translation is the big lever. Wire the MT step so it can be called with a budget.
Don’t tune on Japanese. It is the easy case and it will lie to you.
Expose overruns to the user per line. They will fix the two lines that matter far faster than you can tune the model to.

The user-facing version of this, with the W3C and Lyon speech-rate numbers behind it, is here:

https://ai-video-translator.com/blog/why-ai-dubs-run-long

Keeping character names consistent across 1,000 manga pages: why I added a glossary to my AI translator

Peter's Lab — Fri, 30 Oct 2026 09:50:53 +0000

The most common complaint I got from users of my manga translation tool was not about translation quality on any single page. It was about page 190.

Someone would upload a whole volume, the first chapter would read fine, and then halfway through a side character would quietly change names. "Shiori" became "Siori". A guild called "the Black Hand" turned into "the Dark Hand". The honorific "-senpai" was kept on one page and dropped on the next. Each page in isolation was fine. The volume as a whole was not.

This post is about why that happens, why it is harder to fix than it sounds, and what I ended up shipping in AI Manga Translator to deal with it: a per-upload glossary with CSV import and export.

https://ai-manga-translator.com

Why names drift in the first place

A manga page goes through a pipeline: detect speech bubbles, work out reading order, OCR the Japanese, translate, inpaint the original text out, typeset the translation back in. The translation step is where names get decided.

The problem is that a name written in kanji or katakana often has several defensible romanizations, and a made-up term has several defensible translations. An LLM picks one. Ask it again with a slightly different context, and it may pick another. Over 200 pages you accumulate hundreds of these small decisions, and some fraction of them land differently.

Three things make it worse for manga specifically:

Pages are translated in batches, not as one giant prompt. A 1,000-page archive does not fit in a single context window at usable cost, so the model sees chunks. Each chunk can drift from the last.
Japanese drops the subject constantly. A line that is just a verb gets its subject inferred from context. If the model misreads who is speaking, it can also misattribute a name.
Reader expectations are strict. Prose readers might tolerate a one-off spelling variation. Manga readers follow series for years and have strong opinions about whether it is "Zoro" or "Zolo".

What I tried first

Carrying context forward. I pass a summary of recently seen names and terms from earlier chunks into later chunks. This helps a lot and is the main reason the tool already kept names stable for most volumes. But "most" is not "all", and it is a soft signal: the model is told what it used before, not what it must use.

Bigger chunks. Fewer boundaries means fewer chances to drift. It also means slower first results and higher cost per job, and it still does not solve the cross-volume case, where a user translates volume 3 today and volume 4 next month.

Post-processing find-and-replace. Tempting, but you cannot safely regex a name across typeset speech bubbles after the fact, and you do not know which variant the user wanted in the first place.

None of these address the real gap: the person who knows what the names should be is the user, and I was never asking them.

What shipped: a glossary per upload

The fix was boring, which is usually a good sign. Each upload can now carry a glossary: a list of fixed renderings, source term to target term, that the translation step must respect.

Design decisions, with the reasoning:

Per upload, not auto-saved. I considered a global glossary tied to the account. Users translate different series, and a global list would silently apply "Shiori" to a completely different character in another manga. So the glossary attaches to the job. To reuse it, you export it.
CSV import and export. Scanlation groups already keep terminology sheets in spreadsheets. Asking them to retype into a custom UI would have killed adoption. CSV means they paste what they already have, and the exported file travels to the next volume.
Hard cap of 100 entries. Most volumes need 10 to 30. The cap keeps the prompt budget predictable and discourages people from pasting an entire dictionary and wondering why quality dropped.
No extra page charge. Glossary entries add tokens to every translation call, so there is a real cost. I decided it should not be a line item, because the whole point is to make long-volume translation worth doing. The alternative, where a careful user pays more than a careless one, felt backwards.
Works with the style preview. Users can translate a few pages first, see which names appear and how they were rendered, then fill the glossary before running the full archive. Fifteen minutes of setup versus hunting for the wrong page afterwards.

How it fits in the pipeline

At a high level, the glossary is injected as a constraint into the translation step for every chunk, alongside the carried-forward context. The carried-forward context says "here is what you did before"; the glossary says "here is what you must do". When they conflict, the glossary wins.

The OCR step also benefits indirectly. Knowing that a specific katakana string is a name, not a word, lets the pipeline treat it as an atomic token instead of trying to translate it, which fixes a class of errors where a name that happens to look like a common noun gets translated literally.

I will not go into prompt specifics here, but the pattern is general enough that it applies to any batched LLM translation job: separate "what you did before" (soft, advisory) from "what the user requires" (hard, authoritative), and give the user a way to author the second one in a format they already use.

What I would tell someone building something similar

Consistency across batches is a product problem before it is a model problem. The best model still does not know which romanization your users want.
Let users bring their own terminology in the format they already have. For my audience that was CSV. For yours it might be a JSON file or a Notion export.
Put the glossary where the job is, not where the account is. Scope prevents a whole category of silent errors.
Pair it with a cheap preview step so people can discover what needs to go in the glossary before paying for the full run.

If you want to see the end result, the manga translator takes CBZ, CBR, PDF and EPUB archives, with the glossary and style preview available on every upload:

https://ai-manga-translator.com/tools/manga-translator

Happy to answer questions about the batching and context-carrying approach in the comments.

Why I Put a Human Review Step Before TTS in My AI Dubbing Pipeline

Peter's Lab — Wed, 21 Oct 2026 04:54:34 +0000

I run AI Video Translator as a solo dev. This week I shipped three features that all came from the same user complaint: "I only find out the translation is wrong after the dub is done."

The problem: errors surfaced at the most expensive step

The original pipeline ran end to end in one pass:

STT → translation → voice cloning / TTS → time-stretch + remix → output

Users heard the result, then noticed a name translated literally, a tone that didn't fit the video, or a term that came out differently in episode 3. The only fix was rerunning the whole dub.

The cost asymmetry was obvious once I looked at it. Editing a line of text is almost free. Regenerating speech is not. So the goal became simple: let users fix mistakes at the cheapest point in the pipeline.

The new pipeline

STT
 |
 v
Translation (glossary applied)
 |
 v
[NEW] Human review: line-by-line edit
 |
 v
Voice cloning / TTS (per speaker)
 |
 v
Time-stretch + remix
 |
 v
Dubbed video + SRT (burn-in optional)

1. A review step before synthesis

After translation, the script is shown line by line next to the source. Whatever the user edits becomes the input to TTS.

The main UX constraint is line length. Translated speech already tends to run longer than the source. If a user expands a line with extra explanation, it still has to fit the same time slot, so the voice speeds up. The rule I push in the UI: change the meaning if needed, keep the length close.

2. A glossary applied at translation time

Review fixes one video. Without a glossary, the same mistake comes back in the next one and users fix the same word again and again.

So users can now register source terms with a fixed target translation, applied during the translation step. It's the same idea as the glossary layer I built for character names in my manga translator (write-up here), except in video it's product names, technical terms, and channel-specific phrases. CSV import and export make it reusable across a series and shareable across a team.

The two features feed each other: any word you fix twice during review is a glossary candidate. The first episode takes real review time. By episode ten, review is a quick scan.

3. SRT as a separate output

Previously the only output was a video with subtitles burned in. Fixing one caption typo meant re-rendering the whole video.

Now every dub ships with a separate SRT file, and burn-in is optional. The SRT is built from the reviewed script, so edits land in both the audio and the captions. Creators can upload a clean video to YouTube and add the SRT in YouTube Studio, or upload captions separately to a course platform.

This sits on top of the multi-speaker detection I shipped a day earlier, where each speaker in the video keeps their own voice.

What I got wrong at the start

I originally pitched "fully automatic, from translation to dub" as the main selling point. But users dubbing long series and course content didn't want less control. They wanted a place to intervene. The automation stays; I just added a handle at the point where human input is cheapest.

I wrote a fuller, user-facing walkthrough of the workflow (including a training and course video use case) here: How to Review and Edit AI Dubbing Translations (and Keep Terminology Consistent)

Feedback welcome

Is line-by-line the right review granularity, or would grouping by paragraph or speaker be better?
For per-series glossaries, is CSV good enough, or is there a better format for teams?

Thanks to everyone who asked for these.

Why Vertical Japanese Text Breaks Standard OCR (And How We Engineered an Inpainting Pipeline in Next.js)

Peter's Lab — Tue, 13 Oct 2026 08:39:27 +0000

Reading raw Japanese manga should be seamless in 2026. Yet millions of international readers still hit the same wall: paste a comic panel into standard OCR tools or generic browser extensions, and you get fragmented nonsense covered by ugly white rectangular patches.

When building AI Manga Translator (https://ai-manga-translator.com), our biggest bottleneck was not prompting translation LLMs. It was the complete failure rate of conventional computer vision when confronted with tategaki (vertical text), non-convex speech bubbles, and half-tone screentones.

Here is an architectural breakdown of why generic OCR pipelines break on comics, and how we engineered an in-browser detection and inpainting pipeline using Next.js, SegFormer, and custom client-side heuristics.

The Geometry Breakdown: Horizontal Anchor Bias vs. Tategaki

Standard OCR engines (Tesseract, PaddleOCR, Apple Vision) are tuned on standard horizontal Latin and CJK documents such as invoices, receipts, and books. They rely on horizontal projection profiles to detect text baselines.

Manga breaks every baseline assumption:

• Top-to-Bottom, Right-to-Left (Tategaki): Columns read right-to-left, but glyphs flow top-to-bottom. Horizontal clustering models frequently link glyphs across adjacent columns, merging two distinct thoughts into garbled output.
• Furigana Collisions: Japanese manga frequently prints ruby text (small phonetic furigana) right beside complex kanji. Generic bounding boxes merge furigana into the main glyph, destroying character recognition accuracy.
• Complex Non-Convex Polygons: Dialogue balloons vary from ellipses and rounded rectangles to spiked shock-bursts. Applying axis-aligned bounding boxes (AABB) captures halftone background textures, speed lines, and character hair, triggering severe false positives.

To solve the column-ordering failure mode, we implemented a custom heuristic to cluster vertical text boxes along their true semantic flow:

interface DetectedBox {
x: number;
y: number;
width: number;
height: number;
text: string;
}

export function clusterVerticalColumns(boxes: DetectedBox[], lineThreshold = 18): DetectedBox[][] {
const sorted = [...boxes].sort((a, b) => b.x - a.x);
const columns: DetectedBox[][] = [];

for (const box of sorted) {
const matchedCol = columns.find(col =>
Math.abs(col[col.length - 1].x - box.x) < lineThreshold
);

if (matchedCol) {
matchedCol.push(box);
matchedCol.sort((a, b) => a.y - b.y);
} else {
columns.push([box]);
}
}

return columns;
}

Instead of rigid bounding boxes, we deploy a comic-fine-tuned semantic segmentation backbone based on SegFormer. It outputs pixel-level probability masks separating speech bubble contours, in-bubble vertical text strokes, and sound effects (SFX) alongside ambient text overlays.

Replacing White Rectangles with Inpainting (LaMa Architecture)

The most notorious flaw of browser translation extensions is the "white patch." Standard tools draw an opaque rectangular SVG or div over the bubble, completely destroying hand-drawn screentones, line art, and atmospheric dark backgrounds.

To achieve clean scans, we treat text replacement as an image restoration problem across three explicit passes:

• Stroke Dilation Masking: Instead of blanking the whole bubble, our model generates a tight mask covering only the black ink strokes of the Japanese characters, expanded by a 2px morphological dilation to catch anti-aliasing artifacts.
• Fast Inpainting Pass: We run an optimized resolution-robust inpainting model based on Large Mask Inpainting (LaMa with fast FFT convolutions). It predicts underlying textures, such as halftone screentones or paper grain, without washing out adjacent bubble boundaries.
• Adaptive Canvas Typesetting: Using the HTML5 Canvas API on the client side, the translated target text is dynamically wrapped into the bubble's bounding polygon, automatically balancing font sizing, line height, and hyphenation to prevent text overflowing the bubble boundary.

Edge Pipeline & Performance: Beating the GPU Latency Wall

A 25-page chapter can easily choke backend inference if you stream raw 4K scans. We built an edge-optimized workflow inside our Next.js architecture to keep processing times under sub-second thresholds:

• Client-Side Offloading: For the AI Manga Translator Chrome Extension (https://ai-manga-translator.com/extension), the browser inspects DOM/Canvas elements directly on sites like Pixiv, BookWalker, and DLsite, extracting only dynamic canvas viewports rather than dumping massive full-page payloads.
• Concurrent Chunk Dispatching: Extracted regions are processed via asynchronous micro-batches. While bubble #1 is getting inpainted, bubble #2 is already streaming translation from our fine-tuned comic LLM endpoint.
• SSR Fallback with Next.js 14: On the web platform, we use Next.js server actions to handle parallel file streaming, allowing users to drop raw CBZ, ZIP, or PDF files and immediately inspect real-time progress via Server-Sent Events (SSE).

The Architecture in Action

Eliminating the friction between reading and understanding shouldn't mean turning art into flat, robotic documents.

By treating comic pages as multi-layered graphic compositions rather than flat text files, we preserve the illustrator's line work while rendering native-feeling translations in real time.

If you are curious to see how the segmentation and inpainting hold up on complex raw panels, you can test AI Manga Translator (https://ai-manga-translator.com) directly on the web with 5 free pages—no account sign-up required.

What techniques are you using to handle non-standard text directions in your vision pipelines? Drop your thoughts and benchmarks below.

Keeping Character Names Consistent Across 200 Chapters: A Glossary Layer for LLM Translation

Peter's Lab — Wed, 07 Oct 2026 14:17:59 +0000

LLMs are surprisingly good at translating a single page of dialogue. Tone, slang, implied subjects, they handle a lot of it well.

The problem shows up later. Translate chapter 1 and chapter 40 separately, and the same character can come out with two different spellings. A sect name gets transliterated in one chapter and translated literally in the next. A nickname quietly turns into the character's full name halfway through the series.

Each page looks fine on its own. Read them in sequence and the story falls apart.

This post is about a simple fix that works well in practice: a small glossary layer that sits between your text and the model.

Why this happens

Most translation pipelines process one page or one chapter at a time. The model has no memory of what it decided before, so every page is a fresh guess.

Names are especially fragile because:

Japanese names have multiple readings. The same kanji can be read several ways, and the model has to guess.
Romanization varies. Shō, Sho, Shou and Shoh are all "correct" for the same name.
Chinese and Korean names often look like ordinary words. A name made of characters meaning "forest" and "wind" can get translated literally.
Order varies. Family name first or last? The model may switch depending on context.

Giving the model the whole series as context isn't realistic. What it actually needs is the decisions, not the text.

The glossary

A glossary is just a list of source terms and how they should be rendered. Keep it boring and explicit:

python
from dataclasses import dataclass, field

@dataclass
class Term:
source: str # as it appears in the original text
target: str # how it should appear in the translation
kind: str = "name" # name, place, technique, title, ...
note: str = "" # optional hint for the model
aliases: list[str] = field(default_factory=list)

glossary = [
Term("田中", "Tanaka", note="family name, keep family-name-first"),
Term("翔", "Shou", note="given name, use 'Shou' not 'Sho' or 'Shō'"),
Term("林风", "Lin Feng", note="person's name, do not translate the meaning"),
Term("九天雷掌", "Nine Heavens Thunder Palm", kind="technique"),
]

The note field matters more than it looks. "Do not translate the meaning" or "keep the honorific" gives the model the why, which helps it handle variations it hasn't seen before.

Only send what's relevant

You don't want to stuff a 500-entry glossary into every prompt. It costs tokens, and long irrelevant lists can actually make the model less reliable.

Instead, scan the source text and include only the terms that appear in this chunk:

python
def relevant_terms(text: str, glossary: list[Term]) -> list[Term]:
hits = []
for term in glossary:
if term.source in text or any(a in text for a in term.aliases):
hits.append(term)
return hits

For Japanese and Chinese, plain substring matching works surprisingly well because there are no spaces to worry about. For languages with inflection or particles attached to names, you may need a light normalization step first.

One catch: short terms can produce false matches inside longer words. Sorting terms by length and matching longest first reduces that.

Put it in the prompt as a hard constraint

Format the matched terms as a clear table and tell the model they are non-negotiable:

python
def build_prompt(text: str, terms: list[Term], target_lang: str) -> str:
lines = [f"- {t.source} → {t.target}" + (f" ({t.note})" if t.note else "")
for t in terms]
glossary_block = "\n".join(lines) if lines else "(none)"

return f"""Translate the following manga dialogue into {target_lang}.

Use these renderings exactly. Do not change spelling, order or form:
{glossary_block}

Dialogue:
{text}
"""

Short and explicit beats clever. Models follow a small, clearly formatted list far more reliably than a paragraph of instructions.

Check the output anyway

Even with the glossary in the prompt, models occasionally drift. A cheap post-check catches most of it:

python
def missing_terms(source: str, translation: str, terms: list[Term]) -> list[Term]:
"""Terms that appear in the source but whose target is missing from the output."""
return [t for t in terms
if t.source in source and t.target.lower() not in translation.lower()]

If something is missing, you have options: retry with a stronger instruction, flag the line for review, or log it so you can see which terms cause trouble.

This check isn't perfect. A translation might legitimately use a pronoun instead of repeating a name. But as a signal, it's very useful, and it costs almost nothing.

Growing the glossary over time

The interesting part is how the glossary gets built. A few approaches that work:

Seed it manually for the main cast. Ten to twenty entries cover most of the dialogue in a typical series.
Ask the model to extract candidates. After translating a chapter, ask it to list proper nouns it encountered and how it rendered them. New entries go to a review queue rather than straight into the glossary.
Promote on repetition. A name that appears in three or more chapters is worth locking in.
Let humans override. If a reader or translator corrects a name, that correction should win everywhere, retroactively if possible.

Scanlation teams have done this by hand for years: a shared document of every name, term and spelling decision for the series. The glossary layer is really just that practice, made machine-readable.

What it doesn't solve
Readings that only appear once. If a name's reading is given only in a tiny furigana on the first appearance and OCR misses it, the glossary starts with a wrong entry. Garbage in, consistent garbage out.
Context-dependent rendering. Sometimes a character is called by a nickname in casual scenes and by their full name in formal ones. That's a style decision, and a flat glossary can flatten it.
Honorifics. Whether "-san" or "-kun" stays attached to a name is a series-wide convention, best handled as a separate rule rather than per-name entries.

Wrapping up

The core idea is simple: LLMs don't need your whole series as context. They need a short, explicit list of the decisions you've already made, scoped to the text in front of them, plus a cheap check that they actually followed it.

It's a small piece of code, but in long-running translation work, consistency is often what separates something readable from something that feels machine-made. This is one of the problems I keep coming back to while working on AI Manga Translator.

Beyond General Vision: How We Scaled Real-Time Manga Localization for Indie Hackers

Peter's Lab — Mon, 21 Sep 2026 07:31:26 +0000

Most general vision models fail miserably when pointed at a manga page. They get confused by the non-linear reading flow, misinterpret vertical typography, and butcher the typesetting inside speech bubbles.

If you are building media localization tools using standard OCR and off-the-shelf LLMs, you are likely burning API credits on broken formatting. We ran into the same brick wall while building our platform—so here is how we engineered a robust, production-ready pipeline that actually works.

The Bottleneck of Standard Vision Pipelines
When processing raw manga scans, standard multi-modal models stumble upon three classic roadblocks:

The Panel Flow Trap: Reading order in manga flows from right-to-left and top-to-bottom. General models default to Western left-to-right parsing, completely scrambling the narrative context.

Bubble Segmentation Noise: Sound effects (SFX) overlapping with character art often get misclassified as dialogue text, leading to messy translation artifacts.

Typesetting Nightmares: Translating text back into speech bubbles often results in overflow because characters require different aspect ratios and bounding box constraints depending on the target language.

Our Engineered Solution
To bypass these hurdles, we decoupled the detection and translation phases using a streamlined Next.js 14 backend paired with a custom spatial-aware parser:

TypeScript
// Simplified pipeline snippet for bounding box validation
async function processMangaPanel(imageBuffer: Buffer) {
const detectedBubbles = await spatialModel.detectBubbles(imageBuffer);
const sortedBubbles = sortMangaReadingOrder(detectedBubbles);

return await translateAndTypeset(sortedBubbles);
}
By enforcing strict right-to-left sorting algorithms before sending text chunks to the translation engine, we reduced structural errors by over 40%.

You can check out how we implemented this in production over at AI Manga Translator—our live tool dedicated to solving raw comic translation for creators and readers worldwide.

Scale and Ship Fast
Building cool tech is only half the battle; distribution is where indie revenue happens. Once your article goes live on Dev.to, ensure you execute these deployment steps:

Google Search Console (GSC): Submit your new post URL for immediate indexing.

Sitemap Ping: Ensure your publication or portfolio sitemap is up to date.

Backlink Distribution: Share the snippet on Reddit (r/IndieHackers, r/LocalLLaMA) and X to drive initial velocity.

Keep shipping, stay resilient, and let’s keep pushing forward!

How We Engineered an AI Manga Translator: Solving RTL OCR, Screentone Inpainting, and Edge Pipelines

Peter's Lab — Wed, 16 Sep 2026 09:06:33 +0000

Building a production-ready manga translation pipeline means fighting vertical text, preserving fragile halftone screentones, and optimizing heavy image processing at the edge. Here is our architectural breakdown.

Building a translation tool sounds simple on paper: extract text, pass it to an LLM, translate it, and render it back onto the image. But the moment you point a standard OCR and translation pipeline at a raw Japanese manga chapter, everything breaks.

Manga is not a standard document, and it is not a webtoon. It is a complex visual medium featuring vertical typography, right-to-left (RTL) reading flows, and intricate halftone dot patterns known as screentones.

Curious to see it in action? You can test the web app directly over at the AI Manga Translator or install the Chrome Extension to read raw chapters seamlessly right in your browser.

Here is a technical look at the architectural challenges we faced while building this pipeline and how we solved them.

1. The OCR & Layout Nightmare: RTL vs. LTR
Standard OCR pipelines (like basic Tesseract configurations or default cloud vision APIs) assume a Left-to-Right (LTR) horizontal flow. When applied to manga, they fail catastrophically:

The Reading Order Crisis: Speech bubbles and text blocks must be parsed strictly from right to left, top to bottom. If your bounding box clustering algorithm assumes LTR, the dialogue sequence gets completely scrambled, turning a dramatic conversation into gibberish.

Vertical Text (Tategaki): Japanese characters are frequently written vertically. Standard OCR engines often misinterpret vertical glyph bounding boxes as independent horizontal tokens.

Our Fix: We implemented a custom layout-awareness parser wrapper that clusters speech bubbles based on spatial proximity while prioritizing right-to-left coordinate sorting before sending token payloads downstream.

2. The Inpainting Challenge: Protecting Screentones
Once you extract and translate the Japanese text, you have to clear the original text box. This is where standard image editing algorithms fail.

If you use naive pixel-filling or aggressive blurring to erase text, you destroy the screentones—the delicate dot patterns used by mangakas to create shading and gradients. A bad inpainting pass leaves ugly white smudges or muddy grey patches over the artwork.

Our Fix: We built a context-aware inpainting pipeline optimized for comic book line art. By isolating text bounding masks from the underlying vector/raster grid, the model repairs only the pixels occupied by the glyphs while preserving the surrounding halftone frequency and character linework.

3. Edge-Optimized Performance and Frontend Rendering
Manga pages are heavy assets. Processing multi-page chapters via slow server-side Python monoliths introduces massive latency, killing the user experience during rapid chapter binge-reading.

To keep the application fast and responsive, our tech stack relies on a high-performance architecture:

Next.js & Edge Runtime: Utilizing Vercel Edge routing to handle lightweight proxying and session states instantly.

Client-Server Hybrid Pipelines: Offloading initial asset preprocessing and utilizing optimized API endpoints to process image slices concurrently without choking bandwidth.

Final Thoughts
Building vertical SaaS tools in the indie hacking space forces you to solve hyper-specific engineering problems that generic APIs ignore. By treating manga translation as an advanced computer vision and layout problem rather than a simple text translation wrapper, we built a pipeline that respects the art as much as the language.

How I Built a Programmatic SEO Architecture with Next.js 14 and SSR for AI Apps

Peter's Lab — Mon, 10 Aug 2026 02:36:12 +0000

A practical breakdown of combining SSR, Next.js 14 App Router, and dynamic intent pages to drive organic traffic while managing AI API overhead.
When building an AI application, driving initial user acquisition without a massive ad budget comes down to one critical engineering decision: how you handle your dynamic landing pages for search engines.

While building PersonalizedSong.ai—a platform that transforms personal stories and shared memories into custom, studio-quality tracks—I knew organic search was going to be our main growth engine.

However, user intent in the gifting space is fragmented:

custom birthday song

anniversary gifts for wife

personalized song for husband

Instead of manually creating dozens of static routes, I architected a dynamic, Programmatic SEO (pSEO) system built on Next.js 14. Here is how it works under the hood and why SSR was non-negotiable.

1. Why Client-Side Rendering (CSR) Fails for Intent-Based AI SaaS
Many modern AI wrappers are built as heavy Single Page Applications (SPAs) rendered purely on the client side. While this keeps initial development fast, it creates a massive SEO disadvantage:

Hydration Delays for Crawlers: Even though search bots have gotten better at executing JavaScript, reliant CSR routes often delay rendering key structured data.

Dynamic Meta Tags: Social previews (OpenGraph) and dynamic JSON-LD schemas need to be served instantly in the raw HTML payload for proper indexing and social sharing.

By leveraging Next.js 14 App Router with Server-Side Rendering (SSR), every dynamic intent page evaluates the incoming request, fetches the corresponding metadata, and returns fully-rendered HTML straight to the edge.

2. The Dynamic Route Architecture
Using Next.js 14, dynamic routes allow us to serve thousands of long-tail keyword combinations through unified dynamic pages:

TypeScript
// app/[occasion]/page.tsx
import { Metadata } from 'next';
import { getOccasionData } from '@/lib/seo-data';

type Props = {
params: { occasion: string };
};

export async function generateMetadata({ params }: Props): Promise {
const data = await getOccasionData(params.occasion);

return {
title: ${data.title} | Personalized Song Generator,
description: data.description,
openGraph: {
title: data.ogTitle,
images: [data.ogImage],
},
};
}

export default async function OccasionPage({ params }: Props) {
const data = await getOccasionData(params.occasion);

return (

{data.heading}

{data.subheading}

{/* Interactive Creation Form Component */}

);
}

3. Balancing SSR Performance with API Costs
In a typical AI app, you don't want your server rendered pages hitting expensive AI LLMs or music generation endpoints on every single request or bot crawl.

The architecture separates Intent & Pre-rendering from Audio Generation:

SSR Layer: Delivers lightweight, ultra-fast pre-rendered HTML containing structural context, custom prompts, schema markup, and UI controls.

Client/API Layer: The actual AI audio generation is triggered only upon user interaction (answering our 4 prompt questions).

This separation keeps server response times sub-second while protecting server logs and API keys from automated crawlers.

Takeaways for Web Devs
Treat SEO as Architecture: Don't bolt on SEO as an afterthought. Structure your App Router layouts and page hierarchies from day one.

Server-Side Render Intent Pages: Give crawlers pure HTML containing semantic tags and JSON-LD schemas.

Decouple Dynamic Pages from Heavy AI APIs: Serve fast SSR pages first; trigger AI tasks strictly via authenticated client events.

I’d love to hear how other full-stack devs are structuring their dynamic routes and handling SSR caching in Next.js 14! Check out PersonalizedSong.ai to see the frontend in action, and drop your questions in the comments below.

How to Build Interactive "Tap and Hold" Twitter Images (And Why They Drive Engagement)

Peter's Lab — Sat, 25 Jul 2026 17:27:05 +0000

Have you ever stumbled across a tweet where the thumbnail shows one thing, but as soon as you tap and hold on mobile (or open the full media viewer), a completely different image or secret message appears?

It’s one of the most effective interactive media tricks on Twitter/X right now. Whether you are building an indie project, running a viral marketing campaign, or launching an interactive puzzle, "tap and hold" images spark instant curiosity and massively boost user engagement.

In this post, we’ll break down the technical mechanics behind how tap-and-hold images work on Twitter's rendering engine and how you can implement or generate them easily.

The Tech Behind Twitter’s "Tap and Hold" Magic
At first glance, it might look like Twitter supports native multi-layered image toggling. In reality, it relies on how Twitter's mobile feed renders transparent PNGs against its dark/light mode background and how the full-screen media viewer handles image compositing.

Here is what happens under the hood:

Alpha Channel & Transparency Rules: Twitter’s feed preview compresses and flattens image previews over a default background color (often white or light grey depending on the theme).

Foreground vs. Background Contrast: By carefully manipulating pixel transparency, color channels, and alpha values, you can craft an image where:

State A (Feed View): Low-contrast alpha pixels blend into the feed background, making only Layer 1 visible.

State B (Tap & Hold / Viewer View): When a user taps and holds the image, Twitter opens the media lightbox with a dark/black backdrop. This shift in backdrop illumination instantly reveals Layer 2 while making Layer 1 blend into the dark background.

Understanding color matrix transformations and PNG alpha channel encoding is essential if you are constructing this manually via canvas or WebGL.

How to Create Tap and Hold Images Programmatically or via Web Tools
If you want to build this into your own pipeline or quickly generate one for your next product launch, you don't need to manually spend hours tweaking alpha levels in Photoshop.

You can check out our dedicated technical walkthrough and generator here:
👉 How to Make Tap and Hold Images Guide

If you want to convert your images right away without writing custom image processing scripts, you can use the main tool at TapAndHold.com.

Basic Concept (JavaScript / HTML5 Canvas Idea)
If you're building a custom client-side generator using HTML5 Canvas, the high-level steps look like this:

JavaScript
// Conceptual snippet for blending two images into a tap-and-hold PNG
const canvas = document.createElement('canvas');
const ctx = canvas.getContext('2d');

// 1. Load preview image (Image A) and hidden image (Image B)
// 2. Extract ImageData arrays for both images
// 3. Calculate alpha values pixel-by-pixel so Image A shows on bright backgrounds
// and Image B reveals on dark backgrounds
// 4. Export as a uncompressed PNG to preserve precise alpha thresholds
Why Indie Hackers & Creators Are Using Tap and Hold Media
Higher Engagement Rates: Interactive tweets require active participation from the user, signaling positive engagement signals to Twitter’s recommendation algorithm.

Easter Eggs & Product Teasers: Perfect for showing "Before / After" code refactoring, revealing hidden discount codes, or teasing unreleased feature updates.

Meme Culture: It fits seamlessly into modern social media mechanics where curiosity drives clicks.

Try It Out
If you're launching a product on X soon, give this format a try to see how your audience responds.

Full Guide: How to Make Tap and Hold Images

Main Tool: TapAndHold.com

Have you experimented with alpha-channel image tricks on Twitter or other social platforms before? Let’s chat in the comments!

How "Tap and Hold" Hidden Images Work on X (and How I Built a Free Client-Side Generator)

Peter's Lab — Sat, 25 Jul 2026 06:37:09 +0000

If you have spent any time on X (Twitter) recently—especially in tech or anime communities—you have likely seen the viral "Tap and Hold" (長押しで変化) image trend.

You see a preview image on your feed, but when you tap and hold it to view it in full high-resolution, the background or hidden details suddenly change.

As a full-stack developer, my first thought was: How does this actually work under the hood? And how can we make it effortless for anyone to build one?

Here is a quick breakdown of the technology behind it, and how I built a 100% client-side tool to generate these images instantly.

The Tech Behind "Tap & Hold" Images

At its core, this effect relies on how social media platforms (like X) render PNG transparency and alpha channels differently between the inline feed preview and the full-screen modal view.

Alpha Channel & Background Rendering: X renders image previews with a white/light background in the timeline grid, but switches to a dark modal background when long-pressed or opened.

Layering Tricks: By strategically adjusting pixel opacity (Alpha) and color values across specific regions of a PNG, you can make elements invisible against the default background, only revealing themselves when the dark overlay kicks in.

While you could manually tweak layers in Photoshop or Figma, it is tedious and time-consuming.

Building a 100% Browser-Based Generator

I wanted a solution that was:

Fast: Instant preview with zero server roundtrips.

Private: Your images should never leave your machine.

Free: No forced logins, paywalls, or ugly watermarks.

I built Tap and Hold Image Maker (https://tapandhold.com/) using modern Web APIs and Next.js 14.

Key Implementation Highlights:

HTML5 Canvas Processing: All image manipulation happens directly on the client side using the Canvas API (ctx.getImageData() and ctx.putImageData()).

Zero Server Overhead: Because rendering happens in the user's browser, there are zero API costs or privacy risks regarding uploaded media.

Real-Time Preview: You can toggle between timeline view and long-press view instantly before exporting.

Try It Out

If you want to create interactive images for your X posts or inspect how the preview works:

Tool: https://tapandhold.com/

How to use:

Upload your image.

Select what stays hidden during normal viewing.

Export your PNG and post it directly to X!

I would love to hear your thoughts on client-side canvas optimization or feedback on the UI! What features should I add next?

Built by Peter Anderson | Indie Hacker & Full-Stack Developer

Stop Manual Sculpting: Accelerating Game Dev Pipelines with AI 3D Mesh Generators published: true

Peter's Lab — Wed, 08 Jul 2026 17:26:36 +0000

Let's be honest: 3D modeling is an absolute time sink. If you're an indie game developer, a solo creator, or a technical artist working inside Blender, Unity, or Unreal Engine, spending 4 to 5 hours just topology-mapping a single background prop, building asset, or character draft can completely destroy your shipping momentum.

As a developer, my philosophy has always been to automate the repetitive grunt work so we can focus on building actual value. That’s exactly why I engineered an optimized AI 3D mesh generator to handle the initial geometric heavy lifting for you in under 60 seconds.

Breaking the Asset Pipeline Bottleneck

The standard pipeline from 2D concept art to a game-ready asset usually requires a series of friction-heavy steps: sculpting a high-poly draft, manually retopologizing it to maintain acceptable performance, and packing/baking textures.

By utilizing a next-generation depth-extraction architecture, this web-based pipeline allows you to drop any 2D sketch, concept artwork, or character photo straight into the browser and instantly output clean, structured geometry.

Why Clean Topology Matters for Production

Many early-stage generative 3D tools suffered from a critical flaw: they spit out messy, unoptimized "high-poly junk" that causes extreme vertex stretching and clipping inside game engines. For a tool to be truly production-ready, it must focus heavily on clean topology:

Engine-Ready Geometry: Generates optimized topology designed to prevent clipping and polycount bloat out of the box.
Universal Formats: Download your generated assets as .glb, .obj, or .fbx and drag them straight into your scene.
Zero Scanner Required: No specialized photogrammetry rigs or hardware needed—just standard flat images.

Speeding Up the Feedback Loop

This isn’t about replacing 3D artists; it’s about speeding up the feedback loop for solo makers and small dev teams. Instead of spending days prototyping a level layout, you can generate 3D assets on the fly, test scale and mechanics immediately, and refine the artistic direction faster.

Stop building every standard background asset the hard way. Check out the automated pipeline, test your own raw concept sketches, and significantly accelerate your development cycle today.

try it out yourself: https://image-to-3d-model.com/

How AI Video Translation Is Changing Global Content Creation in 2026

Peter's Lab — Sun, 21 Jun 2026 11:23:19 +0000

Video content is becoming the dominant format on the internet.

But there's still one big problem:

Language barriers.

A creator might produce a great video in English, but lose 80% of potential audience because it doesn't reach Spanish, Japanese, Korean, or other regions.

Traditionally, video localization required:

Manual dubbing
Subtitle editing
Voice actors
High production cost
Long turnaround time

This made global content scaling extremely slow.

In 2026, AI video translation is changing this workflow completely.

The Old Way of Translating Videos

Before AI tools, video translation looked like this:

Transcribe audio manually
Translate script line by line
Hire voice actors for dubbing
Sync subtitles with timestamps
Edit and re-render video

This process could take days or even weeks for a single video.

And it didn't scale.

For creators, marketers, and educators, this was a major bottleneck.

What AI Video Translation Actually Does

Modern AI video translation tools automate the entire pipeline:

Speech recognition (ASR)
Translation (NLP)
Voice cloning
Lip-sync adjustment
Subtitle generation

Instead of multiple manual steps, everything happens in one workflow.

The result:

One video → multiple languages in minutes

Why This Matters for Creators and Businesses

AI video translation is not just a convenience tool.

It changes distribution strategy.

Before:
One video = one language audience
Now:
One video = global audience

This means:

YouTube creators can expand globally
Startups can localize product demos
Educators can scale courses internationally
Marketers can run multilingual ad campaigns

The bottleneck is no longer production.

It is only content quality.

Key Features Modern AI Video Translators Offer

Most advanced tools now include:

Automatic speech translation
Voice cloning (preserving original tone)
Lip-sync correction
Multi-language subtitle generation
Fast rendering without manual editing

This removes the need for traditional dubbing studios entirely.

Example Workflow Using AI Video Translator

A typical workflow looks like this:

Upload your video
Select target languages
AI processes audio and visual alignment
Generate translated versions
Export ready-to-publish videos

What used to take days now takes minutes.

Use Cases

YouTube Creators

Reach new audiences without re-recording content.

Marketing Teams

Translate ads for different regions instantly.

Educators

Turn one course into global learning content.

SaaS Companies

Localize product walkthroughs for international users.

Why AI Video Translation Is Growing Fast

Three major trends are driving adoption:

1. Global Content Demand

Audiences expect content in their own language.

2. Short-Form Video Explosion

TikTok, Reels, and Shorts require fast scaling.

3. AI Voice Cloning Quality Improvement

AI voices now sound natural enough for production use.

Tools That Enable This Workflow

One example of this new generation of tools is:

👉 https://ai-video-translator.com/

It allows creators to translate videos into multiple languages using AI-driven dubbing and subtitle generation.

The goal is simple:

Remove language barriers from video content creation.

Limitations (Important)

AI video translation is powerful, but not perfect:

Cultural nuance may be lost
Some accents may sound unnatural
Emotion-heavy speech still needs human review
Brand-critical videos may require editing

Best results come from combining AI + light human correction.

We are moving toward a world where:

Content is created once, and distributed everywhere.

AI video translation is one of the key technologies enabling this shift.

For creators and businesses, the advantage is clear:

Faster localization
Lower cost
Wider reach
Global scalability

And this is still just the beginning.

Tool Mention

If you're exploring AI video translation workflows:

https://ai-video-translator.com/