DEV Community: qcrao

The Engineering Challenge of Turning YouTube Into an ESL Corpus

qcrao — Fri, 24 Apr 2026 03:34:47 +0000

Language learning apps have spent a decade chasing the same pattern: curate a 2,000-word "high-frequency vocabulary" list, wrap it in spaced repetition, ship. Users grind, retention looks great in the app, and then they meet an actual English speaker and freeze, because recognizing a word on a flashcard is not the same skill as catching it in running speech. The information is in their head but it is not wired to sound, pace, register, or context.

The intuition behind context-based acquisition — learning words in situ, inside real discourse — is old and well supported in second-language acquisition research. The problem has always been that the "real discourse" part is hard to deliver at scale. Textbook dialogues are not real. Classroom tapes are not real. Even podcasts are a curated subset.

YouTube is real. It is also the single largest corpus of native-speaker content in every register you care about: casual vlogs, lectures, interviews, comedy, news, gameplay commentary, technical talks. For ESL specifically, the fact that speakers vary in accent, speed, and slang is a feature, not a bug.

The engineering question is: what would it take to turn YouTube into a usable ESL corpus?

The interactivity problem

Watching YouTube with auto-subtitles on is already useful for listening comprehension. The gap is that subtitles are read-only. A learner hits an unfamiliar word, pauses the video, tabs to a dictionary, types the word, gets a translation, tabs back, loses their place. After three such interruptions in a 10-minute video most learners give up and either:

stop pausing (and therefore stop learning from the unfamiliar words), or
abandon the video entirely.

The right interaction is click-a-word → instant translation + pronunciation + example sentence → optionally save as flashcard, all without leaving the player. That turns a 10-minute video into a vocab-building session instead of a comprehension test.

Why this is harder than it looks

A few things get in the way:

Subtitle alignment. YouTube auto-subs are word-timed for about 80% of videos; manual subs are sentence-timed. A click-a-word UI has to handle both gracefully, ideally highlighting the clicked word with <50ms latency.
Tokenization across languages. Clicking "running" should map to the lemma "run" for dictionary lookup. Clicking "auf" in a German phrase should resolve to the correct sense given context. Clicking "不好意思" in Chinese should resolve as a multi-character idiom, not char-by-char.
Disambiguation. "Bank" in a finance video is different from "bank" in a kayaking video. A naive dictionary lookup gives the most common sense; a better system checks surrounding context.
Personalization. A B2 learner does not want to be interrupted every time "the" appears. The system needs to model what the learner already knows and surface only likely-unknown words — ideally inferred from past clicks, not a placement test.
Flashcard hygiene. Saving raw dictionary entries produces terrible flashcards. The good ones include the word in its original sentence, the speaker, optionally a short audio clip. This turns retention from "definition recall" into "episodic recall," which is massively stronger.

What it looks like when it works

I have been using tubevocab.com for a month as a hosted implementation of the click-a-word-on-YouTube pattern. Drop in a video URL, watch with interactive subtitles, click a word to see the translation and an AI-generated example sentence, save it to a flashcard deck with the original sentence attached, let spaced repetition handle scheduling. UI is in 10 languages which matters for learners whose L1 is not English.

What I noticed over the month:

Retention is visibly better than flat Anki decks, because you remember the speaker and the scene along with the word.
Listening comprehension improves faster than raw vocab count. You start catching phrases you would have missed before, including phrases you never actually studied.
The cost of saving a card is near zero — one click, inline — which is what makes the workflow stick. Anki's friction cost is why most learners quit it.

Free tier covers the dictionary, the flashcards, and the spaced repetition, which is enough to evaluate whether the loop works for a given learner without committing.

Why I am bringing this up

From an engineering standpoint, "interactive learning layer on top of YouTube" is a genuinely interesting systems problem: you are doing real-time NLP on streaming caption data, building a personalized word-knowledge model, and rendering a low-latency overlay on a player you do not control. Most of the research attention in language-learning tech has gone to generative tutors and chatbots; the infrastructure for exposure-driven acquisition is comparatively under-built.

For ESL learners specifically, the payoff is pragmatic: the gap between "I studied 3,000 words" and "I can follow a normal conversation" closes a lot faster when the 3,000 words were learned from real speakers saying real things, and the sentences attached to them when you hit review.

Not a pitch for any particular tool — mostly an argument that the "click-a-word-on-real-native-content" pattern is underbuilt in this space, and the tools that get it right are worth the 10 minutes to evaluate.

Why Character Consistency Is Hard in AI Comic Generation

qcrao — Fri, 24 Apr 2026 03:31:47 +0000

When you feed a story prompt into a generic image AI — say, "a detective with a red scarf walks into a neon-lit bar, then sits down at the counter, then pulls out a notebook" — you will usually get three images back where the detective has three different faces, two different scarves, and in one panel the scarf has become a tie. This is the character consistency problem, and it is the single biggest reason why text-to-image tools are bad at comics.

This post is a short walk through why it happens, what the current workarounds look like, and where the FLUX.1-Kontext-based approach fits in.

Why do characters drift?

Every text-to-image inference is in effect a fresh sample from a very high-dimensional distribution. The model has no state between generations. Prompt A and prompt B may both say "detective with red scarf," but the specific pixel arrangement that the sampler lands on is governed by the noise seed, the scheduler, and a thousand tiny decisions inside the U-Net. Two calls that share a prompt but not a seed will produce two different people who both roughly match the description.

Put differently: the model does not have a character. It has a prompt. Every panel is a new roll of the dice against the same loose description.

Classical diffusion workflows try to fix this with three tricks, none of which are great:

Seed locking. Use the same random seed for every panel. Works only if the prompt is essentially unchanged — the moment you add "sitting down" or "pulling out a notebook," the composition changes and the seed lock stops helping.
Textual inversion / DreamBooth. Fine-tune a small adapter on reference photos of the character. Effective but slow, expensive, and brittle — you are training a new adapter for every character in your comic.
Multi-image prompting. Paste the previous panel into the prompt as a reference. Some models accept it; most do not; when they do, they often regress to the mean face after a few hops.

What FLUX.1-Kontext adds

FLUX.1-Kontext is Black Forest Labs' image-to-image-conditioned variant of FLUX. The relevant design choice is that it treats the reference image not as "inspiration" (loose style transfer) but as hard conditioning during the denoising process. You pass in a reference sheet — the character's face, outfit, key features — and the generation is pulled toward that reference, not just textually but pixel-wise, through cross-attention.

For comics this is almost exactly the right primitive. The workflow becomes:

Generate a reference sheet for each character once (face, outfit, distinctive props).
For every panel, pass the relevant character's sheet + the scene description.
The model respects the sheet as a constraint, not a suggestion.

The same detective now has the same face, the same red scarf, and the scarf actually stays a scarf.

What breaks and what does not

In practice the approach works well for:

Frontal and three-quarter faces. The reference sheet is usually a clean portrait; panels that echo that framing stay on-model.
Distinctive clothing and props. A red scarf, a specific hat, a tattoo — these get preserved reliably.
Short stories (6–12 panels). Drift is minimal within a single story.

It still struggles with:

Extreme poses. A character leaping mid-air from behind is a composition the reference sheet does not cover, so the model extrapolates and sometimes loses the face.
Background characters. Secondary characters without their own reference sheet still drift. You either sheet them too or accept drift.
Long-form continuity across chapters. After 50+ panels the accumulated small variations become visible. Re-anchoring to the sheet every 10 panels helps.

A practical note on tooling

You can run this stack yourself — the FLUX.1-Kontext weights are open — but assembling the pipeline (reference sheet generator, scene scripter, panel renderer, single-panel regenerator, style picker) is a fair amount of plumbing.

I have been using comicory.com as a hosted implementation of roughly this architecture. Drop in a story paragraph, the system handles the scripting and reference-sheet step, and the multi-panel output keeps the same character recognizable. Eight art styles available (manga, Western comic, watercolor, ink wash, etc.), and critically, single-panel regeneration is supported — if panel 4 drifts, you redo only that panel without rebuilding the rest of the story. Free tier is 30 images per month which is enough to evaluate the workflow.

Not a pitch; mostly flagging it because I spent a couple of weeks trying to glue the same pipeline together locally and it was a lot of YAML.

Closing thought

The character consistency problem is a nice example of how architectural fixes beat clever prompting. For the first three years of diffusion-for-comics, the whole field was trying to solve consistency at the prompt level — longer prompts, locked seeds, character templates, multi-image prompting. None of it really worked. The real unlock was a model class that takes a reference image as first-class conditioning.

When a generation problem resists prompt engineering for long enough, the answer is usually that the model architecture is wrong for the task, and someone will eventually ship a new one. FLUX.1-Kontext is that ship for multi-panel comics. I am curious what the equivalent "right architecture" looks like for the remaining hard cases — long-form continuity, multi-character scenes with physical interaction, and expressive pose variation.