DEV Community: yubin hong

CapCut's AI Captions Went Pro-Only — So I Built a Free Alternative

yubin hong — Fri, 17 Jul 2026 11:54:24 +0000

Hey, I run a small YouTube channel.

Not a huge subscriber count, but I edit almost every day. I add captions to basically every video, and for a long time I relied on CapCut's AI auto-caption feature. It transcribes what you say automatically, which saves a ton of time compared to typing everything out line by line.

Then the problem started

At some point, CapCut locked the AI caption feature behind a Pro subscription.

For small creators like me, monetization isn't always stable yet, and another monthly subscription just for captions adds up fast. I kept thinking, "do I really need to pay every month just to add subtitles?" and figured a lot of other small creators were probably in the same boat.

So I decided to just build it myself.

I built a free AI caption site

After a few weeks of working on it in my spare time, I finished a site where you upload a video and AI automatically transcribes the speech into captions.

Here's what it does:

Upload a video and AI analyzes the audio to generate captions automatically
Download the captions as text or burn them directly into the video
Free to use for everyone, no subscription or Pro tier required
Runs entirely in the browser, no heavy software to install like CapCut

To be clear, I didn't invent some groundbreaking new technology. I mostly took existing speech recognition AI and shaped it into something small creators can use without worrying about cost. But if you've been stuck on the same caption problem I was, it should be genuinely useful.

Why give it away for free

No grand mission here. I needed this feature myself and didn't want to pay for it. I'm putting it out there hoping other small YouTubers and creators just starting out can handle captions without another subscription eating into their budget.

If you've ever hesitated before paying for an editing subscription, give it a try.

Questions or feedback are welcome in the comments. If you try it out, let me know what could be better — I'm actively improving it.

# Engineering Log #03 — The honest asterisk got cashed in (product F1 0.887), and the model finally became a product

yubin hong — Mon, 06 Jul 2026 01:42:04 +0000

A build-in-public engineering log. Every number below is pulled straight from the test reports — measured values, not rounded to look pretty, not invented. A success you cherry-picked isn't a success.

Log #02 ended on an asterisk. We had a video action classifier at 94% accuracy and an end-to-end highlight pipeline at recall 0.86, but the newly-swapped "pixel finder" was measured under a generous assumption — perfect tracking, GT-interpolated boxes. Its product operating point, measured honestly against real kinematics, was only R 0.52 / P 0.57. We wrote, in plain text, that we'd have to wire it into the real pipeline with actual tracking lock and re-confirm with product_eval before claiming anything.

This log is that re-confirmation — plus the bigger thing that happened while we weren't looking: the model stopped being a model and became a product.

Where we are now:

Product pipeline (real tracking, end-to-end): F1 0.887 — recall 0.868, precision 0.908 on the holdout config.
The generous upper bound from Log #02 didn't collapse when we made it honest. It held.
Shipped an actual product surface: a web app (upload → lock a player → auto-edit → done) and a cinematic slow-motion render pipeline.
Deployment reality check: we pivoted the whole serving path from CPU to GPU.

Part 1 — Cashing the asterisk: the product number, measured honestly

In Log #02 we were careful not to celebrate the spotter's headline recall, because it was measured with GT-interpolated boxes — i.e. assuming the tracker never loses the player. That's the spotter's home-field advantage. The real question was always: what happens when the boxes come from the actual tracker, running end-to-end, on the real product config?

We ran it. Here are the receipts, straight from the eval reports, all on the same 45-video / 68-critical holdout set:

Config	Recall	Precision	F1	Fired	TP
`config.yaml` (product baseline)	0.853	0.906	0.879	64	58
`config_gpu_a` (GPU parity)	0.853	0.906	0.879	64	58
`config_spotter_v2` (holdout)	0.868	0.908	0.887	65	59
`config_gpu_b` (precision-leaning variant)	0.809	0.917	0.859	60	55

The number to sit with: the Log #02 product operating point was R 0.52 / P 0.57. The re-confirmed, real-tracking, end-to-end product point is R 0.87 / P 0.91 / F1 0.887.

We want to be precise about why this isn't cheating. The 0.52/0.57 in Log #02 was the kinematic-only operating point — the old finder, before the pixel spotter was integrated with tracking lock. The honest fear was that the spotter's 0.86-ish recall was an artifact of perfect boxes and would fall apart on real tracks. It didn't. When we integrated the spotter into the real pipeline with the actual tracker, the end-to-end product number landed at F1 0.879–0.887 depending on config. The generous assumption and the honest measurement converged instead of diverging. That's the good outcome, and it's the one we were least sure we'd get.

Two things we're nailing down honestly:

1. GPU parity is exact, not approximate. config.yaml on Apple Silicon (mps) and config_gpu_a on CUDA produce the identical R 0.853 / P 0.906 / F1 0.879 — same 64 fired, same 58 TP. That matters: it means moving to GPU is a throughput decision, not a quality one. The model doesn't behave differently on the box we ship on.

2. gpu_b is a knob, not an upgrade. The precision-leaning variant trades recall (0.853 → 0.809) for a hair more precision (0.906 → 0.917). Same 68 criticals, fewer firings. It's there so we can pick a false-positive tolerance per deployment, not because it's "better." For a highlight product, missing a moment is worse than an occasional extra clip, so the recall-leaning baseline is the default.

Part 2 — The model became a product

Log #02 was entirely about the model. This log is where the model grows a body. Two big pieces shipped.

The render pipeline: from "we found the moment" to "here's the clip"

Detecting an impact timestamp T is worthless to a parent with a phone unless it comes out the other end as something watchable. So we built the output layer — an orchestrator that turns (video + target trajectory + impact T) into a vertical slow-motion highlight mp4. The flow is deliberately ordered (stabilize, then retime):

Decode only the window around T ([pre, post]) at native fps — never the whole video.
Resample the target trajectory to real frame times and smooth it with a Kalman/RTS pass.
Generate stabilized frames with a moving-crop virtual camera.
Integrate a speed ramp and invert it to get per-output-frame (index, α).
Interpolate and stream straight to the writer — no buffering the whole clip in memory.

One design decision worth calling out because it's counterintuitive: V1 does not crop or zoom in. On low-resolution real-world footage, a lock-on crop just destroys image quality — the parent already framed the subject big enough. Vertical reframe and lock-on zoom are deferred to V2. We'd rather ship the un-degraded frame now than a shaky zoomed one.

The slow-motion itself uses a lead-anchored ramp: kinematic impact T tends to get detected late (at landing / follow-through), so we pull the slow-motion center ahead of T to cover the decisive action — the wind-up and contact — instead of slow-mo'ing the part after it's already over.

On top of single clips sits a reel compiler: rank highlights by criticality, greedily pick the best ones that fit a 15/30/60s budget, then re-order them chronologically so the reel still tells the game's story, with short crossfades at the seams. (Both render and reel are pure cv2/numpy, FFmpeg-free, silent MVP — audio is Phase 2.)

The web app

There's now an actual front end: a bilingual (KO/EN) React/TS app with the whole flow a non-technical user needs — upload, lock the player you care about, watch the AI edit, get the finished vertical highlight. Screens for landing/upload, target lock, processing, gallery, and done. The pitch is one sentence: upload your kid's game, get the highlights, don't touch an editor.

Part 3 — Deployment reality: CPU → GPU

Log #02's deployment plan was Oracle Cloud's Always Free tier — 24/7 on ARM CPU, torch-CPU, no GPU. The goal was a real server that stays up when the laptop sleeps, at zero cost. Two things made us pivot.

First, the free tier shrank. Oracle cut Always Free A1 from 4 OCPU / 24 GB to 2 OCPU / 12 GB (2026-06). Less headroom for exactly the workload we run.

Second, and decisively, CPU inference is too slow for the pipeline we actually shipped. The production path is RT-DETR detection plus r2plus1d_18 dense scoring across the whole video — on CPU, a single job takes minutes. That's survivable for an async "we'll notify you" MVP, but it caps daily throughput hard. So we built the GPU serving path: Dockerfile.cuda, a one-command deploy_gpu.sh (rsync → remote provision of Docker + nvidia-container-toolkit + firewall + build + start), and a T4 benchmark step. The pixel spotter (video_scorer.pth) bakes into the image as the deploy artifact, and — per Part 1 — GPU output is bit-for-bit the same quality as local, so this is purely about serving more jobs per day.

Part 4 — Two dead ends we're keeping dead (receipts included)

Honesty means reporting the things that didn't work and staying disciplined about not reviving them.

RF-DETR fine-tuning stays shelved. Back in Log #01 territory we fine-tuned RF-DETR to mAP 0.716 and it moved end-to-end recall backwards. We re-confirmed on the real domain: no improvement. We're keeping baseline RT-DETR. Swapping to rfdetr remains a config flag away, but the measured verdict is that it doesn't earn its complexity.

The pose-TCN action classifier is now legacy. The pixel spotter is the confirmed production finder-and-judge (F1 0.887). The old pose-TCN path is effectively disabled (crit_tcn_floor: 1.0) because it showed no discriminative power on the real domain. It's still hot-swappable if we ever want it back, but nothing in the default path depends on it.

Neither of these is a failure we're hiding — they're two forks we explored, measured, and closed. The 0.887 didn't come from adding more model. It came from finding the right one and refusing to bolt the wrong ones back on.

Where we stand

Product number, honest: F1 0.887 (R 0.868 / P 0.908) end-to-end with real tracking. The Log #02 asterisk resolved in our favor — the generous upper bound survived contact with reality.
GPU parity: identical quality on mps and CUDA; the CPU→GPU move is throughput, not accuracy.
It's a product now: upload → lock → auto-edit → vertical slow-mo reel, behind a real web app.
Discipline held: RF-DETR and pose-TCN both measured, both closed, neither revived.

The remaining star, unchanged

The domain caveat from Log #02 still stands and we're not going to let it fade: these numbers are on SVW-derived research footage. The real test is still a commercial rebuild on genuine user uploads. If F1 0.887 survives that domain shift, we have a product. If it drops, you'll see the drop here first — same as every other number in this log.

If you're building in the same space and hitting the "the model works but is it a product" wall, I'd love to compare notes in the comments — especially on the CPU-vs-GPU serving math for video pipelines.

Engineering Log #02: What our sports CV model actually scores right now (94% classifier, 0.86 highlight recall)

yubin hong — Sun, 05 Jul 2026 01:06:30 +0000

Build-in-public engineering log. Every number below is a real measurement pulled straight from our test reports — nothing rounded up to look nicer, nothing invented. If you only ever hear the wins, the wins mean nothing.

In Log #01 we told the messy story of getting highlight recall from 0.56 → 0.86. Since then we built the piece that sits underneath that pipeline: a video action classifier that decides what is happening in a clip. This log is the honest scoreboard for both — because we were curious ourselves what the model actually scores today.

TL;DR current state:

Action classifier: 94.0% test accuracy, macro-F1 0.94, across 4 classes.
Highlight pipeline (end to end): recall 0.86, up from 0.56.
Training data grew 475 → 762 clips between v1 and v2.

Part 1 — The action classifier: 94% today

The classifier answers a narrow but load-bearing question: given a short clip, is this a baseball swing, a basketball layup, a soccer shot, or nothing (background play)? Get this wrong and every downstream highlight decision inherits the mistake.

The model is nothing exotic — an R(2+1)D-18 backbone, 16 frames sampled per clip at 112×112. Deliberately small: it has to run cheaply on real uploaded footage, not win a leaderboard.

Here's the current test report, verbatim from video_scorer_testreport.json:

Class	Precision	Recall	F1	Support
baseball_swing	1.000	0.971	0.986	35
soccer_shooting	0.933	0.933	0.933	15
none	0.938	0.918	0.928	49
basketball_layup	0.850	0.944	0.895	18
Overall accuracy			0.940	117

Macro-F1 lands at 0.94. Baseball swing is nearly solved (precision 1.000 — it never fires falsely). Basketball layup is our weakest link at F1 0.895, and the confusion matrix says exactly why.

Reading the confusion matrix honestly

                 pred: bball  base   none  soccer
baseball_swing  [   34      0      1      0  ]
basketball_layup[    0     17      1      0  ]
none            [    0      3     45      1  ]
soccer_shooting [    0      0      1     14  ]

Almost every error is a confusion with none, not a cross-sport mix-up. The model rarely calls a layup a swing; it occasionally can't decide whether a play is a highlight at all. Concretely: 3 background clips get misread as layups (that's what drags layup precision to 0.85), and a scatter of real actions leak into none.

That's the good kind of error profile. The sports are cleanly separable; the remaining work is sharpening the action vs. no-action boundary, which is a data problem, not an architecture problem.

Part 2 — More data, honestly labeled

The single biggest lever between v1 and v2 wasn't a clever loss function. It was clips.

	v1	v2
Total clips	475	762
`none` (train)	119	335

We roughly tripled the negatives (none). That's not glamorous, but the confusion matrix above is exactly why it mattered: our errors live on the action/no-action boundary, and you only teach that boundary by showing the model far more of what a non-highlight looks like.

One caveat we keep visible in our own metadata: the current set is a research prototype. The note in dataset_meta.json literally reads "commercial model = rebuild on user / commercially-usable video." The 94% is real, but it's 94% on prototype-domain data — we're not going to quietly let that number imply more than it earned.

The augmentation bet (this one paid off)

Phone footage is ugly: motion blur, compression noise, players filmed tiny and far away, handheld shake. So we simulate all of it at train time rather than pretend clean clips generalize:

motion blur up to a 21px kernel
downscale to 0.35–0.7× ("player filmed far away")
JPEG quality floor of 40 (phone compression artifacts)
mild shift/scale/rotate for handheld shake

Each real clip spawns ~2 augmented variants. The point isn't more data for its own sake — it's data that looks like the mess the model meets in production.

Part 3 — The pipeline recall story (the Log #01 recap, with the receipts)

The classifier feeds a larger highlight-detection pipeline. That pipeline is the one that went 0.56 → 0.86 recall. Two lessons from that arc are worth re-stating, because they shaped how we built Part 1:

1. We fixed the wrong layer first. Our instinct was "make the detector see better," so we fine-tuned RF-DETR to a respectable 0.716 mAP — and end-to-end recall moved 0.60 → 0.59. Backwards. When we decomposed every miss by stage, detection owned only 15% of failures; 55% died at the pose/kinematic stage. We'd polished the layer that wasn't broken.

2. Never trust a batched change. We turned on four good features at once and the score dropped 0.769 → 0.692. A one-at-a-time ablation exposed a single culprit (video_mode) dragging the batch down. Three of four features were genuinely helping; one bad apple made the whole basket look rotten.

Per-sport, the recall gains held across the board rather than being carried by one lucky category — basketball alone went 0.43 → 0.82.

Where we actually stand

Classifier: 94.0% accuracy, macro-F1 0.94. Sports are cleanly separated; the only soft spot is the highlight/no-highlight boundary, and we know it's a negatives-data problem.
Pipeline: 0.86 recall end-to-end, robust across sports.
Known asterisk: prototype-domain data. The commercial rebuild on user footage is the next real test, and that number could move — we'll report it either way.

What's next

Rebuild the dataset on genuinely user-uploaded, commercially-usable footage and re-run this exact report. If 94% survives the domain shift, we have a product. If it doesn't, you'll see the drop here first.

Building something in the same space and hitting the action/no-action boundary too? I'd love to compare confusion matrices in the comments.

We took highlight detection from 0.56 to 0.86 — with zero new footage and zero cloud training

yubin hong — Sat, 04 Jul 2026 01:06:38 +0000

SportZone turns a parent's phone video of a youth game into a highlight reel. The hard part is finding the decisive moment. On real footage we were missing nearly half of them. Here's how a week of measuring — not guessing — fixed it.

01THE PROBLEM
On real phone footage, the model went half-blind

Our classifier scored 0.82 on curated YouTube clips. Confident, we ran it on genuine parent-filmed smartphone video for the first time. The number that matters — did we catch the decisive moment? — came back at 0.56. We were missing 44% of the highlights. A highlight tool that misses half the highlights isn't a tool.

The curated-vs-real gap is the whole game. So the first thing we built wasn't a fix — it was a way to measure honestly on the real domain.

02 THE WRONG TURN

We bet on the obvious culprit. We lost.

"The boxes look tight — the detector must be missing people." Reasonable. So we did the expensive thing: assembled ~9,100 commercially-licensed sports images, fine-tuned a detector on a cloud GPU, hit a healthy mAP of 0.716, and plugged it back in.

Hours of data-wrangling and training bought us nothing. Frustrating — but the failure was the clue. If a better detector changes nothing, detection was never the bottleneck. We just didn't have the evidence yet.

03 THE MEASUREMENT THAT CHANGED EVERYTHING

We decomposed the failures instead of arguing about them

We took every missed moment and tagged where in the pipeline it leaked: detection → tracking → pose → impact. One script, no opinions. The breakdown ended the debate:

Detection was 15%. The real leak — 55% — was downstream: the person was found and tracked, but the pose signal was too weak for our kinematics to register the impact. We'd been paving the wrong road. Lesson: decompose before you invest.

04 FIXING THE REAL BOTTLENECK

Sharper pose signal broke a ceiling we'd been stuck under

Now aimed at the right target, the fixes were cheap and pure-software: upscale the pose crop (256 → 384), raise model precision (complexity 1 → 2), and stop cropping off the legs with an asymmetric crop bias. No data. No GPU. This wasn't moving an operating point — it was a genuinely cleaner signal, and it broke past an F1 ceiling of 0.54 that pure threshold-tuning had never cracked.

05THE TRAP

The improvement first looked like a regression

We flipped on four improvements at once and recall dropped: 0.769 → 0.692. The tempting move: lower the detection threshold until the number looks good again. That would have papered over a real defect instead of finding it. So we ran a clean one-at-a-time ablation.

This is the same lesson as the detector detour, one layer deeper: the number lying to you is more dangerous than the number that's low. Ablation is how you tell them apart.

06THE RESULT
0.56 → 0.86, and the weak sports came home

● Shipping note Precision sits at 0.51 — a few extra false highlights. For a highlight reel that's the right trade: better to over-catch than miss the goal. We tighten it after beta.

07WHAT WE'D TATTOO ON OUR ARM

Three lessons, paid for in wasted GPU hours

i.
Decompose before you invest. We burned a cloud fine-tune chasing a bottleneck that was 15% of the problem. A one-afternoon failure breakdown would have redirected the whole week.

ii.
Ablate one change at a time. Four fixes at once hid a regression inside a net gain. Isolation named the single culprit in one pass.

iii.
Distrust the number that recovers too easily. Lowering a threshold would have masked the video-mode bug. The convenient fix and the correct fix are rarely the same move.

the winning config — all software, no new data

z_threshold: 2.0 # impact sensitivity
video_mode: false # the one flag that was the culprit
crop_size: 384 # sharper pose signal
model_complexity: 2 # pose precision
bottom_bias: 0.15 # stop cropping off the legs
tcn_rescue: on # classifier rescues weak candidates