This is a submission for Weekend Challenge: Dog Days Edition
What I Built
My dog has opinions. She has never once shared them.
So I built BarkReels: you upload a photo, and it gives her a voice. The mouth actually moves with the audio, she blinks, her ears twitch, subtitles burn in, and you get a real MP4 at the end.
Demo
The goal was to see how far a serious video pipeline could go with no backend at all. No upload, no render queue, no "your video will be ready in 4 minutes" email. Your photo goes to the vision API you picked and nowhere else. Cutting your dog out of the background, animating her, and encoding the MP4 all happen on your own machine, with your own GPU.
What you get to play with
Everything here is a control in the UI, not a config file.
Where the words come from. You pick before analysis, so the vision model only does the job you need:
- Let the AI write the monologue from the photo
- Write the script yourself, pick a voice
- Record your own voice and use it directly
Voice personas: Deep & Wise 🧙, Playful Pup 🎾, Dramatic 🎭, Sassy Diva 💅.
Animation styles, least to most motion:
| Preset | What it does |
|---|---|
| 🗿 Locked Off | Only the face moves. The default, and most convincing. |
| 🍃 Barely There | A whisper of drift |
| 🎙️ Portrait | Gentle sway |
| 🙃 Belly Roll | Lazy sideways flop |
| 🏀 Bouncy | Springy on the beat |
| 💨 Zoomies | Maximum chaos |
Then sliders for mouth movement, head motion, emphasis nods, camera push-in, handheld shake, background parallax, blink rate and ear twitch.
Backgrounds: blurred, original, your own uploaded image, plus procedural Sunset, Studio, Park, Neon and Solid. Your dog gets segmented out of the photo, so the background genuinely sits behind her and moves independently.
Subtitles: Karaoke (rolling window, highlighted word), Pop-up (two words, hard cuts) or Minimal (full sentence). Size, position, uppercase and colour are yours. The colour picker comes pre-loaded with shades sampled from your actual photo, so a brown dog gets warm captions and a pink collar gets pink ones.
Export: MP4 or WebM at 1080x1920, 720x1280 or 1080x1080, 24 to 60fps.
Getting the best results
The animation moves specific anatomical points, so the photo matters more than you would expect. This took a lot of bad outputs to work out:
✅ Head-on portrait. Profile shots hide one eye and one ear, and the rig has nothing to animate on that side.
✅ Face fills the frame. Head and shoulders. In a full-body shot the muzzle is too small to deform cleanly.
✅ Eyes, nose, mouth and ears clearly visible. These are literally the coordinates the animation moves.
✅ One dog. Segmentation keeps a single subject. Two dogs become one blob.
✅ Sharp and well lit. Blur and deep shadow make the cutout edge mushy.
❌ Nothing over the face. Sunglasses, hands, toys. Anything covering an anchor gets deformed along with it.
Two more things I learned the hard way:
Start with Locked Off. It is counterintuitive, but motion is what gives away a puppeted photo. Hold the frame completely still and the viewer reads it as real video of a dog sitting calmly, leaving only the mouth to judge. And the mouth is the one part driven by real audio.
Short scripts win. Two or three punchy sentences. Long monologues drift out of sync with the face and burn your free tier faster.
Demo
🔗 Live app: https://barkreels.vercel.app/
Seriously, go try it. It is faster than finishing this post.
| Step | Time |
|---|---|
| Sign up at ElevenLabs, copy the API key | ~3 min |
| Sign up at Google AI Studio with your Google account, copy the key | ~2 min |
| Open the app, paste both keys, drop in a dog photo | ~1 min |
| Hit analyse, hit generate, hit export | ~4 min |
Both free tiers are enough. No card required for either. The keys live in your browser's local storage and are sent only to the provider you picked, because there is no server of mine to send them to.
⏳ What to expect on your first run. Because background removal happens on your machine, the browser fetches the segmentation model before it can cut your dog out. That is roughly 25MB of ONNX runtime and neural network weights, and on a slow connection it can take a minute or two. You get a progress bar, and the download starts the moment you drop a photo in so it overlaps with everything else. It is a one-time cost. Your browser caches it and every run after that goes straight to work. If the first one feels slow, that is what is happening, and it is not stuck.
No dog? Steal one from Unsplash. It works on any dog and I will not tell anyone.
Code
singhAmandeep007
/
barkreels
Give your dog a voice 🐕
BarkReels 🐕 - client-only AI dog video generator
Upload a photo of your dog. Get back a vertical short-form video where they talk, breathe, blink, and nod - with burned-in subtitles and a real MP4 at the end. Every pixel is rendered in your browser. There is no backend.
Built for the DEV Weekend Challenge: Dog Days Edition.
How it works
photo ──┬─→ vision model ──→ breed / mood / monologue / facial anchors
└─→ ISNet (ONNX, in-browser) ──→ dog cutout + alpha
monologue ──→ ElevenLabs ──→ audio + word timestamps
│
├─→ RMS envelope (asymmetric smoothing)
└─→ subtitle timing
envelope + anchors + cutout ──→ animation rig ──→ WebGL warp ──→ frames
│
WebCodecs H.264 + AAC ──┴──→ MP4
The one architectural rule
renderFrame(t) is a pure function of time. It reads no clock, consults no
playback state, and mutates nothing that affects a later call.
That…
MIT licensed. Fork it, enable Pages in repo settings, and push: there is a GitHub Actions workflow that builds and deploys on every push to main. Since there is no backend, static hosting is all it has ever needed.
The interesting files:
src/render/rig.ts the animation formula (pure functions)
src/render/glRenderer.ts WebGL2 renderer, renderFrame(t)
src/render/backgroundPlate.ts pull-push inpainting
src/render/exporter.ts offline WebCodecs encode + muxing
src/services/vision.ts Gemini and Ollama behind one interface
src/services/audioAnalysis.ts envelope, onsets, PCM decode
How I Built It
photo ─┬─→ Gemini ──→ breed, mood, monologue, facial anchor coordinates
└─→ ISNet (ONNX, in-browser) ──→ dog cutout with alpha
monologue ──→ ElevenLabs ──→ audio + word-level timestamps
├─→ loudness envelope ──→ drives the jaw
└─→ word timings ──→ drives subtitles
envelope + anchors + cutout ──→ animation rig ──→ WebGL ──→ WebCodecs ──→ MP4
Google AI does two jobs in one call. Gemini Flash writes the monologue in the dog's voice and acts as a precise vision annotator, returning normalised bounding boxes for the nose, mouth, eyes and ears. Those coordinates are what the shader deforms. When you supply your own script the prompt drops the writing job entirely, so the model spends all its attention on the coordinates.
ElevenLabs is doing more than narration. The /with-timestamps endpoint returns character-level timings alongside the audio. Those timings drive the karaoke subtitles, and the loudness envelope derived from the same audio drives the jaw. The voice is not decoration here, it is the animation input.
The one rule that made everything else work: renderFrame(t) is a pure function of time. It reads no clock and consults no playback state. Preview is requestAnimationFrame(() => renderFrame(audio.currentTime)). Export is a plain for loop over frame indices. Same code, same pixels, so the preview is an actual promise about the output. Export runs offline at roughly 2 to 3 times real time, and backgrounding the tab cannot corrupt it.
The lip sync is not a model. It is the loudness envelope with deliberately asymmetric smoothing: 15ms attack, 90ms release. Muscles snap a jaw open and tissue eases it closed. Symmetric smoothing makes the mouth flutter shut between syllables like a VU meter, and that one asymmetry did more for perceived realism than anything else I tried.
Prize Categories
Best Use of ElevenLabs and Best Use of Google AI. Both are wired into the animation rather than bolted on top of it: Gemini's bounding boxes are what the shader deforms, and ElevenLabs' word timings and loudness envelope are what drive the mouth and the subtitles. See the section above for the details.
What I would build next
- Multi-photo angle switching. Upload and thumbnails work, but the renderer still uses one image. Cross-fading between angles on speech beats is the obvious next move.
- Better anchors for flat faces. Pugs and bulldogs have a short muzzle and my fallback jaw hinge lands too close to the chin. A breed-aware hinge would fix it.
- Head turn. The head is a flat layer today. A depth estimate would allow a small yaw and sell the 3D far harder.
-
Cross-origin isolation. ONNX runs single-threaded because enabling
COOP/COEPwould break the Gemini and ElevenLabs calls. Moving segmentation into an isolated worker would make it roughly 4 times faster. - Tail wag and a real ground shadow for full-body shots, which currently get no love at all.
Ten minutes, start to finish, and both free tiers cover it. If you make one, post it in the comments. I genuinely want to see what your dog has been holding back.
Top comments (0)