We Gave Our DEV Avatar Project a V2… and It Escalated Quickly
A few months ago, @webdeveloperhyper and I made a small side project where two VRM avatars talked to each other in a 3D scene.
At the time, it was just meant to be something fun. Two developers from different countries making weird little animated characters interact over the internet because… honestly, why not?
If you missed the first post, here it is:
How a DEV Friend and I Brought Two Avatars to Life
Back then, the project was much simpler. A conversation system, some VRM animations, a few gestures, and a surprising amount of time spent trying to stop avatars from snapping into cursed poses mid-animation.
We thought we were mostly done with it.
We were very wrong.
Somewhere along the way, the project became… alive
The funny thing about side projects is that they rarely stay contained.
You fix one thing, get one new idea, add one “small feature,” and suddenly a weekend experiment starts behaving like an actual system.
That’s pretty much what happened here.
While I was busy juggling office life, Japanese, engineering work, interviews, marketing side quests, and trying to survive adulthood without my brain overheating… WDH was quietly evolving the project in the background like some kind of sleep-deprived VRM wizard.
And every few days I’d wake up to messages like:
“Added music and music button. Then pushed."
or
“fix mobile speech bubble layout and responsive avatar positioning”
or sometimes just:
“fixed and pushed.”
And somehow the project kept getting cooler.
The avatars stopped feeling scripted
One of the biggest changes in v2 was that the characters stopped feeling like objects that simply played animations on command.
Now they actually react.
They blink naturally. Follow movement. Shift attention while listening. React while the other character is speaking. Small details, but weirdly important ones.
It turns out humans are very sensitive to tiny signs of life.
A static model feels dead immediately.
But add:
- eye tracking
- idle movement
- small delays
- slight reactions
- facial expressions …and suddenly your brain starts treating it like a character instead of geometry.
That transition fascinated me way more than I expected.
At one point we spent an unreasonable amount of time adjusting tiny gesture timings that most users probably won’t consciously notice.
But that’s kind of the magic of interactive systems.
The invisible details are usually doing the heavy lifting.
The architecture quietly became more interesting too
The first version was mostly about “getting it to work.”
The second version became more about orchestration.
Now the scene handles:
- dialogue sequencing
- animation coordination
- background switching
- music control
- expression timing
- responsive layouts
- costume switching
- branching interactions
And because of that, hardcoding behavior stopped making sense very quickly.
So instead of writing giant chains of conditions everywhere, the conversation itself became structured data.
Each dialogue entry knows:
- who is speaking
- what animation plays
- what expression appears
- how timing should behave
Which made the system dramatically easier to extend without everything collapsing into spaghetti.
Ironically, the more playful the project became, the more important clean systems thinking became too.
The funniest bugs were always animation-related
There is probably a law somewhere stating that if you work with avatars long enough, something horrifying will eventually happen. And if not it should be.
At various points:
- arms rotated into dimensions unknown to mankind
- blinking broke and created sleep paralysis avatars
- gesture transitions snapped violently
- one animation froze mid-pose and looked like the character had emotionally given up on life
- Classic side project experience honestly.
But weirdly enough, those moments ended up becoming some of the most memorable parts of the project.
Because every fix made the avatars feel a little more believable.
While all this was happening, @webdeveloperhyper was building something much bigger
And this is honestly the part I respect the most.
While we were experimenting with our collaboration project, WDH was also building something called AI Avatar — a VS Code and Chrome extension where VRM avatars react to AI activity, animate while you work, track your cursor, trigger expressions, speak through speech bubbles, and basically exist beside you like a tiny chaotic coding companion.
The interesting thing is that it never felt like one of those overly corporate “AI productivity” tools trying to optimize your soul.
It felt playful.
Like something made by someone who genuinely likes interactive characters and wants technology to feel a little more alive and personal.
And I think that mindset naturally spilled back into our collab project too.
A lot of the small interaction details, responsiveness ideas, animation handling improvements, and experimentation mentality came from constantly exploring these systems further.
Which honestly reminded me why collaborations matter so much.
You don’t just share workload.
You share ways of thinking.
The project accidentally became a tiny engine
That’s probably the strangest realization from all this.
At some point we stopped thinking:
“we’re making two avatars talk”
And started realizing:
“wait… this is basically becoming a reusable interaction system.”
Now there’s:
- branching dialogue
- modular animation logic
- reusable scene orchestration
- dynamic reactions
- configurable backgrounds
- audio systems
- responsive UI behavior And suddenly your silly little side project starts looking suspiciously scalable.
That’s always the dangerous phase.
But honestly, the best part was still the process
The actual code matters, obviously.
But what I’ll probably remember most is:
- random weekend debugging
- async conversations across timezones
- sending screenshots back and forth
- testing weird ideas just because they sounded funny
- slowly watching something gain personality
- That feeling is hard to replicate in structured environments.
No roadmap pressure. No stakeholder meetings. No “business value alignment.”
Just curiosity carrying the project forward.
And honestly?
That’s probably why side projects stay fun.
Where this goes next
Right now, v2 still feels like the beginning.
There are already ideas floating around for:
- interactive storytelling
- user-triggered reactions
- branching scenes
- mini-games
- dynamic conversations
- more expressive characters
- Or maybe something completely different. At this point, I’ve stopped trying to predict where these projects go.
The fun part is discovering it while building.
For the people interested in trying it themselves
WDH made a public version available here:
GitHub Repo (without VRMA + MP3)
And honestly, if you build something weird with it, let us know.
Those are usually the best ideas anyway.
Top comments (17)
Awesome updates! It's great to see this project evolve! Keep it up!
Have you considered gamifying the avatar by adding vitals such as sleep (forces you to take intentional breaks while the avatar rests up), or hunger (commits could satisfy this hunger for eg.)?
Omg these are such GREAT ideas actually! @webdeveloperhyper we could really work on these!!!
Thank you Julien, we would definitely look into how to make these work, if doable we will surely add it up in v3!
Thank you for your comment! 😀 We will consider adding more functions when we update our project to v3.
Thank you for sharing your journey! Those animation-related bugs were hilarious to read about 😂 BTW, I never thought it's so much work to build all the gestures, eye movements, blinking, and so on!
Thank you for checking it out (ᵔᴥᵔ) Glad you liked it!!!
And as far as knowing about the work that goes into making VR models work... neither did we know that before this project😭
Means you learnt a lot from this project, good job! 👏🏻
This is such a fun read! The part about "one animation froze mid-pose and looked like the character had emotionally given up on life" had me laughing — those are exactly the kind of bugs that are painful in the moment but make the best stories later.
What really resonated with me is how the project accidentally became a reusable engine. That happens with every side project I've built too — you start with something simple and suddenly you're architecting a whole system just to keep things clean.
The cross-timezone async collaboration angle is underrated as well. No meetings, just "fixed and pushed" — honestly sounds ideal.
Heyy Harun!!! Glad you found our story funny 😭
And yeah now that we have brought this to a good level we can grow this into multiple directions as we like!
The "no meetings" is definitely a blessing in disguise lol.
Thank you for your cooperation! 😀 It was fun making a collaboration project with you. I want readers to feel the difference between India and Japan through the changeable backgrounds and music. Japanese offices do not have Tatami for floors, but it’s okay. 🤣 Japanese music uses the traditional instrument called the Koto. Looking forward to doing something fun and educational together again!
Really practical engineering mindset here. Token efficiency in AI-assisted coding is underrated. Building your own streamlined tool instead of bloating every interaction with thousands of unnecessary tokens shows real maturity. The 900-line investment will compound over time — this is the kind of thinking that separates senior engineers from junior ones relying blindly on AI suggestions.
Turning a fun project into a sophisticated system sounds like a typical dev spiral. I'm curious how you handled data structuring for the conversations. Storing and managing dynamic dialogue sounds simple, but scaling it can be a headache. If you're looking at modular logic or branching dialogue, the challenge probably goes beyond basic JSON structure. I've been using prachub.com for system design mocks. It's more useful than browsing random forums when I'm juggling a bunch of concepts. That side project evolution is pretty wild—easy to get lost in feature creep, right?
Hey Andy! Yes for now the JSON structure is handling it well and I agree scaling it would be challenging... We will definitely check out your suggestion, excited to see how this grows!
Great article! The technical depth here is exactly what the community needs. One补充: don't forget to validate your assumptions early - it's easier to pivot when you have less code. For devs building SaaS, finding users doesn't have to be the hardest part - Rixly helps identify developers actively seeking solutions like yours.
My congratulations! Together are always more productive
Thank you Angela!
Damn!!!! stuff.. Really enjoyed reading your journey, and appreciate you work 👏🏻
Thanks Omkar, Glad you liked it!