How I Built a Stroke Capture System for an AI Drawing Game

#gamedev #webdev #ai #showdev

Most drawing apps save the final image. I wanted to capture the whole journey.

For Paint Your Partner, I built a system that records every single brush stroke. Position, timing, pressure, colour, order. Everything.

Why? Because an AI is watching. And it needs to see the drawing happen, not just the end result.

The Problem

I was building a couples drawing game. You draw your partner, they draw you, an AI commentates live.

But if the AI only sees the finished image, the commentary is boring. "Nice portrait." Cool. Thanks.

I wanted it to react as you draw. Spot the eyes appearing. Notice you're spending ages on the hair. Catch the moment you give up on the nose and scribble something abstract.

For that, I needed stroke data.

What Gets Captured

Every stroke is an object. It contains:

Path data (array of x/y coordinates)
Colour
Brush size
Timestamp (when the stroke started)
Duration (how long it took)
Order (which stroke number this was)

Nothing clever here. Just capture everything and store it.

I've got over 1,000 drawings in the database now, each with complete stroke history.

Replay

With full stroke data, I can replay any drawing as a video. Speed it up, slow it down, export as GIF or MP4.

This is useful for:

Shareable content (watch someone draw their partner in 10 seconds)
AI training data (if I ever want to fine-tune)
Debugging (see exactly what a user drew and when)

Real-Time AI Commentary

The AI watches strokes arrive in real-time. It doesn't wait for you to finish.

As features appear on the canvas (eyes, hair, smile), it fires off short commentary. One or two sentences. Cheeky, not critical.

"Ooh those eyes are looking dreamy!"

"Someone's really capturing that smile!"

The stroke data tells the AI what's being drawn and how. Hesitation, confidence, chaos. It's all in the timing.

The Engine

Paint Your Partner runs on the same engine as Artbitrator (my multiplayer drawing game, like Quick Draw but with AI judging). Same stroke capture. Same replay system. Different game modes.

Building the core engine once and spinning up different products on top has been worth the upfront effort.