Building SSR-Friendly Avatars with In-Browser AI: How I Trained Python Models and Ported Them to TensorFlow.js

We just shipped the initial release of Avatune, an open-source avatar system that combines native SVG rendering with experimental in-browser ML models. Here's what makes it different and why you might care.

The Problem

Avatar libraries typically fall into two camps:

1. Canvas-based - Fast, but breaks SSR and accessibility
2. SVG-as-image - SSR-friendly, but no dynamic theming or component composition

I wanted both: true SSR compatibility AND intelligent avatar generation from user photos.

Native SVG = First-Class SSR

Every avatar in Avatune renders as a real SVG element, not a canvas or base64 image. This means:

• Zero hydration mismatch - Server renders identical markup to client
• Accessibility built-in - Screen readers can access SVG semantics
• CSS styling works - Target elements with selectors, use CSS variables
• Inspectable in DevTools - Debug like any DOM element

import { Avatar } from '@avatune/react'
import theme from '@avatune/pacovqzz-theme/react'

// This SSR renders as clean SVG markup
function UserCard({ seed }: { seed: string }) {
  return <Avatar theme={theme} seed={seed} size={200} />
}

Experimental In-Browser ML Predictors

Here's where it gets interesting. I trained several CNN models in Python using TensorFlow/Keras on the CelebA and FairFace datasets, then converted them to TensorFlow.js for browser inference:

• Hair Color Predictor - black, brown, blond, gray
• Hair Length Predictor - short, medium, long
• Skin Tone Predictor - light, medium, dark
• Facial Hair Predictor - clean-shaven vs facial hair

The training pipeline uses Marimo notebooks (think Jupyter, but reactive). Models are quantized to uint8 and served via CDN. Total bundle for a predictor: up-to 2MB.

Type-Safe Themes

Each theme exports strongly-typed color enums and layer configurations:

import type { ReactAvatarItem } from '@avatune/types'
import { createTheme, fromHead } from '@avatune/theme-builder'

const theme = createTheme()
  .withStyle({ size: 500, borderRadius: '100%' })
  .addColors('hair', [HairColors.Black, HairColors.Brown, HairColors.Blond])
  .addColors('skin', [SkinTones.Light, SkinTones.Medium, SkinTones.Dark])
  .connectColors('head', ['ears', 'nose'])  // ears + nose inherit head's skin color
  .setOptional('glasses')
  .mapPrediction('hairColor', 'brown', [HairColors.Brown, HairColors.Auburn])
  .toFramework<ReactAvatarItem>()
  .build()

TypeScript knows exactly which colors and items are valid for each theme. No more runtime surprises.

Custom Rsbuild Plugins for SVG → Component

When you have 500+ SVG files across multiple frameworks, you hit a problem: SVG id and mask collisions. Put two avatars on a page, and their internal gradients and masks conflict.

I built two Rsbuild plugins to solve this:

@avatune/rsbuild-plugin-svg-to-svelte and @avatune/rsbuild-plugin-svg-to-vue

They transform SVG files into proper framework components with:

• Auto-prefixed IDs via SVGO (no collisions)
• Preserved viewBox for responsive scaling
• Query-based imports for flexibility

<script>
  import Icon from './icon.svg?svelte'
</script>

<Icon />

<script setup>
import Icon from './icon.svg?vue'
</script>

<template>
  <Icon />
</template>

The Stack

• Turborepo - Monorepo orchestration with caching
• Bun - Package manager (2x faster installs)
• Rspack/Rslib - Build tooling (10x faster than webpack)
• Biome - Linting + formatting (replaces ESLint + Prettier)
• uv - Python package manager (10-100x faster than pip)

10 themes, 5 framework renderers (React, Vue, Svelte, Vanilla, React Native), 5 ML predictors. All from one monorepo.

Try It

• Website + Docs: avatune.dev
• GitHub: github.com/avatune/avatune
• Playground: avatune.dev/docs/playground

npm install @avatune/react @avatune/pacovqzz-theme

The ML models are experimental - I'd love feedback on accuracy and performance. If you're into training better attribute predictors, PRs welcome.