DEV Community

Cover image for Token Factory: Making LLM Inference and Memory Visible
Harish Kotra (he/him)
Harish Kotra (he/him)

Posted on

Token Factory: Making LLM Inference and Memory Visible

#supermemory #ai #programming #dailybuild2026

How we built an interactive 3D educational app that turns tokens, decoding, and Supermemory into a living factory — and why memory is the spine, not a side feature.

TL;DR

Token Factory is a Next.js + React Three Fiber web app that teaches how large language models generate text by animating every step as physical “token cubes” moving through industrial rooms: tokenizer, embeddings, logits, softmax, temperature, top-k, top-p, sampling, context window, KV cache, speculative decoding — bookended by a Supermemory vault that recalls crystals before generation and stores conversations after.

Repo: github.com/harishkotra/token-factory

npx supermemory local          # required memory layer on :6767
npm install && cp .env.example .env.local
# set SUPERMEMORY_API_KEY=sm_...
npm run dev
Enter fullscreen mode Exit fullscreen mode

1. The problem we set out to solve

Most explanations of LLM inference are either:

  1. Walls of math — logits, softmax, nucleus sampling as formulas, or
  2. Black-box chat UIs — a text box, a spinner, a paragraph.

Neither builds intuition. Temperature feels abstract until you see a distribution flatten. Top-p is hard until you watch a probability bucket fill and stop. Memory layers sound like “RAG with better marketing” until a crystal flies from a vault into the context conveyor.

We wanted something closer to:

  • Factorio (systems you can watch)
  • Brilliant.org (learn by interaction)
  • Apple onboarding (motion does the explaining)

…with a serious technical point: modern agents aren’t just decoders; they are memory-augmented systems. So Supermemory Local (localhost:6767) is not an optional plugin. It is the product’s core infrastructure.

2. Product framing

Elevator pitch

Instead of invisible mathematics, every token becomes a physical cube moving through an AI factory. Users don’t read about AI — they watch it happen. And the factory remembers.

Core loop (always) 

User prompt
  → Supermemory search + profile
  → Enriched context (memory / profile / prompt cubes)
  → Educational decoding pipeline (rooms)
  → Append tokens → answer streams in the Response panel
  → Persist episodic crystal → vault refresh
Enter fullscreen mode Exit fullscreen mode

There is no “disable memory” toggle in production UX. Decoding knobs (temperature, top-k/p, KV cache, speculative) sit on top of a memory spine that never turns off.

3. High-level architecture 

┌─────────────────────────────────────────────────────────────────┐
│  Browser (Next.js App Router)                                    │
│                                                                  │
│  ConnectionGate ──► connection-store (health must be green)      │
│  HUD: TopBar · StageChrome · ResponsePanel · PromptDock · Vault  │
│  FactoryScene (R3F / Three.js) ◄── factory-store timeline        │
│         │                                                        │
│         │  ALWAYS: recall → decode rooms → persist               │
└─────────┼───────────────────────────────────────────────────────┘
          │  /api/health  /api/memory  /api/memory/search
          │  /api/seed    /api/profile
          ▼
┌─────────────────────────────────────────┐
│  MemoryService (src/lib/supermemory)    │
│  Official supermemory SDK               │
│  baseURL → http://localhost:6767        │
└──────────────────┬──────────────────────┘
                   ▼
         supermemory-server
         (graph · embeddings · auth)
Enter fullscreen mode Exit fullscreen mode

Why this split?

Layer Responsibility
Zustand stores Orchestrate timed animations + UI state
Next API routes Keep the Supermemory API key server-side
MemoryService Single source of truth for health, seed, search, list, persist
llm-simulator + answer-planner Educational decode + coherent demo answers
R3F scene Visual rooms; never empty when idle

Failure policy is intentional and strict:

  • Missing API key → misconfigured gate
  • Supermemory down → disconnected gate; Start disabled
  • Recall fails mid-run → abort (no fake success)
  • Tour mode exists only as camera-only, labeled non-generative

4. Tech stack

Concern Choice Why
App framework Next.js 16 (App Router), TypeScript Fast API routes + React UI
Styling Tailwind CSS v4 Utility-first HUD; glass panels
3D Three.js, React Three Fiber, Drei Declarative scenes, OrbitControls
Motion Framer Motion Drawer / gate / response transitions
State Zustand Lightweight store for long generation timelines
UI primitives Radix + Lucide Accessible sliders, switches
Memory supermemory SDK → :6767 Local-first Memory API for the hackathon

Important Tailwind note for anyone cloning: do not let Tailwind v4 auto-scan binary dirs. Supermemory’s local DB under .supermemory/data can produce invalid CSS if scanned. We restrict sources in globals.css:

@import "tailwindcss";

@source not "../../../.supermemory";
@source not "../../../.next";
@source not "../../../node_modules";
@source "../../**/*.{js,ts,jsx,tsx,mdx}";
Enter fullscreen mode Exit fullscreen mode

5. Factory rooms (what each mode teaches)

Each room has a short blurb shown in stage chrome. The design rule: one sentence, never a paragraph.

Room Teaching moment
Memory Vault Crystals = conversations / facts; search before generate
Tokenizer Text → token cubes (incl. crude subword splits)
Embedding Glow = “vector of meaning” (no raw numbers)
Transformer Attention atmosphere — gears & beams, not matrix dumps
Logits Candidates scored; taller/brighter = higher score
Softmax Scores → probabilities that sum to 100%
Temperature Low T peaks; high T flattens creativity
Top-K Only K survive the laser gate
Top-P Fill a probability bucket until ≥ P
Sampling One token chosen (lottery claw)
Append Winner joins the answer string; loop
Context Finite window; oldest fall off
KV Cache Green stamps skip recompute
Speculative Small draft vs large verify

Idle rooms still show demo cubes/props so navigation never lands on an empty floor. During a run, only the active room ± neighbors mount heavy content to reduce flicker and cost.

6. Memory layer: Supermemory as infrastructure

Client configuration 

// src/lib/supermemory/client.ts (conceptual)
import Supermemory from "supermemory";

export function getSupermemoryClient() {
  return new Supermemory({
    apiKey: process.env.SUPERMEMORY_API_KEY!,
    baseURL: process.env.SUPERMEMORY_BASE_URL || "http://localhost:6767",
  });
}
Enter fullscreen mode Exit fullscreen mode

Operations we implement 

// Conceptual MemoryService surface
checkHealth()           // documents.list({ limit: 1 }) + latency
ensureSeeded()          // idempotent customId seeds (Phaser, tokens, …)
searchAndProfile(q)     // search.memories + profile
listVault()             // documents.list for the Vault UI
persistGeneration(...)  // client.add with episodic metadata
Enter fullscreen mode Exit fullscreen mode

Seed memories

On first successful connect, we upsert semantic seeds such as:

  • Phaser is a 2D HTML5 game framework
  • Tokens are subword pieces of text
  • Temperature controls randomness
  • KV cache speeds up autoregressive inference

So demos like “Tell me about Phaser” hit real search scores, not a hardcoded offline array as the source of truth.

API routes

Route Role
GET /api/health Connection badge + gate
GET/POST /api/memory List vault / persist generation
POST /api/memory/search Recall for the pipeline
POST /api/seed Idempotent demo seeds
GET /api/profile Static/dynamic profile lines

Keys never ship to the browser; the client only talks to same-origin Next routes.

7. Generation orchestration

The heart of the experience is runGeneration() in the factory store: a timed state machine that advances room, phase, candidates, and generated[] with sleeps so the camera and 3D scene can follow.

Pseudocode:

async function runGeneration() {
  await ensureSupermemoryConnected(); // else abort + gate

  // 1) Recall
  const { memories, profile } = await search(prompt);

  // 2) Plan a coherent answer from memory (educational sim)
  const plan = planAnswerTokens(prompt, memories, maxTokens);

  // 3) Build context cubes: profile + memory + prompt
  let context = mergeContext(profile, memories, prompt);

  // 4) For each planned token: animate the full decode path
  for (const forced of plan) {
    showRoom("transformer");
    let cubes = makeCandidateCubes(context, 8, { forcedWinner: forced });
    cubes = applySoftmax(cubes, temperature);
    cubes = applyTopK(cubes, topK);
    cubes = applyTopP(cubes, topP).cubes;
    const picked = sampleOne(cubes, { forcedWinner: forced, greedy: true });
    append(picked.text);
    context = trim(context + picked, contextWindow);
  }

  // 5) Persist episodic crystal
  await persist({ prompt, completion: generated.join(" ") });
  await refreshVault();
}
Enter fullscreen mode Exit fullscreen mode

Why an “answer planner”?

Early prototypes sampled purely from a toy vocabulary. That produced vault entries like:

robot accept of reject token token

Cute for chaos mode; terrible for demos. The answer planner turns top Supermemory hits (or heuristics) into a token sequence, while the factory still shows logits → filters → sampling. The planned token is injected as the highest-logit candidate so UI lessons stay true and the Response panel stays readable.

// src/lib/answer-planner.ts (excerpt)
export function planAnswerTokens(
  prompt: string,
  memories: MemoryCrystal[],
  maxTokens = 20
): string[] {
  if (memories.length > 0) {
    const best = /* highest score/importance */;
    const words = cleanWords(best.content || best.summary);
    if (words.length >= 3) return words.slice(0, maxTokens);
  }
  // heuristics for Phaser, temperature, tokens, …
  // soft fallback
}
Enter fullscreen mode Exit fullscreen mode

Softmax (educational core) 

export function applySoftmax(cubes: TokenCube[], temperature: number) {
  const T = Math.max(0.05, temperature);
  const scaled = cubes.map((c) => c.logit / T);
  const max = Math.max(...scaled);
  const exps = scaled.map((z) => Math.exp(z - max));
  const sum = exps.reduce((a, b) => a + b, 0);
  return cubes.map((c, i) => ({
    ...c,
    probability: exps[i] / sum,
    intensity: exps[i] / sum, // drives cube height / glow
  }));
}
Enter fullscreen mode Exit fullscreen mode

Top-k and top-p then prune alive flags for the laser gate and bucket animations.

8. 3D and UX engineering notes

Contained stage

Early versions used full-viewport absolute HUD over a full-bleed Canvas. Result: overlapping text, unreadable “AI is responding,” and chaotic flicker as the camera teleported through 14 rooms.

Current layout:

┌ Top bar (logo, connection, Help) ─────────────┐
│  Contained 3D stage (rounded frame)           │
│  + StageChrome (mode blurb, zoom, room rail)  │
├ Response panel (prompt + streaming answer) ───┤
└ Prompt dock (Start / Decode / Levels) ────────┘
Enter fullscreen mode Exit fullscreen mode

Camera & zoom

OrbitControls owns the camera. Room changes and stageZoom set target + distance once (effect-driven), instead of fighting scroll zoom every frame:

// Conceptual camera update
const look = new THREE.Vector3(...meta.lookAt);
const baseCam = new THREE.Vector3(...meta.camera);
const dist = baseCam.distanceTo(look) * stageZoom;
camera.position.copy(look.clone().add(dir.multiplyScalar(dist)));
controls.target.copy(look);
controls.update();
Enter fullscreen mode Exit fullscreen mode

Buttons: zoom in (stageZoom /= 1.2), zoom out (*= 1.2), reset (1). Scroll still works via OrbitControls.

Connection UX

ConnectionGate blocks the experience until Supermemory is healthy, with copy-paste steps for npx supermemory local and .env.local. The connection badge shows live latency (e.g. Supermemory · live · 37ms).

9. Project layout (for navigators) 

src/
  app/api/{health,memory,memory/search,seed,profile}/
  components/
    factory/          # R3F scene, rooms, cubes, conveyor
    hud/              # gate, badge, stage chrome, response, help, vault
    ui/               # button, slider, switch, label
  lib/
    supermemory/      # client, config, service
    llm-simulator.ts  # tokenize, logits, softmax, top-k/p, sample
    answer-planner.ts # memory-grounded token plans
    rooms.ts levels.ts types.ts
  store/
    factory-store.ts
    connection-store.ts
ARCHITECTURE.md
Enter fullscreen mode Exit fullscreen mode

10. What we deliberately did not do

  • No silent offline memory store as source of truth — demos must exercise Supermemory.
  • No hosted LLM requirement for the factory path — keeps the hackathon demo offline-friendly (except local SM) and focused on pedagogy.
  • No walls of documentation in-UI — Help drawer + one-line mode blurbs only.

A future path is clear: swap the answer planner for a real model API while keeping the same visual decode path and Supermemory spine.

11. Try it yourself 

git clone https://github.com/harishkotra/token-factory.git
cd token-factory
npx supermemory local          # copy sm_ key
cp .env.example .env.local     # paste key
npm install && npm run dev
Enter fullscreen mode Exit fullscreen mode

Suggested first run:

  1. Wait for Supermemory · live
  2. Prompt: Tell me about Phaser
  3. Watch vault → context colors (amber memory / violet profile / cyan prompt) → decode rooms
  4. Read the Answer panel
  5. Open Vault — new episodic crystal

12. Closing thought

Token Factory is a bet that systems literacy for AI needs the same visual language we use for games and product design. If a student can feel why temperature changes creativity in thirty seconds, and see a memory crystal join the context conveyor, we’ve done more than ship a hackathon demo — we’ve made the invisible pipeline a place you can walk through.

Build something you can point a camera at. Then make it remember.

Code & more: https://www.dailybuild.xyz/project/196-token-factory

Top comments (0)