Building a Pixel-Art AI Interrogation Game with Rust, Tauri, and Memvid

I wanted an interrogation game where AI dialogue feels dynamic, but evidence remains immutable.

That led to this model:

• The suspect can bluff in conversation.
• The player can challenge claims.
• Memvid .mv2 memory acts as the source of truth.

What We Built

The app now combines two layers:

1. Forensic retrieval layer (Memvid-backed search/timeline)
2. Pixel-art game layer (interrogation room, sprites, speech bubbles, stress meter)

The result is less “debug dashboard” and more “interactive detective scene.”

Stack

• Rust + Tauri 2
• memvid-core with lex, vec, temporal_track
• React + TypeScript + Vite
• vis-timeline
• @fontsource/press-start-2p for retro pixel typography

High-Level Architecture

Rust Backend: Command Design

src-tauri/src/lib.rs exposes three key commands:

• generate_suspect_memory
• search_suspect_memory
• load_suspect_timeline

Search command snippet

let response = memory.search(SearchRequest {
    query: trimmed.to_string(),
    top_k: top_k.unwrap_or(12).clamp(1, 100),
    snippet_chars: 220,
    uri: None,
    scope: None,
    cursor: None,
    temporal: None,
    as_of_frame: None,
    as_of_ts: None,
    no_sketch: false,
    acl_context: None,
    acl_enforcement_mode: AclEnforcementMode::Audit,
})?;

Frontend: Pixel-Art Room + Evidence UI

The scene is composed from custom sprite maps and palette dictionaries rather than raster assets.

Sprite approach

const DETECTIVE_SPRITE = [
  '..111111..',
  '.12222221.',
  '.12333221.',
  '..1ffff1..',
  // ...
]

A reusable PixelSprite component renders rows/cells into blocks, allowing palette swaps, animation, and stress-state effects.

Fast Investigation UX

The original frame-by-frame investigation felt slow and unclear. We replaced it with burst scanning.

Burst scan loop

const batchSize = 16
const tickMs = 60

const timer = window.setInterval(() => {
  const end = Math.min(timeline.length, progress + batchSize)
  setScanProgress(end)
  setSelectedTimelineIndex(Math.max(0, end - 1))
  // append contradiction candidates found in this batch
}, tickMs)

Why this works better

• The player sees immediate momentum.
• Progress and contradiction counts are explicit.
• Contradiction feed is clickable and evidence-driven.

Interaction Model

What Developers Can Build Next

Gameplay

• Claim-vs-contradiction adjudication mode
• Stress-driven branching with blade-ink
• Evidence pinning board with React Flow

AI

• Memory Oracle with OpenAI/Ollama RAG responses
• Contradiction severity classifier
• Better temporal reasoning on suspect statements

Visuals

• More sprite states (talking, sweating, breakdown)
• Animated tile map room sets
• CRT/VHS post-processing overlays

Final Takeaway

The key pattern is separating:

• Behavioral AI layer (dialogue can mislead)
• Immutable memory layer (retrieval is authoritative)

Once you enforce that boundary, interrogation mechanics become both fun and technically robust.

Github Repo: https://github.com/harishkotra/memento.os