Edgi Axel

Posted on Oct 31 Edited on Oct 31 AI-assisted

The Ghost Doesn't Need the Cloud: Building a Local AI Music Curator with Gemma 3

#devchallenge #hf26challenge #ai #opensource

Hacktoberfest Open-Source AI Challenge: Ghost in the Machine Submission 👻🌼

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine

What I Built

GhostWave is an open-source, local-first music player, crate manager, and audio-reactive visualizer with a Halloween-inspired design.

It is built around a simple idea: your music library should remain useful even when the cloud isn't.

Most of us have personal music collections that have accumulated over the years: FLAC albums, MP3 downloads, anime soundtracks, Vocaloid tracks, live recordings, and music that might not even be available on streaming platforms anymore.

But having the files is only half the problem. Libraries become messy. Metadata goes missing. Filenames become inconsistent. Finding the right tracks for a particular mood can take longer than actually listening to them.

GhostWave brings those tasks together in one application:

Local Music Library: Index and organize MP3, FLAC, WAV, OGG, and AAC files in a searchable library.
Audio Player & Visualizer: Web Audio API playback, queue management, and reactive visualizers with Halloween-inspired themes.
Local AI Playlist Curator: Use Google's Gemma 3 4B through Ollama to create playlist proposals from natural-language requests.
Library Health: Identify missing metadata, filename inconsistencies, and duplicate files, with proposed changes available for review.
Audio Conversion: Non-destructive conversion to 16-bit PCM WAV.
Listening History: Local playback tracking, skip statistics, and transparent music recommendations.
Optional Metadata Enrichment: MusicBrainz, Cover Art Archive, LRCLIB, and SerpApi integrations for finding additional metadata.

The central feature is the Ghost in the Machine: an open-weight AI model that can help organize your music collection without requiring a remote LLM API.

Demo

Watch the product film:

Links:

YouTube Product Film: https://youtu.be/dPluGwZ3hds
Live Web App: https://ghostwave.onrender.com
GitHub Repository: https://github.com/edgiaxel/GhostWave
Product Film Source: Remotion video in the repository
Original Automated Walkthrough: Playwright recording

The product film combines animated product storytelling with recorded application footage. The external metadata responses shown in the scripted walkthrough are demonstration fixtures, not evidence that those particular lookups were performed against live services.

Code

edgiaxel / GhostWave

Open source local music player with local AI Integration.

🎃 GhostWave — Local-First Music Player & AI Crate Curator

A spooky, local-first music player, crate manager, and audio-reactive visualizer powered by local open-weight AI (Gemma 3 via Ollama).

The Origin Story • Key Features • Architecture • Getting Started • Local AI Setup • Tauri Desktop • QA & Testing • Deployment

⚡ The "Ghost in the Machine" Origin Story

(An imagined product scenario inspired by seasonal thunderstorms and connectivity concerns)

Imagine Halloween Eve. Outside, a violent seasonal thunderstorm batters the neighborhood. The grid blinks—and the house plunges into darkness.

When power snaps back on ten minutes later, the internet connection is dead. A gathering of friends is waiting for music, but the host's phone streaming app hangs on a spinning wheel: buffering error 404, no connection.

An offline backup drive of local music is plugged in—thousands of anime themes, Vocaloid tracks, electronic synths, and classical recordings. But…

View on GitHub

The most interesting part of the project is how GhostWave turns an AI-generated playlist idea into a proposal grounded in the user's actual music library.

Here are two important parts of that implementation.

1. Serialize only the metadata needed by the model

Instead of sending audio files or local file paths to the AI, GhostWave builds a compact catalog containing track metadata:

static serializeLibrary(
  tracks: Track[]
): CompactTrackSummary[] {
  return tracks.map((t) => ({
    id: t.id,
    title: t.title,
    artist: t.artist,
    album: t.album,
    genre: t.genre?.length
      ? t.genre
      : undefined,
    durationSeconds: t.durationSeconds,
    format: t.format,
    bitrateKbps: t.bitrateKbps,
    year: t.year,
  }));
}

This gives the model enough context to make selections without including the actual audio bytes or filesystem paths.

2. Resolve AI suggestions against real library records

An LLM can generate an ID that looks valid but does not exist. GhostWave doesn't blindly trust the response.

The service resolves proposed IDs against the actual library:

const trackMap = new Map(
  allTracks.map((track) => [track.id, track])
);

const matched: Track[] = [];
const hallucinatedTrackIds: string[] = [];

for (const id of plan.matchedTrackIds) {
  const track = trackMap.get(id);

  if (track) {
    if (!matched.some((item) => item.id === track.id)) {
      matched.push(track);
    }
  } else {
    hallucinatedTrackIds.push(id);
  }
}

Unknown IDs are rejected, duplicate selections are avoided, and duration and track-count constraints are enforced by application code rather than trusting the model's arithmetic.

The result is a playlist proposal, not an unrestricted AI operation. The user can inspect the selected tracks before saving the crate.

Relevant implementation: AiPlaylistService.ts.

How I Built It

GhostWave uses a modular TypeScript architecture with React on the frontend and a collection of focused services for audio, AI, metadata, history, and platform integration.

Technology stack:

Frontend: React 18, TypeScript, Vite, Tailwind CSS.
Audio: Web Audio API and Canvas 2D visual rendering.
Metadata: music-metadata-browser.
Local Storage: IndexedDB.
AI: Google Gemma 3 4B (gemma3:4b) through Ollama.
Desktop: Tauri 2 and Rust.
Testing: Vitest, Testing Library, and Playwright.
Observability: Optional Sentry integration.

The local AI pipeline

The AI workflow is deliberately separated into several stages:

The user enters a natural-language playlist request.
GhostWave serializes the local library into compact track metadata.
The prompt and catalog are sent to the configured Ollama endpoint.
Gemma 3 generates a structured playlist plan.
GhostWave parses and validates the response.
Proposed track IDs are resolved against the actual IndexedDB library.
The application calculates the final duration and presents the proposal for user review.

The model is instructed to return a structured response resembling this:

{
  "name": "Fast Electronic Workout",
  "description": "High-energy electronic tracks",
  "matchedTrackIds": [
    "track-001",
    "track-014",
    "track-027"
  ],
  "explanation": "Selected for energetic genres and tempo."
}

The response is not accepted just because it is valid JSON. The application checks its required fields and verifies the proposed IDs against real track records.

Running Gemma 3 locally

GhostWave uses Ollama's local API rather than a hosted LLM API.

# Download the model
ollama pull gemma3:4b

# Start the Ollama service
ollama serve

The default local endpoint is:

http://127.0.0.1:11434

The model must be installed and Ollama must be running for local AI features to work. The core music library and playback features do not depend on Gemma.

Testing and quality assurance

The project currently reports:

121 Vitest tests across 12 test suites.
30 Playwright end-to-end tests across 10 test suites.

The automated checks cover library operations, playback, visualizers, playlist management, AI workflows, metadata handling, and other application features.

Some external-provider responses are mocked in automated tests so that the tests remain reproducible. This is different from testing against live third-party services.

Why Does Open Innovation Matter?

Music collections are personal. They contain more than just audio files: they preserve memories, interests, and years of discovery.

A closed AI service can be convenient, but it also introduces dependencies. Your application may depend on an external API, its pricing, its availability, and the data you must send to use it.

Open-weight models and open-source tools give developers another option.

With Gemma 3 and Ollama, GhostWave can perform its playlist-curation inference locally. The model does not need a remote LLM subscription, and the catalog sent to that local inference process does not need to include audio files or filesystem paths.

It also means developers can inspect and adapt the implementation instead of treating the intelligence as an inaccessible service.

There are still important boundaries:

Local AI requires a compatible machine, an installed model, and a running Ollama service.
Optional metadata providers require internet access.
Optional Sentry diagnostics can transmit sanitized telemetry.
SerpApi uses a user-provided key stored in browser-side settings and sent directly to its service. It is not protected by a server-side secret proxy.

Open innovation does not automatically make every integration private or offline. It gives developers the tools and freedom to build systems where those boundaries can be understood, inspected, and controlled.

For GhostWave, that means keeping the core music experience local while making external services optional.

The Spirit of It

An imagined scenario inspired by seasonal thunderstorms and connectivity concerns.

Imagine Halloween Eve.

You're hosting a small gathering with friends. The weather outside is getting rough, and the internet connection suddenly becomes unreliable.

Someone tries to start a playlist through a streaming service. It buffers. Then it stops.

Fortunately, you have a local music collection: years of Vocaloid tracks, anime soundtracks, synthwave, electronic music, and other recordings.

But there's another problem. The library is messy. Some tracks have missing tags, filenames are inconsistent, and finding the right music for the evening is becoming a project of its own.

That's the situation GhostWave was designed around.

You open your local library, ask Gemma 3 for a spooky electronic playlist, review its suggestions, and start playing. The visualizer responds to the music while the rest of the gathering carries on.

No dramatic cloud infrastructure required. Just your music, your machine, and a little ghost in the machine.

Prize Categories

Hacktoberfest Open-Source AI Challenge — Ghost in the Machine
- An open-source music application with a Halloween-inspired visual identity and local AI-powered playlist curation.
Google Cloud / Gemma
- Uses Google's open-weight Gemma 3 4B through Ollama for local natural-language playlist generation and structured output.
Sentry
- Integrates Sentry for optional application diagnostics and AI lifecycle monitoring, with sanitization of sensitive application data before telemetry dispatch.
SerpApi
- Integrates SerpApi as an optional metadata research provider, returning candidate information for human review.

Built with React, TypeScript, Gemma 3, Ollama, and a lot of love for local music libraries.

Thanks to DEV, GitHub, and MLH for organizing Hacktoberfest 2026!

DEV Community