This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine
What I Built
The Midnight Archivist is a local-first literary archive and AI-powered storytelling application.
The idea was simple: What if forgotten documents could tell new stories?
Users can import PDF, TXT, and Markdown documents into a private archive, search their contents by meaning, and use relevant passages as source material for AI-generated fiction.
The application supports two writing modes:
- Archive-grounded generation: Creates stories using passages retrieved from the user's documents, with source references.
- Freeform generation: Creates stories from a premise and selected atmosphere without requiring archive material.
It also supports multiple writing atmospheres, three story-length presets, and branching narratives. Readers can make choices at the end of a story and explore different continuations without losing the original narrative.
I interpreted the Ghost in the Machine theme through digital memory. The ghost is a forgotten voice in an old manuscript, a memory buried in a document, or a fragment of history that a machine can retrieve and give a new context.
The goal was to build something that does more than generate text: it gives old documents a way to inspire new stories.
Demo
The Midnight Archivist runs locally rather than relying on a hosted AI service.
Demo video: https://youtu.be/H06pOBN4Nk0
The walkthrough will demonstrate document importing, semantic search, archive-grounded story generation, branching narratives, and story export.
Code
GitHub Repository: The Midnight Archivist
The repository includes the complete React frontend, FastAPI backend, document-processing pipeline, local AI integration, automated tests, screenshots, and installation instructions.
License: MIT for the original application source code. Model and third-party licenses are documented separately in THIRD_PARTY_NOTICES.md.
How I Built It
The project uses a React frontend, Python backend, SQLite database, and Ollama for local AI inference.
Technology stack:
| Layer | Technology |
|---|---|
| Frontend | React 18, TypeScript, Vite |
| Backend | Python, FastAPI, Pydantic |
| Database | SQLite, SQLAlchemy |
| Document processing | pypdf, Markdown and TXT parsers |
| Embeddings | nomic-embed-text |
| Story generation | llama3.2:3b |
| AI runtime | Ollama |
| Retrieval | NumPy cosine similarity |
1. Document ingestion and semantic search
Uploaded documents are parsed and divided into smaller passages. Each passage is converted into a 768-dimensional embedding using nomic-embed-text through Ollama.
When a user enters a search query, the application creates its embedding and compares it with stored document embeddings using cosine similarity. This allows users to retrieve passages based on meaning instead of relying only on exact keyword matches.
2. Local AI story generation
The generation engine uses Meta's Llama 3.2 3B model through Ollama.
For archive-grounded stories, the system retrieves relevant passages and supplies them as source material to the generation engine. For freeform stories, it uses the user's premise and selected atmosphere.
Long stories use a multi-stage generation process with a three-act outline, sequential generation, length validation, and bounded correction passes.
3. Branching narratives
Stories and their continuations are stored as parent-child relationships in SQLite. Each continuation creates a new branch rather than overwriting the original story.
The Reading Room reconstructs the selected narrative path, allowing readers to explore multiple possible directions.
4. Testing and verification
The final release verification recorded:
- 151 passing backend tests.
- Seven passing Playwright end-to-end workflows.
- Successful frontend TypeScript checks and production build.
- Local Ollama inference testing.
- Responsive layout checks for desktop, tablet, and mobile.
Why Does Open Innovation Matter?
Open innovation matters because this application works with material that can be deeply personal: diaries, unpublished manuscripts, research notes, and historical documents.
Sending such documents to a hosted AI provider is not always desirable.
By using open-weight models with Ollama, the project keeps document processing, embedding generation, retrieval, and story generation on the user's own machine.
It also avoids dependence on a paid hosted AI API or an external vector database. SQLite handles persistence, while NumPy performs similarity calculations.
Open models make experimentation possible. The embedding and generation models can be changed independently, and developers can inspect and modify the surrounding pipeline.
The result is more control over data, model selection, and the complete AI workflow.
Initial setup requires downloading dependencies and models. After setup, the normal inference and retrieval workflow runs locally.
Top comments (0)