Unearthing the Ghosts of the Early Internet with Local AI 🕯️

#devchallenge #hf26challenge #ai #opensource

The Internet Left a Trail. I Built an AI Séance to Follow It. 🕯️

What if we could investigate the forgotten corners of the early internet—not through nostalgia or guesswork, but through the evidence that archived webpages actually left behind?

That question inspired Web Séance, my project for Hacktoberfest 2026 Week 4: Ghost in the Machine.

🎬 A Glimpse Into Web Séance

This animated preview walks through the project's six main stages. It is a visual walkthrough of the experience; the full application and its source code are available below.

Interactive demo walkthrough: https://kaushanikoner20-hub.github.io/Web-Seance/demo.html
GitHub repository: https://github.com/kaushanikoner20-hub/Web-Seance

🕸️ What I Built

Web Séance is an AI-assisted digital archaeology tool for exploring forgotten internet culture through historical webpages preserved by the Internet Archive.

The idea is simple: archived pages contain traces of how the early web connected. Outgoing hyperlinks can reveal references, shared topics, and paths worth investigating—but they cannot prove that two people knew each other or collaborated.

Web Séance tries to preserve that distinction.

The application helps users:

Retrieve historical webpage captures using the Internet Archive's CDX API.
Extract outgoing hyperlinks from archived HTML.
Normalize and classify links.
Assign traceable Evidence IDs to extracted links.
Explore observed hyperlink trails through a D3.js network graph.
Ask a locally running Gemma model to interpret the collected evidence.

The goal isn't to manufacture a story about the past. It's to make the available evidence easier to inspect.

🔬 How It Works

1. Retrieve the archive

The application uses the Internet Archive CDX API to locate historical captures and retrieve archived HTML.

2. Extract evidence

BeautifulSoup is used to parse archived HTML and extract outgoing hyperlinks. The extracted links receive identifiers so they can be traced back to their source page.

3. Explore connections

The application visualizes observed hyperlink relationships in a D3.js network graph. Each connection represents a link found in the archived page—not an assumed personal relationship.

4. Ask the local AI historian

Gemma 3:4b, running locally through Ollama, helps interpret the evidence. The system checks whether cited Evidence IDs exist in the collected evidence set.

That check is useful, but it is not a guarantee that every AI-generated interpretation is factually correct. The distinction between evidence, hypotheses, and unknowns remains important.

🧠 Why Gemma?

I used Gemma 3:4b through Ollama as the local AI historian.

I wanted the model to work with evidence collected by the application rather than inventing historical connections from vague prompts.

The intended workflow separates:

Confirmed facts: What the archived HTML and extracted links directly support.
Hypotheses: Possible interpretations that need further investigation.
Unknowns: Questions the available evidence cannot answer.

A hyperlink is evidence that a page linked somewhere. It is not proof of friendship, endorsement, or collaboration.

🛠️ Tech Stack

Python — application logic
Streamlit — user interface
BeautifulSoup — HTML parsing and link extraction
Requests — HTTP requests
Pandas — structured data handling
D3.js — network graph visualization
Internet Archive CDX API — historical capture discovery
Ollama + Gemma 3:4b — local AI interpretation

🌍 Why Does Open Innovation Matter?

The early internet was built through shared ideas, open standards, personal websites, mailing lists, and communities that often worked in public.

A lot of that culture is now scattered across old domains and archived pages.

Open-source tools can help people explore and preserve those traces. They also make it possible for others to inspect the methodology, question the conclusions, improve the implementation, and build on the work.

For me, this project is a small experiment in combining open web archives with open AI while keeping evidence and interpretation separate.

🕯️ The Spirit of the Project

The internet doesn't remember everything.

Some pages disappear. Some links break. Some histories become difficult to reconstruct.

But sometimes, a small trace remains.

Web Séance is an attempt to follow those traces responsibly—with archival evidence first, AI interpretation second, and uncertainty always visible.

If you're interested in digital archaeology, forgotten internet culture, open-source AI, or the history of the web, I'd love for you to explore the project and share your thoughts.