DEV Community: Shivam Singh

LastWhisper : Search your voice

Shivam Singh — Fri, 30 Oct 2026 12:32:48 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine

What I Built

I built LastWhisper, a local-first voice memory archive that helps people preserve, search, and revisit the stories hidden inside their recordings.

The idea behind it is simple: a voice recording can outlive the moment, place, or even the person who created it. That felt like a natural interpretation of Ghost in the Machine. The “ghost” is not an AI pretending to be someone. It is the real voice, story, and memory that already exists inside the recording. The machine simply helps you find it again.

With LastWhisper, a user can:

Import WAV, MP3, M4A, or FLAC recordings into a private local archive.
Transcribe audio locally with open Whisper inference.
Read timestamped transcript segments and click one to seek the original audio.
Search naturally, even without remembering the exact words used in the recording.
Inspect genuine transcript evidence instead of receiving an invented answer.
Open a search result at its exact source timestamp.
Save important evidence as a memory and add a personal title or note.
Rebuild the semantic index without requiring a separate vector database.
Optionally generate transcript-grounded summaries with a locally configured Ollama model.

The default setup does not require a hosted transcription API, embedding API, cloud LLM, analytics service, or remote storage.

Code

Th-Shivam / Lastwhisper

LastWhisper

LastWhisper is a local-first voice memory archive. It imports personal audio, transcribes it with open-source Whisper inference, indexes transcript evidence with local sentence-transformer embeddings, and lets people search, replay, and save source-grounded memories.

No hosted transcription, embedding, LLM, analytics, or telemetry service is required. Model files may need a one-time download before offline use.

Features

Private local audio archive for WAV, MP3, M4A, and FLAC files.
CPU-first transcription with faster-whisper, language detection, progress, retry, and cancellation.
Native playback and timestamped transcript segments that seek into the source audio.
Optional source-grounded summaries through a user-configured local Ollama model.
Local semantic search with normalized sentence-transformer embeddings stored in SQLite.
Evidence snippets with recording IDs, segment IDs, dates, and source timestamps.
Saved memories with immutable source evidence plus editable titles and notes.
Rebuildable, transcript-revision-aware search index.
Additive migration support for existing Phase 1, Phase 2, and early Phase 3 databases.
Responsive…

View on GitHub

The repository includes:

Native CPU installation instructions.
Docker Compose with persistent archive and model-cache volumes.
Architecture and privacy diagrams.
Model and third-party license documentation.
CI, dependency audits, migration coverage, and offline unit tests.
A separate opt-in integration test for real cached AI models.
Contribution, security, and code-of-conduct documentation for public open-source development.

How I Built It

The frontend uses React, TypeScript, Vite, Tailwind CSS, and Lucide. It communicates with a loopback FastAPI backend. SQLAlchemy and SQLite persist recordings, transcript segments, semantic chunks, saved memories, and embedding metadata. Original audio is stored outside the web root under generated UUID filenames.

Local transcription

LastWhisper uses faster-whisper with CTranslate2. The verified default path uses the pinned SYSTRAN faster-whisper-base conversion on CPU with int8 compute.

The worker stores:

Detected language.
Ordered transcript text.
Start and end timestamps for every source segment.
Processing progress and safe retry/cancellation state.

Clicking a transcript segment sets the audio player to that segment's real Whisper timestamp.

Local semantic search

For retrieval, LastWhisper uses sentence-transformers with the Apache-2.0 paraphrase-multilingual-MiniLM-L12-v2 model pinned to an immutable revision.

Long transcripts are split into bounded overlapping chunks. Every chunk remains linked to its recording and contributing transcript segment IDs. The generated 384-dimensional vectors are normalized and stored directly in SQLite as little-endian float32 BLOBs alongside:

Model and revision metadata.
Vector dimension.
Transcript fingerprint.
Source segment IDs.
Faithful source text.
Start and end timestamps.

Search embeds the query locally, computes cosine similarity, applies a configurable threshold, and returns ranked transcript evidence. The score is presented as a ranking signal—not a probability or guarantee of correctness.

Source-grounded memories

Saving a result creates an immutable snapshot of the source evidence while keeping a foreign-key link to a real transcript segment. Users can edit only the memory title and personal note; editing never rewrites the transcript.

Deleting a recording cascades through its transcript segments, semantic index, and saved memories. Re-transcription requires explicit confirmation because replacing source segment IDs can invalidate memories tied to the old transcript.

Verification

I tested the release on a Ryzen 5 5600H with 14 GiB RAM:

Real Whisper base CPU/int8 inference detected English and produced timestamped transcript segments.
Real sentence-transformer inference produced normalized 384-dimensional embeddings.
Semantic search ranked the intended recording across multiple transcript memories.
Both model caches reloaded with offline flags enabled and outbound socket creation blocked.
The real-model integration test passed upload, transcription, indexing, search, source attribution, memory editing, deletion cleanup, and restart persistence.
38 offline backend tests passed.
Two frontend DOM tests passed, including source timestamp navigation.
TypeScript, ESLint, production build, Docker health checks, npm audit, pip-audit, and git diff --check passed.

The full verification record is here:

https://github.com/Th-Shivam/Lastwhisper/blob/master/docs/VERIFICATION.md

Why Does Open Innovation Matter?

Voice recordings can contain identity, health information, relationships, locations, family history, and moments that were never meant to become somebody else's training or analytics data.

A closed transcription or embedding API would make this archive depend on another company's:

Retention and privacy policy.
Pricing and rate limits.
Network availability.
Model behavior.
Product lifetime.

Open models made a different architecture possible.

After the model files are cached, LastWhisper can transcribe and search without uploading the recording or transcript to a hosted AI provider. The user can inspect where every result came from, replace a model, rebuild the index, back up ordinary files and SQLite, or continue using the archive even if an external service disappears.

Open also made the system more honest. The application records the exact model revision and embedding dimension that produced an index. If that configuration changes, the index can be rebuilt instead of quietly comparing incompatible vectors.

At the same time, I did not want to misuse the word “offline.” Initial dependency and model downloads require internet access. LastWhisper exposes cache state, supports local-files-only modes, and fails with actionable errors when required weights are unavailable instead of pretending inference succeeded.

My Agent Session

I built and release-hardened LastWhisper with meaningful AI-agent assistance through OpenCode. The process included repository audits, migration and concurrency reviews, real-model debugging, privacy hardening, frontend accessibility work, Docker verification, and adversarial release review.

I have not embedded the raw private session transcript because it contains local development paths and operational details that require curation before public sharing. The public engineering trail is preserved in 31 focused commits, automated tests, the verification report, and the repository's architecture/security documentation.

Commit history:

https://github.com/Th-Shivam/Lastwhisper/commits/master

The Spirit of It

The most important moment in LastWhisper is not when a similarity score appears.

It is when someone clicks Open source, the recording jumps to the exact timestamp, and the original voice plays again.

The model does not replace that voice. It does not impersonate it or generate a polished version of what it thinks was said. It helps the listener return to the real evidence—the pause, tone, words, and context that made the memory meaningful in the first place.

That is the spirit of this project: using a machine not to manufacture a ghost, but to keep a real voice from being lost inside a forgotten audio file.

Prize Categories

I am entering LastWhisper for the overall Hacktoberfest Open-Source AI Challenge: Week 4 prize.

I am not claiming a partner prize category because the core application intentionally uses local open-source components rather than adding a sponsor technology only for eligibility.

Team submission: Solo project.

This submission and implementation were created with meaningful AI assistance. I reviewed the technical claims, code, verification results, and final article, and will set DEV's AI disclosure to some_ai before publishing.

Git Out: A Local Voice Companion That Helps Developers Touch Grass

Shivam Singh — Sun, 11 Oct 2026 10:10:16 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

Git Out is a local-first voice companion for developers who want to think through
technical problems without staying glued to a screen.

The idea is simple: take a walk, ask a question out loud, and get a useful answer
without opening a laptop, sending your voice to a cloud service, or losing your
train of thought to a dozen tabs.

Git Out is built for developers who are walking, commuting, taking a break, or
working away from their desk. It can help with questions such as:

“What is the safest way to refactor this module?”
“How should I think about this pull request?”
“What Git workflow should I use for this change?”

The app records one deliberate voice turn, transcribes it on the Android device,
generates an answer with a local Qwen model, and reads the completed answer aloud
with the installed Android voice.

The screen is intentionally the shortest part of the experience. Pocket Mode has
large, high-contrast controls and explicit states for Ready, Listening,
Transcribing, Thinking, Speaking, and Error. There is no wake word,
no always-on microphone, and no automatic microphone restart in the background.

Git Out also includes an optional read-only GitHub workspace. When the backend is
configured, the app can show the connected account, repositories, assigned issues,
pull requests, and recent commits. The GitHub token stays on the server and is
never bundled into the mobile app.

Demo

Video demo:

The demo should show:

Loading the Qwen GGUF model from Android Settings.
Starting Pocket Mode and asking a question by voice.
The on-device transcript appearing on screen.
A local answer being generated and spoken by Android TTS.
Locking or backgrounding the phone while an answer is already speaking.
Returning to the app and showing that the answer remains readable.
Optionally opening the GitHub tab to show repository context.

This is an Android development build, so a short screen recording is the most
useful demo format. The project is designed to be tested on a physical Android
device because installed speech packs, TTS voices, audio focus, headset events,
and lock-screen behavior vary by device.

Code

The complete open-source project is available here:

Git Out on GitHub

The repository contains the React Native/Expo mobile app, the FastAPI backend,
the local AI abstraction, the Android speech and TTS integrations, GitHub API
services, tests, and setup documentation.

How I Built It

Git Out is an Android-first React Native app built with Expo and a small FastAPI
backend for the optional GitHub workspace.

Local AI

The core model is Qwen2.5-1.5B-Instruct Q4_K_M GGUF, loaded on the Android
device with llama.rn. The model file is
selected and stored locally by the user; it is not bundled into the repository or
downloaded automatically by the app.

The mobile conversation path is:

Voice question
    ↓
Android on-device speech recognition
    ↓
Local transcript
    ↓
Qwen GGUF through llama.rn
    ↓
Readable answer + optional Android TTS playback

Speech input and output

expo-speech-recognition checks for an installed Android on-device language pack before recording.
The app refuses to silently switch to network recognition when the requested offline language is unavailable.
A handwritten GitOutTts Expo module wraps Android TextToSpeech.
The TTS path handles exact installed voice matching, audio focus, headset removal, stop/cancel actions, and supported media-session controls.
A scoped media playback foreground service is used only while an already-started answer is being spoken. It never starts the microphone.

Pocket Mode

Pocket Mode is a one-shot voice workflow rather than a continuous assistant. Each
tap starts one question, and the lifecycle is guarded so stale recognition or TTS
callbacks cannot submit an accidental second turn.

The state machine makes the interaction understandable while the user is moving:

Ready → Listening → Transcribing → Thinking → Speaking
                         ↘ Error

Optional GitHub workspace

The FastAPI service uses a server-only GITHUB_TOKEN and GitHub's REST API. It
exposes sanitized read endpoints for the authenticated user, repositories,
assigned issues, pull requests, and commits. The mobile app only receives
whitelisted response fields; it never receives the token.

GitHub is intentionally separate from local inference. Local conversation keeps
working when the backend or network is unavailable.

Visual design

The app uses a dark, minimal skeuomorphic interface: raised cards, recessed input
wells, tactile buttons, subtle shadows, and clear status pills. This gives the
controls a physical-device feel without adding a heavy UI library or image assets.

Why Does Open Innovation Matter?

Open innovation is what makes Git Out fit the Touch Grass theme instead of becoming
another cloud chat interface.

The open-weight Qwen model lets the main conversation run locally on the phone.
That makes several important things possible:

Offline use: after the model and Android speech/TTS data are installed, a user can walk without Wi-Fi or mobile data.
Private voice conversations: prompts, transcripts, and answers do not need to leave the device for the local AI loop.
Model control: the GGUF model is a user-owned file that can be replaced or upgraded without rewriting the whole product around a hosted API.
No per-request dependency: the core experience does not require a paid API key or a vendor's uptime.
Inspectable building blocks: the app can combine an open-weight model, llama.rn, Expo modules, native Android speech APIs, and a local-first state machine in a way that can be inspected and changed.

A closed hosted API could make a quick prototype, but it would move the central
conversation onto a remote service and make a no-signal walking workflow much less
reliable. Open components gave me the control needed to make privacy and offline
behavior product features rather than marketing claims.

The trade-off is honest: a small on-device model has less capacity than a large
cloud model, and loading a GGUF requires enough memory on the Android phone. Git
Out chooses that trade-off because a private, available answer while walking is
more useful here than a larger answer that requires a network connection.

Prize Categories

I am entering the overall Hacktoberfest Open-Source AI Challenge. I am not claiming
a partner category because Git Out does not currently use partner-specific
technology.

What I Verified

The repository includes automated checks for the local AI guards, speech
recognition behavior, TTS lifecycle, voice-conversation transitions, Pocket Mode
state changes, and duplicate media-command handling.

The final frontend checks passed:

npm run typecheck   ✓
npm test            ✓ 50 tests passed
npx expo export --platform android   ✓
git diff --check    ✓

For the final device demo, I will report the phone model, Android version, speech
service, and TTS engine used. Device-specific speech, headset, background
playback, and lock-screen behavior can vary by device, so those details are part
of the demo notes rather than assumptions about every Android phone.

Closing

Git Out is a small experiment in making developer tooling less desk-bound. The
goal is not to keep people in an AI chat window. The goal is to let them ask one
useful question, hear a practical next step, and get back to the world outside
the screen.

Guardian — The AI Safety Companion I Built for My Friends & Family

Shivam Singh — Sun, 04 Oct 2026 19:21:09 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

This project started with a real incident.

My friend Shubham got scammed on the day his pocket money arrived.

He decided to buy lottery tickets and purchased two tickets for ₹1,000 each. Later, when he checked his lottery number on a website, it showed that he had supposedly won ₹15 lakh.

Soon after, he received messages telling him that he had won the lottery and that he needed to pay ₹2,000 through a QR code to process the ₹15 lakh payment.

He believed it.

He paid the ₹2,000.

After that, the person who had contacted him stopped responding.

The lottery tickets had already cost him ₹2,000, and the additional payment added another ₹2,000, making the total loss ₹4,000.

That incident made me think about a problem I see much more often.

My parents, relatives, and friends regularly send me messages like:

"Is this real?"

"Should I click this link?"

"They are asking for money. Is this a scam?"

Not everyone knows how to inspect a suspicious URL, recognize social-engineering tactics, or understand whether a payment request is legitimate.

So I built Guardian.

Guardian is a WhatsApp-first AI safety companion that lets someone simply forward a suspicious message before acting on it.

It can analyze:

Text messages
Suspicious URLs
Screenshots and images
Voice notes
English, Hindi, and Hinglish

Instead of giving a complicated technical explanation, Guardian responds with three simple things:

Risk → Why → What to do

The goal isn't to replace human judgment or claim that something is "100% a scam."

The goal is much simpler:

Give people a second opinion before they click, pay, or share something sensitive.

And that is where the name comes from:

Before you trust it, ask Guardian.

Demo

So I recreated the story that inspired Guardian with my friend Shubham.

The video starts with me asking him:

"Shubham, batao tumhare saath kya hua?"

He explains how he received a message saying that he had won ₹15 lakh in a lottery, and was then asked to pay money through a QR code to process the prize.

After hearing his story, I introduce the simple solution:

Before paying, clicking, or sharing anything suspicious — forward it to Guardian.

Then, instead of just explaining how Guardian works, I actually use it in the same video.

I forward the suspicious message to Guardian and show how the entire process works from a user's perspective:

Forward → Guardian analyzes → Get a simple explanation → Decide safely

The important part is that the user doesn't need to understand AI, phishing, URLs, OCR, search APIs, or any technical details.

They just need to know how to forward a message on WhatsApp.

That's the experience I wanted to build for my parents, relatives, friends, and anyone who isn't comfortable evaluating suspicious digital messages themselves.

Video Demo

https://youtu.be/9LDguc0cTao?si=9hfbhi0uaWPZ-g9O

The demo shows:

The real problem that inspired Guardian
The lottery scam scenario
How Guardian identifies the risk
How simple the WhatsApp interaction is
How a user can get a second opinion before making a payment

Try Guardian Yourself

Don't just read about Guardian, try it.

Send a suspicious message, link, screenshot, or voice note directly to Guardian on WhatsApp:

📱 WhatsApp: +91 9236851502

👉 Try Guardian on WhatsApp

Guardian will analyze it and reply with a simple:

Risk → Why → What to do

If you've ever received a message and wondered "Is this real or a scam?" — send it to Guardian.

For testing, please use sample or non-sensitive messages. Never send real OTPs, passwords, banking details, or other confidential information.
I wanted the demo to show the problem first, not just the technology.

Code

The complete source code is open source on GitHub:

Guardian — GitHub

The repository contains the FastAPI backend, WhatsApp Web bridge, OCR pipeline, Gemma reasoning layer, SerpApi URL verification, ElevenLabs speech-to-text integration, Sentry tracing, and deployment configuration.

Architecture

WhatsApp
   ↓
Baileys WhatsApp Web Bridge
   ↓
Message normalization
   ↓
Safety signal analysis
   ↓
URL extraction + SerpApi verification
   ↓
Gemma risk reasoning
   ↓
Safe response formatting
   ↓
WhatsApp

How I Built It

Guardian is built around Gemma, an open-weight AI model, as the core reasoning layer. I use Gemma 3-27B Instruct through an OpenAI-compatible interface, which keeps the model layer replaceable and leaves a clear path to local inference later.

Gemma 3-27B Instruct analyzes the message and produces a structured risk assessment.
SerpApi provides live web evidence when a suspicious URL needs verification.
Tesseract OCR extracts text from screenshots locally.
ElevenLabs Speech-to-Text converts voice notes into text before sending them through the same Guardian pipeline.
Baileys connects Guardian to WhatsApp, allowing users to simply forward suspicious messages.
Sentry Agent Tracing Guardian's AI workflow is also instrumented with Sentry Agent Tracing so I can see where time is spent across the pipeline. For example, a single request can be traced across message normalization, safety analysis, URL verification, Gemma reasoning, and response formatting.

The backend is built with FastAPI, while the WhatsApp bridge is built with Node.js and deployed on Render.

I intentionally kept the AI layer modular. Today Guardian uses hosted Gemma through an OpenAI-compatible API, but the same architecture can be adapted to a self-hosted/local Gemma deployment without redesigning the entire application.

The goal was not to build another chatbot. It was to make AI-powered scam checking as simple as forwarding a WhatsApp message.

Why Does Open Innovation Matter?

Guardian deals with messages that can contain sensitive information, suspicious links, screenshots, and personal conversations. For this kind of product, I did not want the AI layer to be a completely closed and irreplaceable component.

Using an open-weight model like Gemma gives Guardian a much more flexible foundation. The model can be replaced, self-hosted, fine-tuned, or moved to local inference as the project evolves. The rest of the Guardian pipeline does not need to be redesigned around a single closed AI provider.

Open innovation also makes the architecture more adaptable. Today Guardian uses hosted Gemma through an OpenAI-compatible interface, but the same reasoning layer can be connected to a self-hosted Gemma deployment in the future.

For a safety-focused tool, that flexibility matters because privacy, cost, deployment environment, and model control can change over time.

Prize Categories

Guardian is being entered into the following categories:

Featured Categories

Best Use of Gemma — Gemma 3 27B Instruct is the core reasoning model used by Guardian to analyze suspicious messages and generate risk assessments.
Best Use of Render — Guardian's backend and WhatsApp bridge are deployed on Render.

Partner Categories

Best Use of ElevenLabs — ElevenLabs Speech-to-Text converts WhatsApp voice notes into text so they can go through the same Guardian safety pipeline.
Best Use of Sentry Agent Tracing — Sentry traces Guardian's AI workflow, including message processing, URL verification, Gemma reasoning, and response formatting.
Best Use of SerpApi — SerpApi provides live web evidence for suspicious URLs before Guardian gives its assessment.