DEV Community: Shaurya Srivastava

I Built a Private, Offline Lecture-to-Notes Tool with Whisper and Gemma 3

Shaurya Srivastava — Sun, 25 Oct 2026 13:00:07 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

What I Built

Lecture2Notes is a local-first command-line tool that turns a lecture recording into study material. Give it an MP3, M4A or MP4 and it produces:

structured study notes (overview, definitions, formulas, algorithms, examples, common mistakes, quick-revision summary and revision questions)
a timestamped transcript
a validated list of assignments, quizzes, exams and deadlines, each with the timestamp and the evidence from the recording

It saves the hour of re-listening and note-taking that usually follows a lecture, and it helps anyone who has ever missed an announcement like "the lab report is due next Friday" because they were writing something else down.

The part I care most about is trust. An AI note-taker that makes up a deadline is worse than none, so:

the lecturer's wording is kept as-is ("next Friday")
calendar dates stay null unless you supply the real lecture date
every action item needs evidence from the transcript or it is dropped
if any step fails, the notes carry a visible INCOMPLETE banner instead of pretending everything worked

Everything runs on your own machine: no cloud transcription, no API key, no uploading private recordings.

Demo

Example of what the action-item output looks like:

{
  "type": "assignment",
  "description": "Submit the lab report on binary search trees",
  "deadline_text": "next Friday",
  "deadline_date": null,
  "source_timestamp": "00:01:20",
  "evidence": "The lecturer says the lab report must be submitted next Friday.",
  "confidence": "high",
  "needs_verification": true
}

Code

shauryasri14 / Lecture2notes

Lecture2Notes

Turn a lecture recording into study notes, revision questions and a list of assignments/deadlines, entirely on your own computer.

Lecture2Notes transcribes a recording with faster-whisper, then uses a local LLM served by Ollama (default: gemma3:4b) to write structured notes. No cloud service, no API key, no upload of your recordings.

What it does

Transcribes MP3, WAV, M4A, MP4 and other audio/video files, keeping real segment timestamps.
Splits the transcript into overlapping chunks and summarizes each one.
Merges the summaries into one study document: overview, definitions, formulas, algorithms, examples, common mistakes, quick-revision summary and revision questions.
Finds assignments, quizzes, exams and deadlines in the transcript and saves them as validated JSON with the timestamp and…

View on GitHub

git clone https://github.com/shauryasri14/Lecture2Notes
cd Lecture2Notes
python -m venv .venv && source .venv/bin/activate   # Windows: .\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
ollama pull gemma3:4b
python lecture2notes.py lecture.mp3 --whisper-model base --language en

The README covers setup for Windows, Linux and macOS, plus a troubleshooting table.

How I Built It

Open-source stack

faster-whisper (Whisper speech recognition) for local transcription with real segment timestamps, running on CPU by default
Gemma 3 4B (an open-weight model) served by Ollama for summaries, merging and extraction
Plain Python, requests, watchdog for optional folder watching, and pytest

Pipeline

recording → faster-whisper → timestamped segments → chunker (1200 words, 100 overlap)
  → per-chunk summaries → hierarchical merge → notes.md
  → per-chunk action-item extraction (JSON, validated, de-duplicated) → action_items.json

Design decisions:

Chunking is deterministic and tested separately from the models, so overlap and forward progress are guaranteed even when one segment is huge.
Hierarchical merging keeps very long lectures from overflowing the model's context window.
Action items come from the transcript chunks, not from the summary, and the model's JSON is validated before use. Malformed JSON gets a bounded retry, then fails safely instead of producing made-up items.
Bounded retries and timeouts for every Ollama request, and a failed chunk is recorded rather than hidden.
Resilience on real machines: while testing on a Windows laptop I hit a PyAV version mismatch and a missing CUDA library. The tool now falls back to FFmpeg decoding and to CPU automatically.

The test suite (53 tests) mocks Whisper and Ollama, so it needs no models. I also ran the real HTTP client and CLI against a fake local Ollama server. Those tests check the logic and wiring, not the quality of notes from real audio. [Add your own honest result from running it on a real lecture clip, and what you'd improve.]

Why Does Open Innovation Matter?

Lecture recordings are private: they can contain classmates' voices, a professor's unpublished material and personal details. With open models, all of it stays on the laptop. Whisper and Gemma run locally through Ollama, so there is no per-minute fee, no usage limit and no data leaving the machine. That matters especially for students on limited budgets and unreliable internet. A closed API would have meant uploading recordings and paying per hour of audio.

Open components also let me inspect and control behavior. I could set the temperature, choose the model size to match the hardware, and swap in a different Ollama model with one flag. And when the tool broke, the open libraries let me find the actual cause (a metadata_errors argument PyAV didn't accept) and work around it instead of waiting on a vendor.

Prize Categories

Best Use of Gemma: Lecture2Notes runs Gemma 3 4B locally through Ollama for every language-model step: per-chunk lecture summaries, hierarchical merging into one study document, and structured extraction of assignments and deadlines. Everything runs on a CPU-only laptop with no cloud API, and the model is switchable with one flag (--model).

Why Gemma 3 4B? I wanted something a student could actually run: it fits on an ordinary laptop with no GPU, through Ollama, with one ollama pull gemma3:4b. A small model needs guardrails, so the pipeline works around it. Transcripts are split into ~1,200-word chunks so each request stays small, prompts tell the model to use only what is in the transcript, and the deadline extraction returns JSON that is validated before anything is used. Items without evidence are dropped, and malformed JSON gets a limited number of retries and then fails safely.

Honest limits. A 4B model can miss or misread an announcement, especially in a noisy recording or a long lecture, and speech-recognition mistakes pass straight through to its output. That is why every action item carries a timestamp and evidence quote, and why calendar dates stay null unless you supply the real lecture date. If you have more RAM or a GPU, --model lets you swap in a larger Gemma through Ollama with no code changes.

I built a GST invoice sorter that runs 100% offline with open-source AI

Shaurya Srivastava — Sun, 18 Oct 2026 15:26:20 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

GST Invoice & Receipt Sorter: a local app that turns a messy folder of invoice photos and PDFs into one clean, validated Excel sheet.

In India, freelancers, small shops and small CA (accountant) offices collect dozens of invoices a month, and someone types them into a spreadsheet by hand to file GST returns. It is slow, and one wrong digit in a 15-character tax ID (GSTIN) or a total can cause problems later. Those people also don't want to upload client financial documents to a cloud service, and per-page cloud OCR/LLM pricing adds up.

You drop in files like IMG_2031.jpg or bill_final.pdf and get one row per invoice: vendor, GSTIN, invoice number, date, taxable amount, CGST/SGST/IGST and total. Every row carries its source file, a review status and the exact reason for any warning.

The key design rule: the model only copies what is printed. It never decides whether an invoice is correct. Plain Python checks do that (GSTIN checksum, tax arithmetic, state codes, dates, duplicates), and nothing is "verified" until a person marks it Reviewed.

Demo

Code

shauryasri14 / gst-sorter

GST Invoice & Receipt Sorter

Turns a folder of Indian GST invoices (PDF, JPG, PNG) into one validated spreadsheet. Everything runs on your computer: OCR, the language model, validation, storage and export. No cloud AI, no paid API, no telemetry.

Files → ingest → render PDF pages → preprocess → OCR → local LLM (Ollama) → deterministic GST checks
      → human review & correction → Excel / CSV

The model only copies what is printed. All GST checks (GSTIN checksum, tax arithmetic, state codes, dates duplicates) are plain Python in pipeline/validate.py. Extraction success is never shown as "verified": only a person can mark an invoice Reviewed.

1. Prerequisites

Need	Notes
Python 3.11 or 3.12	Recommended: PaddleOCR/PaddlePaddle wheels lag behind the newest Python. The code and tests also ran on 3.13 with Tesseract.
Ollama	Local model server.
8 GB RAM minimum	The default 7B model needs roughly 5-6

…

View on GitHub

How I Built It

Pipeline: files -> render PDF pages -> OpenCV preprocessing -> OCR -> local LLM -> deterministic GST validation -> human review -> Excel/CSV

Open-weight model: Qwen2.5 Instruct (7B by default, configurable) served locally by Ollama. It returns strict JSON that is parsed with Pydantic, with one repair retry at most. Missing values stay null, never 0.
OCR: PaddleOCR as the primary engine, Tesseract as fallback.
App: Streamlit review screen (original page beside editable fields), SQLite for saved corrections, pandas/openpyxl for export.
Validation (no LLM involved): GSTIN format and mod-36 checksum, total = taxable + taxes within a configurable tolerance, CGST/SGST consistency, state-code vs tax-type warnings, date parsing, duplicate detection, and a check that every extracted value actually appears in the OCR text (to catch hallucinations).
Safety: spreadsheet formula-injection protection, localhost-only Ollama (non-loopback hosts are refused), logs that never contain invoice contents.

What I measured (20 fictitious invoices, 5 degradation types, Tesseract, CPU only):

Field	Found in OCR text
Names, invoice no., date, total, CGST, SGST	100%
Vendor GSTIN / Buyer GSTIN	70% / 55%

GSTINs are where OCR struggles (letter O for digit 0, and so on). Of 40 GSTINs copied from OCR text, 17 were misread and the checksum caught 15 of them; the other 2 were a role mix-up in my test script, not a checksum failure. That is the point of the validation layer.

I also found a real bug through testing: Tesseract's default mode dropped the large vendor-name heading (4/10 found); a different page-segmentation mode fixed it (10/10).

300 automated tests cover validation, malformed LLM output, Ollama being down, failed OCR, multi-page PDFs, persistence of manual corrections and export safety.

Why Does Open Innovation Matter?

Privacy is the product. Invoices contain tax IDs and client amounts. With open weights and local inference, nothing leaves the laptop, and a small CA office can say that to its clients and mean it.
Cost stays near zero. No per-page fee, so the business works even at a few rupees of margin per document.
I could inspect and control every layer. Swapping the model for a smaller one on an 8 GB laptop is one setting. I could pin the output schema, tune OCR settings, and test failures with test doubles, none of which depends on someone else's API changing.
Honest limits: a small local model makes more mistakes than a large closed one, which is exactly why I built the deterministic checks and review screen around it instead of trusting it.

My First User

My first user was a friend who runs a small electronics business and regularly handles supplier invoices and purchase receipts. Before trying the tool, he managed invoice details manually in spreadsheets, which was repetitive and took time away from other business tasks.

I tested the workflow using 10 sample invoices. The tool extracted the key fields into a structured spreadsheet and flagged entries that needed manual review. The biggest challenge was GSTIN recognition, where similar-looking characters could be misread by OCR. This reinforced why automated validation and human review are essential rather than blindly trusting extracted data.

Would he pay? He found the tool useful, particularly because it could reduce manual data entry while keeping financial documents on his computer. We haven't established a price or confirmed his willingness to pay yet.

What he asked for next: Better GSTIN recognition and easier export into accounting software such as Tally. These would make the tool more practical for businesses that already have an established accounting workflow.

This initial feedback helped reinforce the project's core goal: making invoice processing faster without compromising privacy or treating automated extraction as infallible.

I Built StrideWise: Describe a Walk in One Sentence, and a Local AI Plans the Route

Shaurya Srivastava — Sat, 10 Oct 2026 19:16:02 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass.

What I Built

StrideWise is a route planner for walkers and runners. You describe the walk you want in plain English, such as "5 km, shaded, low traffic, finishing near a park," and it generates real routes you can compare on a map.

I wanted to make route planning simpler. Instead of manually checking roads and adjusting filters, you describe what matters to you and let the app find possible routes.

The goal is simple: spend less time planning your walk and more time actually taking it.

Demo

The demo shows how a natural-language request is converted into constraints and used to generate alternative walking or running routes.

Code

GitHub repository:

shauryasri14 / StrideWise

Privacy-first, local-first route planner that uses Ollama, Valhalla, and OpenStreetMap to generate and score walking/running routes based on distance, traffic, shade, surface, elevation, and air quality.

StrideWise

Describe a walk or run in plain English and get a real route you can follow.

"5 km, shaded, low traffic, finishing near a park"

A small language model running on your own machine turns that sentence into structured constraints. A real routing engine builds several candidate routes. OpenStreetMap data is used to estimate traffic, shade and surface. Optional air-quality data is added. The routes are scored with transparent weights, and the top three are shown on a map. Then you close the laptop and go outside.

Built for the Hacktoberfest "Touch Grass" challenge: open-source AI that gets people off the screen.

How it works

Your sentence
   -> Ollama (local LLM)  -> structured JSON constraints (distance, shade, traffic, ...)
   -> Valhalla (routing)  -> about 8 real candidate routes (loops, out-and-back, point-to-point)
   -> OpenStreetMap       -> traffic / shade / surface estimates along each route
   -> Open-Meteo (opt.)   -> air-quality along

…

View on GitHub

StrideWise is a local-first route planner built with Ollama, Valhalla, and OpenStreetMap. It generates and ranks routes based on distance, traffic, shade, surface, elevation, and air quality.

How I Built It

The pipeline is straightforward: understand the request, generate routes, analyse them, score them, and display the best options.

Ollama with Qwen2.5 3B: A local language model converts the user's request into structured JSON constraints, such as distance, shade, traffic preferences, and destination. It does not generate the route itself.
Valhalla: Generates candidate routes using real map data, including loops, out-and-back routes, and point-to-point routes.
OpenStreetMap: Provides geographic data used to estimate traffic levels, shade, and surface conditions. These are estimates, not live traffic measurements.
Open-Meteo: Supplies optional model-based air-quality data for route evaluation.
Scoring system: Ranks the candidate routes using configurable weights and returns three alternatives: Best match, Lowest pollution, and Quietest. Each route includes its metrics and an explanation of why it was ranked that way.
Application stack: FastAPI, React, TypeScript, Leaflet, and Docker Compose.

One design decision was to keep the language model's job narrow. It converts the request into constraints, while the routing engine handles the actual geography. This makes the results easier to test and prevents the model from inventing roads or coordinates.

If Ollama is unavailable, the app can fall back to a built-in keyword parser and indicates that fallback in the interface.

Why Does Open Innovation Matter?

Open-source tools made it possible to build the entire routing pipeline without depending on a paid AI or hosted routing API.

Local inference: Ollama runs the language model on the user's machine.
Real routing: Valhalla generates routes from map data rather than relying on an LLM to invent directions.
Customisable ranking: The scoring weights are ordinary code, so the route preferences can be changed without retraining a model.
Replaceable components: The language model, routing engine, and scoring logic are separated, making it easier to experiment with alternatives.

There are trade-offs. The user must download a map region, and some geographic data requests still go to external services such as Overpass and Open-Meteo. Traffic and shade depend on available map data, while air-quality estimates have limited street-level resolution.

These limitations are documented in the repository rather than hidden behind a general claim of privacy.

I Built My Friend a Free, Offline Chess Coach with Stockfish and a Local LLM

Shaurya Srivastava — Sun, 04 Oct 2026 19:42:03 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

I built a chess game analyzer for my friend. He loves chess, but he doesn't play often enough to pay for a subscription just to find out what he did wrong after a game or a puzzle.

His request was simple: "I want software that analyzes my moves and tells me what I did wrong."

So I built that. You upload a game (a PGN file from Lichess or Chess.com), and the app:

marks every move as best, excellent, good, inaccuracy, mistake, or blunder
shows the position with arrows for your move and the engine's better move
explains each mistake in one or two plain sentences
lets you ask a coach questions about the move you are looking at
counts your recurring problems across the game (for example "7 hung pieces")
checks single puzzle positions: enter a position and your move, and it tells you how good the move was

Lichess already offers free engine analysis, so I'm not claiming nobody can analyze games for free. What this adds is plain-language reasons, a summary of repeated mistakes, and a tool that runs locally and keeps the analysis on your machine.

My goal wasn't to build another chess engine. It was to build something that explains mistakes in a way a casual player can learn from.

Demo

Code

shauryasri14 / Chess-Move-Analyzer

A free, offline chess analyzer built for a friend who can't justify paying for game review. Stockfish judges every move, code tags the cause (hung piece, missed capture, missed mate), and a local LLM via Ollama explains it in simple words. Your games never leave your machine.

♟ Chess Move Analyzer

Free, offline chess game review that tells you what you did wrong, in plain language.

Stockfish judges every move, code tags the cause, and a small local LLM (via Ollama) explains it. No account, no subscription, no game limit, and your games never leave your machine.

Built for a friend who loves chess but doesn't play often enough to justify paying for game analysis Submission for the Hacktoberfest Weekend Challenge: Build for a Friend.

Why this exists

Casual players finish a game or a puzzle and want one thing: what did I do wrong? Many tools lock full analysis behind a paid plan, which doesn't make sense for someone who plays a few games a month.

This project gives that analysis for free, on your own computer:

Stockfish (open-source engine) decides which moves were mistakes.
Code finds the likely cause: a hung piece, a missed capture…

View on GitHub

To run it yourself: docker compose up --build, then open http://localhost:8501. Setup without Docker is in the README.

How I Built It

Open-source pieces

Piece	Role
Stockfish	Evaluates each position; the change in evaluation is used to classify the move. One of the strongest open-source chess engines.
python-chess	Reads PGN, builds boards, and communicates with Stockfish
Ollama with `gemma3:4b`	Runs the language model locally to explain mistakes
Streamlit	The web interface
Docker Compose	One command starts the app and Ollama

The pipeline

PGN → Stockfish evaluates each position → code classifies and tags mistakes → local LLM explains → summary

The most important design rule: the model never judges the chess.

Small language models are bad at evaluating chess positions, so the work is split:

Stockfish evaluates each position once. The evaluation after one move is reused as the evaluation before the next.
Code turns the evaluation drop into a label (for example, 300+ centipawns lost is a blunder) and finds the likely cause: a hung piece, a missed capture, or a missed mate.
The LLM receives only these facts and explains them in one or two sentences. It is told not to invent moves.

The "Ask the coach" chat works the same way: every question carries the current position, the evaluations, the engine's line, and the neighbouring moves. A "What the coach sees" panel shows exactly what the model is given, so you can check its input.

Details that mattered

Evaluations are clamped to ±15 pawns, so trimming a huge lead isn't called a blunder.
Thresholds are editable in the sidebar and re-label instantly, with no engine rerun.
If Ollama isn't running, the app falls back to template explanations instead of breaking.

Honest limitations

The cause tags are simple heuristics and can miss exchange sequences or pins.
A 4B model can still phrase things badly sometimes.

Why Does Open Innovation Matter?

The best engine is open source. Stockfish gives strong analysis for free, and paying for analysis was exactly my friend's problem.
No account, subscription, or game limit. He can analyze one game a month or fifty.
His games stay on his machine. The model runs locally and the app works offline.
I could control the coach. With a local model I wrote the system prompt, limited it to engine facts, showed its input, and could swap models freely. A closed API would make that harder and add a cost per question.

My Friend's Reaction

When I first showed my friend the project, his first reaction was, “Bro, this would be way better if it worked directly on Chess.com.” I was like, “Yeah, I know, but at least appreciate what I made!” 😭

A little later, he actually played a game, exported it from Chess.com, and uploaded the PGN to my app. He went through the analysis himself, and once he saw everything actually working, he was genuinely impressed. He liked that he could go through the game move by move and see not just that a move was bad, but what he did wrong and what he could have played instead.

He uses Chess.com quite regularly and usually ends up using his free analysis/trial features pretty early in the month. So being able to just run this locally and analyze his games whenever he wants was something he actually found useful.

DEV Community: Shaurya Srivastava

I Built a Private, Offline Lecture-to-Notes Tool with Whisper and Gemma 3

What I Built

Demo

Code

shauryasri14 / Lecture2notes

Lecture2Notes

Contents

What it does

How I Built It

Why Does Open Innovation Matter?

Prize Categories

I built a GST invoice sorter that runs 100% offline with open-source AI

What I Built

Demo

Code

shauryasri14 / gst-sorter

GST Invoice & Receipt Sorter

1. Prerequisites

How I Built It

Why Does Open Innovation Matter?

My First User

I Built StrideWise: Describe a Walk in One Sentence, and a Local AI Plans the Route

What I Built

Demo

Code

shauryasri14 / StrideWise

Privacy-first, local-first route planner that uses Ollama, Valhalla, and OpenStreetMap to generate and score walking/running routes based on distance, traffic, shade, surface, elevation, and air quality.

StrideWise

How it works

How I Built It

Why Does Open Innovation Matter?

I Built My Friend a Free, Offline Chess Coach with Stockfish and a Local LLM

What I Built

Demo

Code

shauryasri14 / Chess-Move-Analyzer

A free, offline chess analyzer built for a friend who can't justify paying for game review. Stockfish judges every move, code tags the cause (hung piece, missed capture, missed mate), and a local LLM via Ollama explains it in simple words. Your games never leave your machine.

♟ Chess Move Analyzer

Why this exists

How I Built It

Why Does Open Innovation Matter?

My Friend's Reaction