Atharv Mohite

Posted on Oct 19

VoiceLedger: Turn a 20-Second Voice Note into a GST Invoice Using Open-Weight Gemma and MongoDB Atlas

#devchallenge #hf26challenge

Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle Submission 💸

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

Independent contractors in India—plumbers, electricians, freelance event photographers, mechanics, and home bakers—spend their entire day doing hands-on work. The last thing they want to do at 9:00 PM is sit at a desk, open a spreadsheet, look up HSN/SAC tax codes, separate material costs from labor charges, calculate CGST/SGST splits, and format a PDF bill. Because of this friction, many either delay billing (hurting their cash flow) or send rough text messages on WhatsApp that look unprofessional and make quarterly GST filing a mess.

VoiceLedger turns that 45-minute evening chore into a 20-second voice note recorded right after finishing a job.

A contractor simply taps the microphone and speaks naturally in English or Hinglish:

"Sharma ji ka 2 copper valve change kiya, 1400 parts ka laga, aur 2 ghante fitting labor at my standard 500 rate. 18% GST laga ke bill bana do."

In under two seconds, VoiceLedger:

Transcribes the noisy, code-switched audio using ElevenLabs Scribe (scribe_v1).
Matches the client ("Sharma ji" $\rightarrow$ Rajesh Sharma) against a MongoDB Atlas collection to automatically pull their saved address, phone number, GSTIN, and the contractor's default hourly rate.
Extracts a strict JSON schema using open-weight Google Gemma 2 (gemma2-9b-it) or Llama 3.3 (llama-3.3-70b-versatile), separating physical parts (MATERIALS - HSN 8481) from service work (LABOR - SAC 9954).
Runs deterministic math guardrails in JavaScript to calculate taxable subtotals, CGST/SGST splits, and the grand total—ensuring zero LLM math hallucinations.
Renders a live, editable invoice across three switchable templates (GST Pro, Minimal Studio, Express Receipt) with a dynamically generated UPI Payment QR code (upi://pay?...), 1-click A4 PDF export, WhatsApp share link, and spoken voice confirmation via ElevenLabs Flash TTS.

Who Would Pay For It?

VoiceLedger is designed as a ₹499/month (~$6/month) or $9/month micro-SaaS for independent contractors, solo service professionals, and small studios. Getting even a single overdue invoice paid faster—or saving 10 hours of manual bookkeeping a month—pays for the subscription many times over.

Demo

Live Application: https://voiceledger.tech

(Tip for Judges: If you are testing in a quiet environment without a microphone, you can click any of the **3 Instant Judge Demo Presets* directly below the voice orb to run the full pipeline immediately.)*

Application Screenshots

1. Vaani Studio — Voice-to-Invoice Command Center

Interactive Liquid-Gradient Voice Orb: Radial tick-ring audio visualizer with 1-tap MediaRecorder capture (Ready · Step 1 of 3), built to handle natural English and Hinglish speech.
Open-Weight Model Switcher & Observability Bar: Top navigation bar providing instant model selection (OpenAI GPT-OSS 120B, Gemma 2, Llama 3.3), a live MongoDB Atlas Tax Ledger counter (12 persisted bills), and real-time Sentry Active status.
1-Click Judge Demo Presets: Pre-configured contractor scenarios (Plumber & Electrical, Wedding Photographer, and Artisanal Home Baker) that trigger the full extraction pipeline without requiring microphone access.

2. Split-Screen AI Co-Pilot & Live Editable GST Invoice Workspace

ElevenLabs Scribe & Voice Read-Back (Left Panel): Displays the verbatim transcript ("Create an order bill for Neha Kulkarni: 2 kg Belgian chocolate truffle cake at 1200 per kg, and 12 assorted artisanal cupcakes at 90 each. Apply 5% GST.") alongside an interactive waveform audio player for hands-free confirmation.
Atlas Client Match & Tax Controls (Middle-Left Panel): Automatically populates client details (Neha Kulkarni), configures the applicable tax slab (5% GST), and provides 1-click persistence to MongoDB Atlas.
Live Agent Trace & Unit Economics Card (Bottom-Left Panel): Surfaces the active Sentry Trace ID (#8188b901994e8bf7), end-to-end execution latency (1017ms), and a live cost comparison showing $0.00015 on open-weight inference versus $0.03400 on closed frontier APIs.

* Multi-Template Printable Canvas (Right Panel): Features a 4-way template switcher (`GST Pro`, `Minimal`, `Express`, `Artisan`), inline-editable line items (`₹2,400` + `₹1,080` = `₹3,480` subtotal + `₹174` GST = `₹3,654` Grand Total), a scannable QR code box, Save PDF, and WhatsApp instant sharing.

Code

atharv96k / VoiceLedger

Open-Source AI Voice-to-Invoice & GST Tax Engine for Independent Contractors

VoiceLedger converts unstructured English and Hinglish voice notes into HSN/SAC-compliant GST invoices, automated tax ledgers, and scannable UPI QR payments in under two seconds.

Overview • Architecture • API Reference • Setup

Overview

Independent contractors plumbers, electricians, event photographers, mechanics, and home bakers frequently lose billable hours to manual evening paperwork, itemized quoting, and quarterly tax reconciliation.

VoiceLedger eliminates manual data entry through a 5-stage open-weight AI agent pipeline. A contractor records a quick voice memo after finishing a job (or selects a pre-configured workflow preset). The backend transcribes the audio via ElevenLabs Scribe, retrieves repeat client billing records from MongoDB Atlas, extracts a strict JSON schema using open-weight Google Gemma 2 (gemma2-9b-it) or Meta Llama 3.3 (llama-3.3-70b-versatile), verifies line-item arithmetic deterministically, and returns an editable multi-template invoice with a live UPI payment…

View on GitHub

How I Built It

VoiceLedger is structured as a decoupled full-stack monorepo (/client and /server) built around a 5-stage agentic pipeline[cite: 1, 20].

Architecture & Request Pipeline

[React + Vite Client: MediaRecorder (audio/webm)]
       │
       ▼ POST /api/voice-invoice (multipart/form-data)
[Node.js + Express Backend wrapped in Sentry Root Transaction]
       │
       ├─► Span 1 (ai.stt): ElevenLabs Scribe v1 converts audio buffer to raw text
       ├─► Span 2 (db.query): Mongoose queries MongoDB Atlas for client aliases & rates
       ├─► Span 3 (ai.chat_completions): Open-weight Gemma-2-9b-it / Llama-3.3-70b
       │          extracts lineItems[], category (MATERIALS|LABOR), and HSN/SAC codes
       ├─► Guardrail: Deterministic JS engine computes Subtotal, CGST, SGST & UPI URI
       ├─► Span 4 (ai.tts): ElevenLabs Flash v2.5 synthesizes voice confirmation MP3
       └─► Persistence: Stores finalized document in MongoDB Atlas 'invoices' collection

Technical Stack

Open-Weight AI Core: gemma2-9b-it and llama-3.3-70b-versatile served in strict JSON mode (response_format: { type: "json_object" }) with low temperature (0.1) for deterministic schema extraction.
Speech Processing: ElevenLabs scribe_v1 handles multi-accent English/Hinglish speech-to-text, while eleven_flash_v2_5 generates low-latency spoken confirmations so contractors can verify totals hands-free.
Data Layer & Client RAG: MongoDB Atlas stores two primary collections (Client and Invoice). When a transcript arrives, the backend scans the Client collection for matching names or aliases (e.g., "sharma ji"), injecting the client's saved GSTIN, billing address, and default hourly rate directly into Gemma's system prompt.
Frontend Experience: Built with React 19, Vite, and Tailwind CSS v4 (@tailwindcss/vite). It features inline table editing, real-time SVG QR generation via qrcode.react, client-side PDF compilation via html2pdf.js, and a mouse-responsive spotlight ledger drawer.
Agent Observability: Instrumented with @sentry/node (instrument.js) using custom spans to measure latency and token consumption at every step of the pipeline.

Why Does Open Innovation Matter?

When building a vertical SaaS for price-sensitive independent workers in emerging markets, unit economics and schema control determine whether the business can survive.

1. Viable Unit Economics for a $9/Month Side Hustle

If a busy contractor generates 150 invoices a month—including client context lookups, multi-item tax classification, and retries—running that pipeline on closed frontier audio/reasoning APIs costs roughly $0.034 per run (~$5.10/month in raw API costs per user), wiping out margins once payment gateway fees and hosting are factored in.

By using open-weight Gemma 2 (gemma2-9b-it) for structured JSON extraction, each inference call consumes ~480 tokens at an effective cost of $0.00010 per invoice. Even at 200 invoices per month, the LLM cost per user is under $0.02/month, sustaining a 98.8% gross margin.

Metric	Open-Weight Stack (`gemma2-9b-it`)	Closed Frontier API Stack
Avg. Extraction Latency	~390 – 420 ms	~1,400 – 2,200 ms
Cost per Invoice Run	$0.00011	~$0.03400
200 Invoices / Mo Cost	$0.022	~$6.80
Model Portability	Swap between Gemma & Llama in 1 click	Vendor lock-in

2. Zero Vendor Lock-In & Model Swapping

In the top navigation bar of VoiceLedger, users can switch the underlying extraction engine between google/gemma-2-9b-it, google/gemma-3-27b-it, and meta/llama-3.3-70b. Because our prompt architecture relies on standard open-weight JSON schemas rather than proprietary tool-calling wrappers, we can route requests to whichever open model or inference provider offers the best latency and price-to-performance ratio.

3. Hybrid AI + Deterministic Code

General-purpose closed chatbots try to do everything inside the LLM—including arithmetic, which frequently leads to rounding errors on tax percentages. Our open-weight pipeline restricts the model strictly to semantic entity extraction (identifying item descriptions, quantities, unit rates, and HSN/SAC categories) while delegating all tax math and UPI URI formatting to deterministic JavaScript.

My First User

[Optional: Customize this section with your real test experience before publishing, or keep this structure if you tested it with a local contractor/friend!]

To test whether VoiceLedger actually works outside a developer environment, I put the app in front of [Insert Name / Role, e.g., a local electrical contractor / freelance photographer friend in Kolhapur] right after they wrapped up a job.

Instead of typing out a bill at night, they recorded a 15-second voice note in Hinglish listing the parts replaced and their labor charge.

What worked immediately: Seeing their spoken Hinglish turn into a clean English GST table with separate Materials and Labor columns—and a working UPI QR code that they could immediately forward on WhatsApp—got an instant reaction.
Feedback & what they asked for next: They mentioned they would gladly pay ₹300–₹500/month if the app could also let them snap a photo of a hardware store purchase bill to automatically subtract input GST credit from their monthly tax ledger. That is next on the roadmap.

Prize Categories

I am submitting VoiceLedger for the following partner prize categories:

Best Use of Gemma: Uses Google's open-weight gemma2-9b-it / gemma-3-27b-it as the core structured extraction engine to parse noisy transcripts into line items, classify MATERIALS vs. LABOR, and assign Indian HSN/SAC tax codes.
Best Use of ElevenLabs: Uses ElevenLabs Scribe (scribe_v1) for accurate Speech-to-Text transcription across English and Hinglish accents, and ElevenLabs Flash TTS (eleven_flash_v2_5) for hands-free spoken invoice confirmations.
Best Use of MongoDB Atlas: Uses MongoDB Atlas (Client and Invoice collections) both for pre-inference RAG context lookup (matching repeat clients and default rates) and for persisting the contractor's tax ledger.
Best Use of Sentry Agent Tracing: Wraps the entire Express backend pipeline in a @sentry/node root transaction (voice_to_invoice_agent) with child spans tracking STT, MongoDB queries, Gemma token usage/latency, and TTS synthesis.
Best Use of Render: Deployed the full-stack application on Render.
Best Use of a .tech Domain: Hosted live on a custom .tech domain.

DEV Community

VoiceLedger: Turn a 20-Second Voice Note into a GST Invoice Using Open-Weight Gemma and MongoDB Atlas