This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle
The Problem: NDAs vs. Productivity
If you work in customer support operations, quality assurance (QA), or run a small outsourcing agency, you deal with a massive bottleneck: auditing support tickets and call transcripts. Managers spend hours reading transcripts to score agents on tone, empathy, compliance, and resolution accuracy.
Every agency wants to use AI for this, but they hit a brick wall: Client NDAs and PII (Personally Identifiable Information). You cannot send a batch of 500 bank support chats or healthcare inquiries to a closed cloud API without triggering massive legal and compliance red flags. Even with "zero-data-retention" enterprise tiers, small agencies can't afford the minimum contract sizes or the per-token costs of processing millones of transcript words a month.
The Solution: VaultQA
I built VaultQA, a minimalist, local-first QA auditing suite. You drop in a CSV of raw support tickets (or call transcripts) and define your custom grading rubric. VaultQA processes the entire batch locally on your machine, redacts accidental PII, scores every interaction from 0–100 across custom criteria, flags compliance failures, and exports a clean client-ready audit report.
How It Works (The Open Stack)
Inference Engine: Ollama running locally on localhost:11434.
Open-Weight Models: Llama 3.2 (3B) / Qwen 2.5 (7B-Instruct). I used structured JSON outputs (format: "json") so the model acts as a deterministic grading engine rather than a chatty bot.
Backend Harness: A lightweight Python (FastAPI) local server that handles batch queueing, regex + NER pre-scrubbing, and prompt routing.
Frontend: Next.js + Tailwind CSS running as a crisp, high-contrast local dashboard with live token-speed telemetry and batch progress bars.
Why Open Innovation Matters Here (Open vs. Closed)
This side hustle literally cannot exist on closed AI:
The "Wi-Fi Off" Sales Pitch: When pitching a cautious agency owner or freelance bookkeeper/auditor, telling them "Trust my cloud privacy policy" fails. With VaultQA, my live demo involves loading their transcripts, physically turning off the laptop's Wi-Fi, and watching the audit run at 45 tokens/second. That single moment sells the software.
Unit Economics That Actually Work for a Side Hustle: Auditing 10,000 long support transcripts a month via a frontier closed API eats up margins fast, forcing you to charge messy usage-based subscriptions. Because the open-weight model runs on the user's own hardware, my marginal cost per user is $0.00. I can sell VaultQA as a flat $49 one-time license or $15/month tool with 100% gross profit.
Hot-Swapping & Niche Fine-Tuning: General-purpose APIs love to lecture you or change their tone after a silent model update. With open weights, I locked the exact quantized GGUF model weight (q4_k_m) so the grading strictness never drifts. Plus, if a client runs a bilingual support desk (like Hinglish or Spanglish chats), they can swap in a model or LoRA adapter tuned specifically for regional slang with one dropdown click.
Landing the First User: How It Went
I put the prototype in front of a team lead who manually audits 60 customer support tickets a week for an e-commerce brand.
The Test: We loaded 50 messy, real-world return/refund tickets into VaultQA using a 4-point rubric (Greeting, Policy Accuracy, De-escalation, Closing).
The Result: It chewed through the batch in under 4 minutes on an M2 MacBook Air. It caught two tickets where an agent promised a refund outside the return window—something the lead admitted they might have skimmed past on a Friday afternoon.
Did they pay? They offered to pay $25/month immediately if I packaged it so they didn't have to touch the terminal.
What they asked for next:
Direct PDF "Coaching Cards" they can send to individual agents on Slack.
Local Whisper (whisper.cpp) integration so they can drag-and-drop raw .mp3 call recordings directly instead of text transcripts.
Top comments (0)