DEV Community: Sasidhar Prathipati

I Built a Fully Offline Meeting Transcriber That Runs on the Snapdragon NPU

Sasidhar Prathipati — Wed, 30 Sep 2026 15:22:18 +0000

Cloud transcription tools have a problem nobody talks about: they upload your audio to someone else's server.

For most people, that's a minor privacy tradeoff. For lawyers, doctors, and consultants, it's a dealbreaker. Attorney-client privilege, HIPAA, and corporate data policies all prohibit sending meeting audio to a third party. So these professionals are stuck taking manual notes while everyone else gets AI-powered transcripts.

I built Whisper Notes to fix that — a fully offline meeting transcriber that runs entirely on the Snapdragon X Elite NPU.

What It Does

Whisper Notes captures both your microphone and your system audio (Zoom, Teams, Google Meet) through WASAPI loopback, transcribes in real time, and generates a summary with action items. Everything runs on-device. Nothing touches the network.

Meet Mode — one click captures system audio + mic simultaneously
Real-time transcription — Whisper-Small-Quantized on the Hexagon NPU
Local summarization — action items extracted on-device
Encrypted storage — SQLite with AES-256
Verifiable privacy — open any network monitor; zero bytes sent

Why Snapdragon

This is built on Qualcomm AI Hub's Whisper-Small-Quantized model, compiled for the Hexagon NPU with the QNN runtime.

Component	Latency	Compute Unit
Decoder	~7 ms	NPU
Encoder (30s audio)	~308 ms	NPU

The 7ms decoder latency is the difference between waiting for transcription and watching words appear as people speak. And because it runs on the NPU instead of the CPU or GPU, you get hours of meeting capture on battery — not minutes.

Architecture

Audio Capture (WASAPI loopback + mic)
↓
Whisper-Small-Quantized (NPU via Qualcomm AI Hub)
↓
Local Summarization (Qwen3-0.6B or rule-based)
↓
Encrypted Local Storage (SQLite + AES-256)

text

Every stage runs on-device.

The Interesting Code

Loading the model onto the NPU:


python
from geniex import AutoModel

class WhisperTranscriber:
    def __init__(self, model_path):
        self.model = AutoModel.from_pretrained(
            model_path,
            device_map="qairt"  # this is what sends it to the NPU
        )

    def transcribe(self, audio):
        input_features = self.preprocess_audio(audio)
        output = self.model(input_features)
        return self.decode_tokens(output)
Capturing system audio with WASAPI loopback:

python
import sounddevice as sd

# Find the loopback device (Stereo Mix or default render)
devices = sd.query_devices()
loopback_device = None
for i, dev in enumerate(devices):
    name = dev["name"].lower()
    if "stereo mix" in name or "loopback" in name:
        loopback_device = i
        break

stream = sd.InputStream(
    device=loopback_device,
    channels=1,
    samplerate=16000,
    callback=callback,
    blocksize=1600,
)
The audio callback pushes 100ms chunks into a queue, and the transcription loop pulls 10-second windows for Whisper.

What I Learned
1. NPU inference is genuinely fast — not just a benchmark number.

The 7ms decoder latency isn't marketing. It's the reason the transcript feels live instead of laggy. On a CPU, the same model would be seconds behind.

2. Privacy is the feature, not the constraint.

I kept framing "no cloud" as a limitation while building this. Then I realized: for the target user, offline isn't a downgrade. It's the entire reason they'd use the tool. A lawyer doesn't want a faster cloud transcriber — they want one that never leaves the room.

3. System audio capture is harder than it looks.

WASAPI loopback works, but device discovery is inconsistent across Windows machines. Some laptops expose "Stereo Mix," others don't. The fallback is the default render device, but that's not always right. This is the part I'd spend more time on with more runway.

4. The model download is the biggest setup friction.

Once the AI Hub model is in place, the app is a single python app.py away. But getting the QNN-compiled binaries onto the machine is the step that trips people up. A one-click installer would massively improve adoption.

Try It
The repo is public:

GitHub: github.com/SASIDHARPRATHIPATI/Whisper-notes

How We Built a Churn Detection Agent That Remembers Every Meeting

Sasidhar Prathipati — Mon, 28 Sep 2026 17:54:15 +0000

The Problem
Every B2B sales team has the same silent killer: nobody remembers what was said three weeks ago.

A client mentions "budget is tight" in a discovery call. Two weeks later, they ask about flexible payment terms. A week after that, they mention a competitor's demo. Then they start "reconsidering priorities."

Individually, each of these signals looks minor. Together, they're a churn trajectory. But by the time a human notices the pattern, the deal is already lost.

We built an AI agent that catches this pattern before it becomes obvious. Not because the model is smarter than a sales rep — but because it remembers every meeting, and connects the dots across time.

The Insight: Memory Isn't a Feature. It's the Product.
Most AI agents today are stateless. You have a conversation, close the tab, and everything is forgotten. The next conversation starts from zero.

That's fine for one-off tasks. But it's useless for workflows where context accumulates over weeks — sales, customer support, incident response, compliance.

The shift is simple but profound:

Without memory: The agent sees one meeting and says "the client seems hesitant."

With memory: The agent sees one meeting, recalls six previous ones, and says "this client is exhibiting the same pattern as three deals we lost last quarter. Here's the action to take."

That's not a better prompt. That's a different category of agent.

We built our project on Hindsight, a memory system by Vectorize that gives AI agents persistent, queryable memory. Hindsight stores interactions, recalls them using multiple search strategies (semantic, keyword, graph, temporal), and lets the agent improve over time.

The Architecture
Our stack has three layers:

text
Meeting transcripts → Hindsight memory → Groq LLM analysis → Churn alert

Hindsight (memory layer)
Each meeting is stored with retain(), tagged with a timestamp, client ID, and context. When a new meeting arrives, recall() searches the entire memory bank to surface related signals — even if the words used are different.
Groq (reasoning layer)
We use openai/gpt-oss-120b on Groq to analyze the new meeting against the recalled memories. Groq's speed matters here — the agent needs to respond in seconds, not minutes.
Meeting transcripts (input)
Synthetic but realistic. Six meetings for a fictional client "Acme Corp" over three weeks. Each one contains a subtle signal: price concern, budget delay, competitor mention, contract flexibility request.

The Demo
Here's what happens when you run the agent.

First, it stores all six past meetings in Hindsight memory:

text
Stored 6 meetings for Acme Corp
Then a new meeting arrives — the kind that a sales rep would treat as just another check-in:

"Client said they are 'reconsidering the entire initiative' and asked whether we can pause the contract. Leadership is now looking at a cheaper alternative."

The agent queries Hindsight for related signals. Hindsight returns eight memories spanning three weeks. Groq analyzes them and produces:

text
RISK: HIGH

SIGNALS:

Budget concerns and request for flexible payment terms (≈ 2026-09-10)
Competing vendor demo scheduled (week of 2026-09-07)
Competing vendor offered a discount (≈ 2026-09-15)
Client expressed hesitation and need to justify spend internally (≈ 2026-09-15)
Request for contract flexibility and pilot program due to ROI concerns (≈ 2026-09-20)
Inquiry about cancellation policy (≈ 2026-09-20)
Reconsideration of project priorities and request to delay start (≈ 2026-09-25)
New request to pause the contract while leadership evaluates cheaper alternatives (≈ 2026-09-28)

ACTION: Initiate an executive-level retention plan — schedule a senior leadership call to co-create a customized, short-term pilot with revised, value-based pricing and clear ROI metrics, while offering a flexible exit clause to alleviate cancellation concerns.
That's the moment. The agent didn't just summarize one meeting. It connected eight signals across three weeks into a single coherent risk assessment with a concrete recommendation.

A human sales rep, reading the new meeting in isolation, would see "client is hesitant." The agent sees a pattern that's been building for a month.

Why This Matters Beyond Sales
The same architecture applies anywhere context accumulates over time:

Customer support: An agent that remembers every ticket a customer has filed, every workaround that worked, and every escalation pattern.

Incident response: An agent that recalls how similar incidents were resolved before, and which runbooks actually fixed them.

Compliance: An agent that remembers every past audit finding, which controls were tested, and what remediation is still pending.

Product feedback: An agent that connects feedback across support tickets, sales calls, and reviews — and spots emerging themes humans miss.

In all of these, the value isn't in the LLM. It's in the memory layer that lets the LLM reason across time.

The Code
The core loop is simple:

python

Store each meeting

client.retain(
bank_id=f"client-{client_name}",
content=meeting["transcript"],
context=f"Meeting {meeting['meeting_id']} on {meeting['date']}",
timestamp=meeting["date"],
)

When a new meeting arrives, recall related signals

memories = client.recall(
bank_id=f"client-{client_name}",
query="concerns, objections, delays, competitor mentions, budget issues",
)

Analyze with Groq

response = groq_client.chat.completions.create(
model="openai/gpt-oss-120b",
messages=[{"role": "user", "content": prompt}],
)
Three operations. That's the entire intelligence layer.

The magic isn't in the code — it's in what Hindsight does with recall(). It searches semantically, by keyword, by graph relationships, and by temporal proximity. That's why it can find "budget concerns" from three weeks ago when the new meeting says "reconsidering the initiative."

The Day My Research Assistant Finally Got a Memory

Sasidhar Prathipati — Fri, 26 Jun 2026 04:57:08 +0000

I've spent the last few weeks wrestling with a problem that I suspect many AI builders share: my research assistant agent was smart, but it had the memory of a goldfish and the spending habits of a trust fund kid.

Every time I asked it to help me find research papers on AI/ML topics, it would recommend articles I'd already read. It would suggest the same paper three times in a single conversation. And worse—it was using GPT-4 for every single query, even when a simpler model would have worked fine.

So I built something better. A research assistant that actually remembers what I've read and thinks about how much each query costs before it runs.

Here's how I did it.

The Problem: Amnesia Is Expensive

Let me show you what I mean.

Before adding memory and cost controls, my research agent worked like this:


python
# Before: No memory, all queries go to GPT-4
def answer_research_query(query):
    # Every query is expensive and stateless
    response = openai.ChatCompletion.create(
        model="gpt-4",
        messages=[{"role": "user", "content": query}]
    )
    return response.choices[0].message.content
This approach had two fatal flaws:

First: The agent had no idea what I'd already read. I'd ask "What are the latest papers on transformer architectures?" and it would excitedly show me papers I'd already read two weeks ago. I'd say "I've seen that one," and it would show me another paper I'd already read. This would go on for five or six rounds.

Second: Every query cost $0.03-$0.06. For simple questions like "Who wrote this paper?" or "When was this published?"—questions a cheaper model could answer perfectly well—I was burning through my API budget.

The Solution: Memory + Runtime Intelligence
I integrated two technologies that solved both problems:

Hindsight for persistent memory: The agent now remembers every paper I've read, when I read it, and key takeaways

cascadeflow for runtime intelligence: The agent decides which model to use based on query complexity and tracks costs in real-time

Step 1: Adding Memory with Hindsight
Hindsight gives my agent persistent memory that persists across sessions. Here's how I integrated it:
**PROMPT 1**
python
from hindsight import HindsightMemory

# Initialize memory for the research assistant
memory = HindsightMemory(
    namespace="research-assistant",
    embedding_model="text-embedding-3-small"
)

def store_paper_read(paper_data):
    """Store paper information in agent memory"""
    memory.store(
        key=f"paper_{paper_data['id']}",
        data={
            "title": paper_data["title"],
            "authors": paper_data["authors"],
            "abstract": paper_data["abstract"],
            "read_date": paper_data["read_date"],
            "summary": paper_data["summary"],
            "tags": paper_data.get("tags", [])
        }
    )
Now when I ask about papers, the agent first checks what I've already read:

python
def find_papers(query):
    # Step 1: Check memory for already-read papers
    already_read = memory.search(query, limit=10)

    # Step 2: Query arXiv for new papers
    raw_results = search_arxiv(query)

    # Step 3: Filter out papers I've already read
    new_papers = [
        p for p in raw_results 
        if not any(p['id'] == read['id'] for read in already_read)
    ]

    return new_papers
The first time I tested this, the difference was immediate. I asked for papers on "attention mechanisms," and the agent said: "You've already read 'Attention Is All You Need' and 12 related papers. Here are 5 new papers you haven't seen yet."

That moment—when the agent actually knew what I'd read—was when I knew this was going to work.

Step 2: Runtime Intelligence with cascadeflow
But memory alone wasn't enough. The agent was still using expensive models for everything. Enter cascadeflow.

cascadeflow gives me runtime intelligence to route queries to the right model based on complexity and cost:
**PROMPT 2**
python
from cascadeflow import Router, ModelRoute, CostTracker

# Configure routing rules
router = Router()

# Simple queries → cheap model
router.add_route(
    name="Simple Queries",
    condition=lambda query: is_simple_query(query),
    model="gpt-3.5-turbo",
    max_cost=0.005
)

# Complex synthesis → premium model
router.add_route(
    name="Complex Synthesis", 
    condition=lambda query: is_complex_synthesis(query),
    model="gpt-4",
    max_cost=0.05
)

# Search queries → embeddings
router.add_route(
    name="Search",
    condition=lambda query: is_search_query(query),
    model="text-embedding-3-small",
    max_cost=0.001
)

# Track costs in real-time
cost_tracker = CostTracker()
Now the agent automatically chooses the right model:

python
def answer_with_routing(query):
    # cascadeflow routes the query
    route = router.route(query)

    # Execute with cost tracking
    response = route.execute(query)
    cost_tracker.log(response.cost)

    return response
Simple queries like "Who wrote this paper?" go to GPT-3.5-turbo at $0.001 per query. Complex synthesis tasks like "Write a summary comparing these 5 papers" go to GPT-4 at $0.05. The average cost per query dropped from $0.04 to $0.012—a 70% cost reduction.

The Combined System
Here's how everything fits together:
**PROMPT 3**
python
class ResearchAssistant:
    def __init__(self):
        self.memory = HindsightMemory(namespace="research-assistant")
        self.router = Router()
        self.cost_tracker = CostTracker()
        self.papers_read = []

    def research(self, query):
        # Step 1: Check memory for context
        remembered = self.memory.search(query, limit=5)

        # Step 2: Route based on query complexity
        route = self.router.route(query)

        # Step 3: Execute with context from memory
        response = route.execute(
            query,
            context={
                "remembered": remembered,
                "papers_read": len(self.papers_read)
            }
        )

        # Step 4: Store new learnings
        if "paper" in response:
            self.memory.store({
                "query": query,
                "paper": response["paper"],
                "response": response["summary"],
                "cost": response.cost,
                "timestamp": datetime.now().isoformat()
            })
            self.papers_read.append(response["paper"]["id"])

        return response
The Results
After two weeks of using this system, here's what I found:

Memory with Hindsight:

0 duplicate paper recommendations

The agent remembers what I found useful about each paper

Cross-session persistence means my research builds over time

Stored over 50 papers with summaries and tags

Cost control with cascadeflow:

70% reduction in API costs

Simple queries routed to cheap models

Complex synthesis still uses GPT-4 only when needed

Real-time budget tracking prevents surprise bills

Average query cost: $0.012 (down from $0.04)

4 Lessons Learned
1. Memory Changes Agent Behavior in Non-Obvious Ways
I thought memory would just stop duplicate recommendations. But the bigger impact was the agent's confidence. When it can say "I know you've already read this and here's what you thought about it," the interaction feels radically different. The agent stops feeling like a search engine and starts feeling like a collaborator.

2. Cost Controls Are Addictive
Once you see how much you're saving with cascadeflow, you start looking for every opportunity to route smarter. I'm now thinking about dynamic thresholds—if the budget is low, route more queries to cheap models. If there's room, use premium models for more queries.

3. Combined Systems Are Greater Than the Sum
Hindsight and cascadeflow are both powerful alone. Together, they create something better. Memory tells the agent what's important. Runtime intelligence tells it how much each interaction is worth. The agent now prioritizes what to remember and what to spend money on.

4. Always Start with the User Problem
I spent the first few days thinking about technology. But the real breakthrough came when I focused on "researchers hate it when assistants repeat themselves" and "teams are tired of surprise API bills." The technology is the how. The user experience is the why.

What's Next
I'm already planning v2:

Cross-session memory: The agent will remember research interests across weeks

Smart caching: Frequently accessed papers will be cached locally

Budget thresholds: Automatically switch to cheaper models when budget is low

Paper recommendations: Based on reading history, suggest new relevant papers

Final Thoughts
Building this research assistant taught me something important: the future of AI agents isn't just about better models. It's about agents that remember what matters and think about what things cost.

Hindsight gave my agent a memory it could rely on. cascadeflow gave it the intelligence to run efficiently. Together, they turned a frustratingly forgetful goldfish into a genuinely useful research partner.

Want to try this yourself?

Check out:

Hindsight GitHub for agent memory

Hindsight docs to get started

Vectorize agent memory for more on memory systems

cascadeflow GitHub for runtime intelligence

cascadeflow docs to start routing

The best part? The agent doesn't just recommend papers anymore. It remembers what I've read, understands my research interests, and helps me discover new papers I'll actually care about. And it does it without breaking my API budget.

That's the kind of assistant I can actually use.

Built with Hindsight for memory and cascadeflow for runtime intelligence.