DEV Community: Affan Raza

Voice2Note — Give Your Voice Notes a Memory

Affan Raza — Sun, 04 Oct 2026 11:15:05 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

I built Voice2Note, a local-first AI application that turns voice recordings into structured, searchable knowledge.

I built it for a friend who frequently records voice notes to capture ideas, plans, reminders, and thoughts, but often struggles to find or act on that information later.

The problem isn't recording a voice note.

The problem is what happens after the recording.

Important ideas get buried. Tasks get forgotten. Finding something mentioned several days ago becomes difficult.

Voice2Note turns that unstructured voice data into something useful:

Voice Recording
      ↓
Local Speech-to-Text
      ↓
Open-Weight LLM
      ↓
Summary + Tasks + Ideas + Decisions
      ↓
Local Knowledge Base
      ↓
Semantic Search
      ↓
Ask My Notes

Instead of simply transcribing a recording, Voice2Note helps answer:

"What did I say about this before?"

Key Features

🎙️ Record voice notes directly from the browser
📁 Upload existing audio recordings
📝 Local speech-to-text transcription
🧠 AI-generated summaries
✅ Automatic task extraction
💡 Idea and decision extraction
🔎 Semantic search across previous notes
💬 Ask questions about your own voice notes
📌 Source-backed answers
🔒 Local-first/private AI architecture
⚙️ Swappable open-weight AI models

The goal was to build something small enough to actually use, but meaningful enough to solve a real problem for one person.

Demo

🌐 Live Demo: https://voice2note-one.vercel.app

The demo shows the complete workflow:

Record → Transcribe → Understand → Search → Ask

Code

💻 GitHub: https://github.com/affanraza84/voice2note

The repository contains the complete application, local AI setup, architecture documentation, evaluation tests, and setup instructions.

How I Built It

Voice2Note is built around open-source/open-weight AI, rather than using a proprietary AI API as the core intelligence layer.

The architecture consists of several AI components:

1. Local Speech Recognition

Voice recordings are processed using an open speech-recognition model such as Whisper/faster-whisper.

Audio
 ↓
Local Speech Model
 ↓
Transcript

This means the application doesn't need to send a private voice recording to a third-party speech API for its core transcription workflow.

2. Open-Weight LLM

The transcript is passed to a locally running open-weight language model.

The model extracts structured information such as:

summaries
tasks
ideas
decisions
people
topics
important dates

For example:

{
  "summary": "Discussed the college project and deployment plan.",
  "tasks": [
    "Finish the landing page",
    "Deploy the backend"
  ],
  "ideas": [
    "Add semantic search to the project"
  ]
}

3. Local Knowledge Base

The transcripts are chunked and converted into embeddings.

Transcript
    ↓
Chunking
    ↓
Embeddings
    ↓
Vector Store

This allows Voice2Note to understand semantic relationships instead of relying only on keyword matching.

4. Retrieval-Augmented Generation

When the user asks:

"What did I say about my project?"

Voice2Note retrieves relevant sections from previous voice notes and provides them as context to the local LLM.

Question
   ↓
Embedding
   ↓
Semantic Retrieval
   ↓
Relevant Voice Notes
   ↓
Local LLM
   ↓
Grounded Answer

The response also points back to the source note so the user can verify where the information came from.

Why Does Open Innovation Matter?

This is the most important part of Voice2Note.

Voice recordings can contain extremely personal information.

A voice note might contain:

personal plans
private conversations
project ideas
financial information
family information
unfinished thoughts

I didn't want the fundamental value of the application to depend on sending that information to a proprietary AI provider.

With an open/local AI architecture, Voice2Note can keep the core AI pipeline under the user's control:

Microphone
    ↓
Local Speech Model
    ↓
Local LLM
    ↓
Local Embeddings
    ↓
Local Knowledge Base

Open AI made several things possible.

Privacy

The application can process sensitive voice data locally instead of requiring a cloud AI API for its core intelligence.

Model freedom

The application isn't permanently tied to one model provider. The AI provider is abstracted so compatible models can be swapped.

Customization

Because the AI pipeline is under our control, we can customize transcription, extraction, retrieval, prompting, and model behavior around the actual user's workflow.

Lower recurring cost

Once the required models are available locally, the core inference pipeline doesn't require paying a per-request proprietary AI API.

Transparency

I can understand and control the components responsible for processing the user's data instead of treating the AI layer as a black box.

There are trade-offs, of course.

Local AI can require more hardware, model setup can be more complicated, and some closed models may provide better performance for particular tasks.

But for this application, control and privacy are more important than simply choosing the strongest available API.

That's where open innovation made the most sense.

What I Learned

The biggest lesson from building Voice2Note was that AI doesn't have to mean building another chatbot.

The interesting part was designing the pipeline around a real person's workflow:

Raw Voice
   ↓
Information
   ↓
Structure
   ↓
Memory
   ↓
Action

The AI is valuable because it transforms something the user already does—recording voice notes—into something much more useful.

I also learned how different AI components can work together:

speech recognition
structured LLM extraction
embeddings
vector search
RAG
local inference
source attribution

rather than treating a single LLM call as the entire application.

Final Thoughts

Voice2Note started with a simple observation:

My friend already had the information. They just couldn't easily find or use it.

So instead of building another place to create more information, I built a tool that helps turn the information they already create into something searchable, structured, and actionable.

That's what Build for a Friend meant to me.

Not building the biggest AI application.

Building something that one real person would actually want to keep using.

DevDocs Intelligence: A Gemini Agent for Version-Aware Next.js Documentation

Affan Raza — Sat, 03 Oct 2026 14:39:46 +0000

DevDocs Intelligence: A Gemini Agent for Version-Aware Next.js Documentation

A developer documentation intelligence agent powered by Gemini, Sanity Knowledge Base, and Sanity Context MCP.

What I Built

I built DevDocs Intelligence, a Gemini-powered agent designed to answer complex, version-aware questions about Next.js documentation.

The core idea is simple: developer documentation shouldn't always be treated as a flat collection of text.

A question such as:

“Explain the major routing changes between Next.js 15 and Next.js 16, including params, Route Handlers, and the transition from middleware.ts to proxy.ts.”

can require information from multiple documentation types and versions.

So instead of storing everything as unstructured text, I modeled the knowledge as structured, interconnected content in Sanity and exposed it to the agent through Sanity Context MCP.

The resulting architecture is:

User Question
      ↓
Next.js Frontend
      ↓
Application API
      ↓
Gemini Agent
      ↓
Sanity Context MCP
      ↓
Sanity Knowledge Base
      ↓
Structured Sanity Content
      ↓
Grounded Answer + Sources

Why I Built It

Framework documentation changes quickly.

A developer asking a version-specific question may need to understand:

what changed between versions
whether an API behavior changed
whether a feature was deprecated
how to migrate
which APIs or features are affected
how different pieces of documentation relate to each other

A simple keyword search can find matching text, but it doesn't necessarily preserve these relationships.

For DevDocs Intelligence, I wanted to explore a different approach:

Model the documentation itself as structured knowledge and let an AI agent retrieve that knowledge through Sanity Context MCP.

The Structured Content Model

I created four custom Sanity document types:

Content Type	Purpose
Documentation	General Next.js concepts, guides, and feature documentation
API Reference	APIs, parameters, examples, limitations, and related documentation
Migration Guide	Version-to-version changes, breaking changes, migration steps, and affected APIs
Release Note	Version-specific changes, breaking changes, deprecations, and affected features

These documents aren't isolated.

They contain explicit references to related documentation, APIs, migrations, and release notes.

The current corpus contains:

30 structured documents
4 custom content types
128 cross-document relationships

This gives the retrieval layer more context than simply searching a collection of independent text chunks.

Why Sanity?

Sanity is the structured content foundation of the project.

Instead of treating a documentation page as one large piece of text, I can represent different types of knowledge using different schemas and connect them through references.

For example:

Migration Guide
      │
      ├──→ API Reference
      │
      ├──→ Documentation
      │
      └──→ Release Note

This becomes particularly useful for version-aware questions where the answer may span multiple sources.

Sanity's structured content model therefore becomes part of the agent's reasoning context rather than simply acting as a place to store documents.

Sanity Knowledge Base

The structured Sanity corpus is connected to a Sanity Knowledge Base designed specifically for answering complex Next.js documentation questions.

The Knowledge Base contains:

30 source documents
18 Knowledge Base entries
100% source span coverage in the successful build

The goal is to make the structured documentation available as navigable contextual knowledge for the agent.

How Sanity Context MCP Fits In

MCP (Model Context Protocol) provides the interface between the Gemini agent and Sanity's contextual knowledge.

The agent does not directly contain the documentation.

Instead:

Gemini Agent
     │
     │ MCP tool calls
     ▼
Sanity Context MCP
     │
     ▼
Knowledge Base
     │
     ▼
Structured Sanity Content

During testing, the agent used Sanity Context MCP operations including:

initial_context
knowledge_base_search
knowledge_base_read

This allows the agent to retrieve relevant information when it needs additional context instead of placing the entire documentation corpus into the model prompt.

How the Agent Works

When a developer submits a question:

1. The user asks a question

For example:

Explain the major routing changes between Next.js 15 and Next.js 16, including params, Route Handlers, and the transition from middleware.ts to proxy.ts.

2. The Next.js frontend sends the request

The frontend communicates with the application's chat API.

3. The backend initializes the agent

The backend handles the request, validates the input, and connects the agent to the configured Sanity Context MCP endpoint.

4. Gemini determines what context is needed

Gemini acts as the reasoning and generation layer.

It can use the MCP-exposed tools to retrieve relevant contextual information.

5. Sanity Context retrieves the knowledge

The MCP layer accesses the Sanity Knowledge Base and retrieves relevant structured content.

The retrieved information can span multiple document types.

6. Gemini synthesizes the answer

Gemini uses the retrieved content to produce a coherent, grounded response.

7. The response is streamed to the frontend

The backend streams the response progressively to the UI.

8. Sources are displayed

The frontend shows the Sanity sources used to ground the response.

Architecture

                         ┌──────────────────┐
                         │       User       │
                         └────────┬─────────┘
                                  │
                                  ▼
                         ┌──────────────────┐
                         │ Next.js Frontend │
                         │  TypeScript/UI   │
                         └────────┬─────────┘
                                  │
                                  ▼
                         ┌──────────────────┐
                         │ Application API  │
                         │ Agent Orchestration
                         └────────┬─────────┘
                                  │
                                  ▼
                         ┌──────────────────┐
                         │   Gemini Agent   │
                         │ Reasoning +      │
                         │ Generation       │
                         └────────┬─────────┘
                                  │
                             MCP Tool Calls
                                  │
                                  ▼
                         ┌──────────────────┐
                         │ Sanity Context  │
                         │       MCP       │
                         └────────┬─────────┘
                                  │
                                  ▼
                         ┌──────────────────┐
                         │ Sanity Knowledge │
                         │      Base       │
                         └────────┬─────────┘
                                  │
                                  ▼
                   ┌────────────────────────────┐
                   │ Structured Sanity Content │
                   │                            │
                   │ Documentation              │
                   │ API References             │
                   │ Migration Guides           │
                   │ Release Notes              │
                   └────────────────────────────┘

A Real Example

One of the main queries I use to demonstrate the system is:

Explain the major routing changes between Next.js 15 and Next.js 16, including params, Route Handlers, and the transition from middleware.ts to proxy.ts.

This is useful because the answer requires more than retrieving one isolated paragraph.

The agent can retrieve information across:

Documentation
API Reference
Migration Guide
Release Note

The resulting response is grounded in Sanity sources and displayed alongside the answer in the application.

Other Example Queries

Next.js 15 caching

What is the default dynamic staleTime in Next.js 15 and how did it change from previous versions?

Route Handlers

How did Route Handler caching behavior change in Next.js 15?

Version migration

What are the major changes developers should know when migrating from Next.js 14 to Next.js 15?

These queries test whether the agent can connect version-specific information rather than simply return a matching keyword.

Key Features

Gemini-powered documentation agent
Sanity structured content
Sanity Knowledge Base
Sanity Context MCP integration
Four domain-specific content types
Cross-document relationships
Version-aware retrieval
Grounded responses
Sanity source display
Progressive response streaming
Server-side secret management
Error handling and validation
Production deployment on Vercel

Tech Stack

Frontend

Next.js
TypeScript
Tailwind CSS

AI

Gemini
AI SDK

Content & Retrieval

Sanity
Sanity Knowledge Base
Sanity Context MCP

Deployment

Vercel

Technical Implementation

The application is divided into a few major layers:

frontend/
    ↓
Next.js application
    ↓
Chat API
    ↓
Agent layer
    ↓
Gemini + MCP
    ↓
Sanity Context
    ↓
Knowledge Base

The backend keeps sensitive credentials server-side, including:

GEMINI_API_KEY
SANITY_API_READ_TOKEN
SANITY_CONTEXT_MCP_URL

These are not exposed as public frontend environment variables.

Streaming

The application supports progressive responses rather than waiting for the entire answer to finish.

The flow is:

Gemini
   ↓
Agent Backend
   ↓
Streaming/SSE
   ↓
Next.js Frontend
   ↓
Progressive UI Response

This makes the interaction feel more like an active agent session rather than a request that remains blocked until the complete response is generated.

Testing & Validation

I tested the complete workflow with real questions against the hosted Sanity Context MCP endpoint.

The validation covered:

Sanity Context MCP tool invocation
Knowledge Base retrieval
Multi-document retrieval
Grounded Sanity sources
Streaming responses
Empty/malformed request handling
Server-side secret handling
TypeScript validation
Production build

The agent was tested with both focused documentation questions and higher-value cross-document questions involving version changes and related APIs.

Sanity Project Details

Sanity Project ID: 0ovd9f2s

Dataset: production

Content Types:

documentation
apiReference
migrationGuide
releaseNote

Structured Documents: 30

Cross-document Relationships: 128

Knowledge Base Entries: 18

What I Learned

One of the biggest takeaways from this project was that the structure of the knowledge matters as much as the model used to reason over it.

A documentation agent becomes much more useful when the underlying information contains explicit relationships between:

versions
APIs
migrations
release changes
documentation

Sanity made it possible to model those relationships explicitly and expose the resulting knowledge to an agent through Context MCP.

This also changed how I think about RAG systems: retrieval doesn't always have to mean searching a flat collection of chunks. The underlying content model can itself provide useful context.

Future Improvements

Some areas I would explore next:

Expand the corpus beyond Next.js
Add more framework versions
Add evaluation datasets for retrieval quality
Measure grounded-answer accuracy systematically
Add richer source-level explanations
Explore more complex multi-hop documentation queries

Built for the Sanity Challenge

This project was built for the Sanity Challenge 2026 — Path One: Ship an agent that queries real content.

The core implementation focuses on using:

Structured Sanity Content → Knowledge Base → Sanity Context MCP → Gemini Agent

rather than treating Sanity as only a storage layer.

Links

Live Demo:

https://devdocs-intelligence.vercel.app

Source Code:

https://github.com/affanraza84/devdocs-intelligence

Final Note

DevDocs Intelligence is an exploration of what happens when developer documentation is treated as structured, connected knowledge instead of just a collection of searchable pages.

The goal is not simply to make an LLM answer documentation questions, but to give the agent a structured knowledge layer it can actually reason over.

DEV Community: Affan Raza

Voice2Note — Give Your Voice Notes a Memory

What I Built

Key Features

Demo

Code

How I Built It

1. Local Speech Recognition

2. Open-Weight LLM

3. Local Knowledge Base

4. Retrieval-Augmented Generation

Why Does Open Innovation Matter?

Open AI made several things possible.

What I Learned

Final Thoughts

DevDocs Intelligence: A Gemini Agent for Version-Aware Next.js Documentation

DevDocs Intelligence: A Gemini Agent for Version-Aware Next.js Documentation

What I Built

Why I Built It

The Structured Content Model

Why Sanity?

Sanity Knowledge Base

How Sanity Context MCP Fits In

How the Agent Works

1. The user asks a question

2. The Next.js frontend sends the request

3. The backend initializes the agent

4. Gemini determines what context is needed

5. Sanity Context retrieves the knowledge

6. Gemini synthesizes the answer

7. The response is streamed to the frontend

8. Sources are displayed

Architecture

A Real Example

Other Example Queries

Next.js 15 caching

Route Handlers

Version migration

Key Features

Tech Stack

Frontend

AI

Content & Retrieval

Deployment

Technical Implementation

Streaming

Testing & Validation

Sanity Project Details

What I Learned

Future Improvements

Built for the Sanity Challenge

Links

Final Note

sanitychallenge