DEV Community: Affan Raza

Voice2Note: I Built a Local AI Meeting Assistant for Freelancers

Affan Raza — Fri, 16 Oct 2026 10:27:24 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle.

What I Built

Voice2Note is a local-first AI meeting follow-up assistant built for freelance developers, designers, and consultants.

Every client conversation can contain important decisions, project requirements, deadlines, and commitments. But after a meeting, freelancers still need to review their notes, identify action items, clarify responsibilities, and write follow-up emails.

Voice2Note turns that workflow into a structured process.

With Voice2Note, users can:

Record or import voice recordings and generate transcripts.
Extract meeting summaries, decisions, tasks, and client commitments.
Distinguish confirmed deadlines from inferred or tentative dates.
Identify unresolved client questions before starting implementation.
Generate editable follow-up email drafts.
Search across previous conversations using semantic search and grounded AI answers.
Export tasks as CSV and meeting summaries as Markdown.

The application is designed to help freelancers reduce repetitive administrative work while retaining control over confidential client information.

Who would pay for it?

Independent developers, consultants, designers, and small agencies that regularly conduct client calls and need a reliable way to manage their follow-ups.

The business hypothesis is straightforward: if Voice2Note saves freelancers time after each meeting and helps them avoid missed commitments, a subscription could be worthwhile. That willingness to pay still needs validation with real users.

Demo

Live demo : https://voice2note-one.vercel.app

The demo should show a client-call recording being processed, the extracted tasks and commitments, deadline classification, a follow-up email draft, and the final export.

Code

GitHub repository: https://github.com/affanraza84/voice2note

The repository contains the application source code, setup instructions, technical documentation, and evaluation resources.

How I Built It

Voice2Note combines a Next.js application with local AI inference, structured extraction, and semantic retrieval.

Technology stack

Next.js, React, TypeScript: Application interface and server-side functionality.
Whisper (Xenova/whisper-tiny.en): Local speech-to-text transcription using Transformers.js and ONNX.
Llama 3.2 3B via Ollama: Local language-model inference for structured extraction and meeting intelligence.
all-MiniLM-L6-v2: Local text embeddings for semantic search.
SQLite: Embedded storage for application data and vector embeddings.
Zod: Validation of structured AI outputs.

The processing pipeline transforms audio into a transcript, extracts structured information, and indexes the content for future retrieval.

One important design decision is to preserve evidence from the transcript alongside extracted tasks and commitments. Users should be able to verify what the AI found instead of blindly trusting a generated summary.

Voice2Note also distinguishes between explicit dates and inferred timeframes. A tentative statement such as “probably sometime next week” should not silently become a confirmed deadline.

The project includes privacy-oriented processing and storage configuration. The local-private workflow is intended to keep recordings and derived information on the user's machine; this guarantee should be verified against the actual runtime configuration before relying on it for confidential client material.

Why Does Open Innovation Matter?

Open innovation is central to Voice2Note because privacy, cost, and control are important for independent professionals.

1. Greater control over confidential information

Local inference makes it possible to process recordings without sending their contents to a hosted AI inference API, provided the local-only configuration is correctly enforced and verified.

2. Lower marginal inference costs

Once the required models are installed, local inference can avoid per-minute transcription and per-token inference charges. Hardware, electricity, setup, and maintenance still have costs, so local processing is not literally cost-free.

3. Model flexibility

Open-weight models and modular provider interfaces let developers evaluate alternative models, adjust prompts, and choose inference runtimes based on their hardware and requirements.

4. Transparency and reproducibility

The model choices, application code, processing pipeline, and evaluation approach can be inspected and tested. This gives developers more control than relying entirely on a closed hosted service.

For this project, open innovation is not just a technology preference. It is part of the product's value proposition: helping freelancers manage their client conversations without automatically making a third-party AI provider the custodian of that information.

devchallenge #hf26challenge #opensource #ai

Bird Explorer: An AI Birdwatching App That Gets You Outside

Affan Raza — Sun, 11 Oct 2026 18:06:23 +0000

Bird Explorer: An AI Birdwatching App That Gets You Outside

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass.

What I Built

What if AI didn't keep you glued to your screen, but helped you leave it?

That question inspired me to build Bird Explorer, a local-first, AI-powered birdwatching application designed to turn curiosity about nature into real-world exploration.

Most AI applications encourage us to spend more time interacting with a screen. Bird Explorer takes a different approach: it uses AI to help people identify birds, then encourages them to put their phones away and experience the outdoors.

The idea is simple: use AI as a bridge to the physical world, not a destination.

What can you do with Bird Explorer?

AI Bird Identification: Upload a bird photograph and identify the species using a locally running vision model.
Outdoor Quests: Generate birdwatching missions for 15, 30, or 60 minutes, with distance targets that encourage walking and exploration.
GPS-Based Progress: Track walking distance locally and use it to measure quest progress.
Bird Journal: Save observations, photographs, descriptions, and confidence information in browser storage.
Audio Field Notes: Record short audio notes while observing birds outdoors.
Regional Bird Checklist: Explore 48 bird species selected for Jharkhand and the wider eastern and central Indian region.
Manual Field Observations: Record sightings without photographs, including habitat, weather, and bird behaviour.
Offline-First Experience: Access saved observations, checklist data, journal filters, exports, and supported PWA features without a continuous internet connection.

Bird Explorer is designed for birdwatchers, nature enthusiasts, students, families, and anyone who wants a practical reason to step outside.

The goal isn't to spend an hour looking at a bird through a phone. It's to use technology briefly, explore the world, and record discoveries when needed.

Demo

Live application: https://bird-explorer-phi.vercel.app

Demo video: https://drive.google.com/file/d/1ckssDJGcTO5IQI1Ccgd-OfuprqTO-QIc/view?usp=sharing

The project demonstrates the intended experience: identify a bird, start an outdoor quest, explore with GPS-based progress, record observations, and preserve discoveries in a local journal.

The application also includes a regional checklist, manual field observations, audio notes, and offline-first functionality.

Deployment note: The live application cannot automatically access Ollama running on my personal computer. AI identification and AI-generated quests require the relevant API to reach a running Ollama service. The local AI workflow is demonstrated using Ollama and the models described below.

Code

GitHub repository: https://github.com/affanraza84/bird-explorer

Bird Explorer is built with Next.js and TypeScript, with local AI inference powered by Ollama.

The implementation includes:

Bird identification and quest-generation API routes
GPS distance calculation and validation
IndexedDB-based observation storage
Audio recording and playback
Regional bird checklist and manual observation workflows
PWA manifest, service worker, and offline navigation support
Journal search, filtering, statistics, and JSON/CSV exports

The project includes automated tests for journal persistence, quest logic, GPS validation, audio notes, exports, and field observations.

How I Built It

I developed Bird Explorer incrementally, starting with bird identification and expanding it into a broader outdoor observation tool.

1. Local AI Bird Identification

I used Ollama to run open-weight models locally:

qwen2.5vl:3b for image-based bird identification.
qwen3:4b for text generation, including outdoor quest generation.

The Next.js API validates an uploaded image and sends it to the locally running Ollama service. The vision model returns identification information for the application to display.

This avoids requiring a proprietary cloud vision API for the local identification workflow.

2. Outdoor Quests

I built a quest system with 15-, 30-, and 60-minute durations and corresponding walking-distance targets:

15 minutes: 500 metres
30 minutes: 1 kilometre
60 minutes: 2 kilometres

Quest progress combines task completion, unique species discoveries, and walking distance. Repeated observations of the same species do not artificially inflate the unique-species count.

3. Privacy-First GPS

The GPS system uses the Haversine formula to calculate distances between location samples.

It accumulates distance across successive movement segments and filters noisy readings, including inaccurate GPS samples, small stationary movements, and implausible speed spikes.

GPS processing happens in the browser. Exact coordinates are kept in transient memory for distance calculations rather than being persisted or transmitted to a backend.

4. Local Bird Journal

I used IndexedDB to store bird observations and image Blobs in the browser.

The journal supports searching, filtering, observation details, metadata editing, and JSON/CSV exports. Audio field notes are stored alongside observations, with schema migrations designed to preserve existing records.

The manual observation workflow means users can record a sighting even when they cannot photograph a bird.

5. Offline-First PWA

I added a web app manifest, service worker, installation prompt, offline status indicator, and navigation fallback.

Previously saved observations and supported journal features remain accessible offline.

One important distinction: offline application access does not automatically mean offline AI inference on every device. AI identification and model-generated quests require the Next.js API to reach a running Ollama service.

Technology Stack

Frontend: Next.js, React, TypeScript, Tailwind CSS
AI runtime: Ollama
AI models: Qwen2.5-VL 3B and Qwen3 4B
Storage: IndexedDB and localStorage
Browser APIs: Geolocation, MediaRecorder, Service Worker
Testing: Automated tests, ESLint, TypeScript, production build
Deployment: Vercel for the web application

I used an AI coding agent throughout the implementation process, working through multiple milestones from initial bird identification to GPS validation, offline support, and regional field observations.

Why Does Open Innovation Matter?

Open innovation matters because it changes where inference happens and who controls the data.

Privacy and Data Ownership

A bird photograph can reveal more than the bird itself. It may also capture someone's surroundings or other contextual information.

With local inference, bird photographs can be processed on the user's own machine without sending them to a proprietary cloud AI provider. The journal also uses browser storage rather than a cloud-hosted journal database.

The application is designed to avoid transmitting or persisting raw GPS coordinates.

Control Over the AI Stack

Ollama and downloadable open-weight models give me more control over the inference environment.

I can experiment with compatible models, change model versions, and evaluate different trade-offs without rebuilding the entire application around one proprietary AI API.

This makes the AI layer more flexible and gives developers greater control over how models are integrated into their applications.

Cost and Accessibility

Local inference avoids per-request fees from a hosted AI API, although it still requires compatible hardware, storage, and electricity.

Users can run the models on suitable local hardware without needing a dedicated cloud AI account.

AI That Encourages Real-World Activity

The most important benefit isn't simply that the model runs locally. It's that the application is designed around an outdoor activity.

The AI helps initiate the experience, but the actual value comes from walking, observing wildlife, learning species, and recording discoveries.

The screen should be the shortest part of the experience.

My Agent Session

I documented the development process in my DevRelay Agent Session, covering the implementation milestones, technical decisions, and testing workflow.

Explore the Bird Explorer Agent Session — Milestones 1–7

The session provides additional context on how the application evolved from a bird-identification prototype into a broader outdoor observation tool.

What I Learned

Building Bird Explorer reinforced an important engineering lesson: adding AI is not the same as building a useful AI product.

The work involved connecting model inference with reliable application behaviour, validating structured output, handling model failures, preserving browser data through schema migrations, filtering noisy GPS readings, and making the experience useful when offline.

I also learned to separate AI-generated suggestions from deterministic application logic. The model can suggest a quest, but the application should remain responsible for validation, progress tracking, and completion rules.

What's Next?

Potential improvements include expanding the regional bird database, improving species-identification evaluation, adding richer habitat-distribution insights, and supporting standardized wildlife-observation exports.

I would also like to test the application during real birdwatching sessions and gather feedback about whether the quests genuinely encourage people to spend more time outdoors.

Final Thoughts

Bird Explorer began with a simple idea: technology should sometimes help us use technology less.

By combining local AI, GPS-guided quests, field observations, and offline-first storage, I wanted to build something that makes stepping outside a little more interesting.

The goal isn't to watch more birds on a screen. It's to discover more birds in the real world.

devchallenge #hf26challenge

Voice2Note — Give Your Voice Notes a Memory

Affan Raza — Sun, 04 Oct 2026 11:15:05 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

I built Voice2Note, a local-first AI application that turns voice recordings into structured, searchable knowledge.

I built it for a friend who frequently records voice notes to capture ideas, plans, reminders, and thoughts, but often struggles to find or act on that information later.

The problem isn't recording a voice note.

The problem is what happens after the recording.

Important ideas get buried. Tasks get forgotten. Finding something mentioned several days ago becomes difficult.

Voice2Note turns that unstructured voice data into something useful:

Voice Recording
      ↓
Local Speech-to-Text
      ↓
Open-Weight LLM
      ↓
Summary + Tasks + Ideas + Decisions
      ↓
Local Knowledge Base
      ↓
Semantic Search
      ↓
Ask My Notes

Instead of simply transcribing a recording, Voice2Note helps answer:

"What did I say about this before?"

Key Features

🎙️ Record voice notes directly from the browser
📁 Upload existing audio recordings
📝 Local speech-to-text transcription
🧠 AI-generated summaries
✅ Automatic task extraction
💡 Idea and decision extraction
🔎 Semantic search across previous notes
💬 Ask questions about your own voice notes
📌 Source-backed answers
🔒 Local-first/private AI architecture
⚙️ Swappable open-weight AI models

The goal was to build something small enough to actually use, but meaningful enough to solve a real problem for one person.

Demo

🌐 Live Demo: https://voice2note-one.vercel.app

The demo shows the complete workflow:

Record → Transcribe → Understand → Search → Ask

Code

💻 GitHub: https://github.com/affanraza84/voice2note

The repository contains the complete application, local AI setup, architecture documentation, evaluation tests, and setup instructions.

How I Built It

Voice2Note is built around open-source/open-weight AI, rather than using a proprietary AI API as the core intelligence layer.

The architecture consists of several AI components:

1. Local Speech Recognition

Voice recordings are processed using an open speech-recognition model such as Whisper/faster-whisper.

Audio
 ↓
Local Speech Model
 ↓
Transcript

This means the application doesn't need to send a private voice recording to a third-party speech API for its core transcription workflow.

2. Open-Weight LLM

The transcript is passed to a locally running open-weight language model.

The model extracts structured information such as:

summaries
tasks
ideas
decisions
people
topics
important dates

For example:

{
  "summary": "Discussed the college project and deployment plan.",
  "tasks": [
    "Finish the landing page",
    "Deploy the backend"
  ],
  "ideas": [
    "Add semantic search to the project"
  ]
}

3. Local Knowledge Base

The transcripts are chunked and converted into embeddings.

Transcript
    ↓
Chunking
    ↓
Embeddings
    ↓
Vector Store

This allows Voice2Note to understand semantic relationships instead of relying only on keyword matching.

4. Retrieval-Augmented Generation

When the user asks:

"What did I say about my project?"

Voice2Note retrieves relevant sections from previous voice notes and provides them as context to the local LLM.

Question
   ↓
Embedding
   ↓
Semantic Retrieval
   ↓
Relevant Voice Notes
   ↓
Local LLM
   ↓
Grounded Answer

The response also points back to the source note so the user can verify where the information came from.

Why Does Open Innovation Matter?

This is the most important part of Voice2Note.

Voice recordings can contain extremely personal information.

A voice note might contain:

personal plans
private conversations
project ideas
financial information
family information
unfinished thoughts

I didn't want the fundamental value of the application to depend on sending that information to a proprietary AI provider.

With an open/local AI architecture, Voice2Note can keep the core AI pipeline under the user's control:

Microphone
    ↓
Local Speech Model
    ↓
Local LLM
    ↓
Local Embeddings
    ↓
Local Knowledge Base

Open AI made several things possible.

Privacy

The application can process sensitive voice data locally instead of requiring a cloud AI API for its core intelligence.

Model freedom

The application isn't permanently tied to one model provider. The AI provider is abstracted so compatible models can be swapped.

Customization

Because the AI pipeline is under our control, we can customize transcription, extraction, retrieval, prompting, and model behavior around the actual user's workflow.

Lower recurring cost

Once the required models are available locally, the core inference pipeline doesn't require paying a per-request proprietary AI API.

Transparency

I can understand and control the components responsible for processing the user's data instead of treating the AI layer as a black box.

There are trade-offs, of course.

Local AI can require more hardware, model setup can be more complicated, and some closed models may provide better performance for particular tasks.

But for this application, control and privacy are more important than simply choosing the strongest available API.

That's where open innovation made the most sense.

What I Learned

The biggest lesson from building Voice2Note was that AI doesn't have to mean building another chatbot.

The interesting part was designing the pipeline around a real person's workflow:

Raw Voice
   ↓
Information
   ↓
Structure
   ↓
Memory
   ↓
Action

The AI is valuable because it transforms something the user already does—recording voice notes—into something much more useful.

I also learned how different AI components can work together:

speech recognition
structured LLM extraction
embeddings
vector search
RAG
local inference
source attribution

rather than treating a single LLM call as the entire application.

Final Thoughts

Voice2Note started with a simple observation:

My friend already had the information. They just couldn't easily find or use it.

So instead of building another place to create more information, I built a tool that helps turn the information they already create into something searchable, structured, and actionable.

That's what Build for a Friend meant to me.

Not building the biggest AI application.

Building something that one real person would actually want to keep using.

DevDocs Intelligence: A Gemini Agent for Version-Aware Next.js Documentation

Affan Raza — Sat, 03 Oct 2026 14:39:46 +0000

DevDocs Intelligence: A Gemini Agent for Version-Aware Next.js Documentation

A developer documentation intelligence agent powered by Gemini, Sanity Knowledge Base, and Sanity Context MCP.

What I Built

I built DevDocs Intelligence, a Gemini-powered agent designed to answer complex, version-aware questions about Next.js documentation.

The core idea is simple: developer documentation shouldn't always be treated as a flat collection of text.

A question such as:

“Explain the major routing changes between Next.js 15 and Next.js 16, including params, Route Handlers, and the transition from middleware.ts to proxy.ts.”

can require information from multiple documentation types and versions.

So instead of storing everything as unstructured text, I modeled the knowledge as structured, interconnected content in Sanity and exposed it to the agent through Sanity Context MCP.

The resulting architecture is:

User Question
      ↓
Next.js Frontend
      ↓
Application API
      ↓
Gemini Agent
      ↓
Sanity Context MCP
      ↓
Sanity Knowledge Base
      ↓
Structured Sanity Content
      ↓
Grounded Answer + Sources

Why I Built It

Framework documentation changes quickly.

A developer asking a version-specific question may need to understand:

what changed between versions
whether an API behavior changed
whether a feature was deprecated
how to migrate
which APIs or features are affected
how different pieces of documentation relate to each other

A simple keyword search can find matching text, but it doesn't necessarily preserve these relationships.

For DevDocs Intelligence, I wanted to explore a different approach:

Model the documentation itself as structured knowledge and let an AI agent retrieve that knowledge through Sanity Context MCP.

The Structured Content Model

I created four custom Sanity document types:

Content Type	Purpose
Documentation	General Next.js concepts, guides, and feature documentation
API Reference	APIs, parameters, examples, limitations, and related documentation
Migration Guide	Version-to-version changes, breaking changes, migration steps, and affected APIs
Release Note	Version-specific changes, breaking changes, deprecations, and affected features

These documents aren't isolated.

They contain explicit references to related documentation, APIs, migrations, and release notes.

The current corpus contains:

30 structured documents
4 custom content types
128 cross-document relationships

This gives the retrieval layer more context than simply searching a collection of independent text chunks.

Why Sanity?

Sanity is the structured content foundation of the project.

Instead of treating a documentation page as one large piece of text, I can represent different types of knowledge using different schemas and connect them through references.

For example:

Migration Guide
      │
      ├──→ API Reference
      │
      ├──→ Documentation
      │
      └──→ Release Note

This becomes particularly useful for version-aware questions where the answer may span multiple sources.

Sanity's structured content model therefore becomes part of the agent's reasoning context rather than simply acting as a place to store documents.

Sanity Knowledge Base

The structured Sanity corpus is connected to a Sanity Knowledge Base designed specifically for answering complex Next.js documentation questions.

The Knowledge Base contains:

30 source documents
18 Knowledge Base entries
100% source span coverage in the successful build

The goal is to make the structured documentation available as navigable contextual knowledge for the agent.

How Sanity Context MCP Fits In

MCP (Model Context Protocol) provides the interface between the Gemini agent and Sanity's contextual knowledge.

The agent does not directly contain the documentation.

Instead:

Gemini Agent
     │
     │ MCP tool calls
     ▼
Sanity Context MCP
     │
     ▼
Knowledge Base
     │
     ▼
Structured Sanity Content

During testing, the agent used Sanity Context MCP operations including:

initial_context
knowledge_base_search
knowledge_base_read

This allows the agent to retrieve relevant information when it needs additional context instead of placing the entire documentation corpus into the model prompt.

How the Agent Works

When a developer submits a question:

1. The user asks a question

For example:

Explain the major routing changes between Next.js 15 and Next.js 16, including params, Route Handlers, and the transition from middleware.ts to proxy.ts.

2. The Next.js frontend sends the request

The frontend communicates with the application's chat API.

3. The backend initializes the agent

The backend handles the request, validates the input, and connects the agent to the configured Sanity Context MCP endpoint.

4. Gemini determines what context is needed

Gemini acts as the reasoning and generation layer.

It can use the MCP-exposed tools to retrieve relevant contextual information.

5. Sanity Context retrieves the knowledge

The MCP layer accesses the Sanity Knowledge Base and retrieves relevant structured content.

The retrieved information can span multiple document types.

6. Gemini synthesizes the answer

Gemini uses the retrieved content to produce a coherent, grounded response.

7. The response is streamed to the frontend

The backend streams the response progressively to the UI.

8. Sources are displayed

The frontend shows the Sanity sources used to ground the response.

Architecture

                         ┌──────────────────┐
                         │       User       │
                         └────────┬─────────┘
                                  │
                                  ▼
                         ┌──────────────────┐
                         │ Next.js Frontend │
                         │  TypeScript/UI   │
                         └────────┬─────────┘
                                  │
                                  ▼
                         ┌──────────────────┐
                         │ Application API  │
                         │ Agent Orchestration
                         └────────┬─────────┘
                                  │
                                  ▼
                         ┌──────────────────┐
                         │   Gemini Agent   │
                         │ Reasoning +      │
                         │ Generation       │
                         └────────┬─────────┘
                                  │
                             MCP Tool Calls
                                  │
                                  ▼
                         ┌──────────────────┐
                         │ Sanity Context  │
                         │       MCP       │
                         └────────┬─────────┘
                                  │
                                  ▼
                         ┌──────────────────┐
                         │ Sanity Knowledge │
                         │      Base       │
                         └────────┬─────────┘
                                  │
                                  ▼
                   ┌────────────────────────────┐
                   │ Structured Sanity Content │
                   │                            │
                   │ Documentation              │
                   │ API References             │
                   │ Migration Guides           │
                   │ Release Notes              │
                   └────────────────────────────┘

A Real Example

One of the main queries I use to demonstrate the system is:

Explain the major routing changes between Next.js 15 and Next.js 16, including params, Route Handlers, and the transition from middleware.ts to proxy.ts.

This is useful because the answer requires more than retrieving one isolated paragraph.

The agent can retrieve information across:

Documentation
API Reference
Migration Guide
Release Note

The resulting response is grounded in Sanity sources and displayed alongside the answer in the application.

Other Example Queries

Next.js 15 caching

What is the default dynamic staleTime in Next.js 15 and how did it change from previous versions?

Route Handlers

How did Route Handler caching behavior change in Next.js 15?

Version migration

What are the major changes developers should know when migrating from Next.js 14 to Next.js 15?

These queries test whether the agent can connect version-specific information rather than simply return a matching keyword.

Key Features

Gemini-powered documentation agent
Sanity structured content
Sanity Knowledge Base
Sanity Context MCP integration
Four domain-specific content types
Cross-document relationships
Version-aware retrieval
Grounded responses
Sanity source display
Progressive response streaming
Server-side secret management
Error handling and validation
Production deployment on Vercel

Tech Stack

Frontend

Next.js
TypeScript
Tailwind CSS

AI

Gemini
AI SDK

Content & Retrieval

Sanity
Sanity Knowledge Base
Sanity Context MCP

Deployment

Vercel

Technical Implementation

The application is divided into a few major layers:

frontend/
    ↓
Next.js application
    ↓
Chat API
    ↓
Agent layer
    ↓
Gemini + MCP
    ↓
Sanity Context
    ↓
Knowledge Base

The backend keeps sensitive credentials server-side, including:

GEMINI_API_KEY
SANITY_API_READ_TOKEN
SANITY_CONTEXT_MCP_URL

These are not exposed as public frontend environment variables.

Streaming

The application supports progressive responses rather than waiting for the entire answer to finish.

The flow is:

Gemini
   ↓
Agent Backend
   ↓
Streaming/SSE
   ↓
Next.js Frontend
   ↓
Progressive UI Response

This makes the interaction feel more like an active agent session rather than a request that remains blocked until the complete response is generated.

Testing & Validation

I tested the complete workflow with real questions against the hosted Sanity Context MCP endpoint.

The validation covered:

Sanity Context MCP tool invocation
Knowledge Base retrieval
Multi-document retrieval
Grounded Sanity sources
Streaming responses
Empty/malformed request handling
Server-side secret handling
TypeScript validation
Production build

The agent was tested with both focused documentation questions and higher-value cross-document questions involving version changes and related APIs.

Sanity Project Details

Sanity Project ID: 0ovd9f2s

Dataset: production

Content Types:

documentation
apiReference
migrationGuide
releaseNote

Structured Documents: 30

Cross-document Relationships: 128

Knowledge Base Entries: 18

What I Learned

One of the biggest takeaways from this project was that the structure of the knowledge matters as much as the model used to reason over it.

A documentation agent becomes much more useful when the underlying information contains explicit relationships between:

versions
APIs
migrations
release changes
documentation

Sanity made it possible to model those relationships explicitly and expose the resulting knowledge to an agent through Context MCP.

This also changed how I think about RAG systems: retrieval doesn't always have to mean searching a flat collection of chunks. The underlying content model can itself provide useful context.

Future Improvements

Some areas I would explore next:

Expand the corpus beyond Next.js
Add more framework versions
Add evaluation datasets for retrieval quality
Measure grounded-answer accuracy systematically
Add richer source-level explanations
Explore more complex multi-hop documentation queries

Built for the Sanity Challenge

This project was built for the Sanity Challenge 2026 — Path One: Ship an agent that queries real content.

The core implementation focuses on using:

Structured Sanity Content → Knowledge Base → Sanity Context MCP → Gemini Agent

rather than treating Sanity as only a storage layer.

Links

Live Demo:

https://devdocs-intelligence.vercel.app

Source Code:

https://github.com/affanraza84/devdocs-intelligence

Final Note

DevDocs Intelligence is an exploration of what happens when developer documentation is treated as structured, connected knowledge instead of just a collection of searchable pages.

The goal is not simply to make an LLM answer documentation questions, but to give the agent a structured knowledge layer it can actually reason over.

DEV Community: Affan Raza

Voice2Note: I Built a Local AI Meeting Assistant for Freelancers

What I Built

Demo

Code

How I Built It

Why Does Open Innovation Matter?

Tags

devchallenge #hf26challenge #opensource #ai

Bird Explorer: An AI Birdwatching App That Gets You Outside

Bird Explorer: An AI Birdwatching App That Gets You Outside

What I Built

What can you do with Bird Explorer?

Demo

Code

How I Built It

1. Local AI Bird Identification

2. Outdoor Quests

3. Privacy-First GPS

4. Local Bird Journal

5. Offline-First PWA

Technology Stack

Why Does Open Innovation Matter?

Privacy and Data Ownership

Control Over the AI Stack

Cost and Accessibility

AI That Encourages Real-World Activity

My Agent Session

What I Learned

What's Next?

Final Thoughts

devchallenge #hf26challenge

Voice2Note — Give Your Voice Notes a Memory

What I Built

Key Features

Demo

Code

How I Built It

1. Local Speech Recognition

2. Open-Weight LLM

3. Local Knowledge Base

4. Retrieval-Augmented Generation

Why Does Open Innovation Matter?

Open AI made several things possible.

What I Learned

Final Thoughts

DevDocs Intelligence: A Gemini Agent for Version-Aware Next.js Documentation

DevDocs Intelligence: A Gemini Agent for Version-Aware Next.js Documentation

What I Built

Why I Built It

The Structured Content Model

Why Sanity?

Sanity Knowledge Base

How Sanity Context MCP Fits In

How the Agent Works

1. The user asks a question

2. The Next.js frontend sends the request

3. The backend initializes the agent

4. Gemini determines what context is needed

5. Sanity Context retrieves the knowledge

6. Gemini synthesizes the answer

7. The response is streamed to the frontend

8. Sources are displayed

Architecture

A Real Example

Other Example Queries

Next.js 15 caching

Route Handlers

Version migration

Key Features

Tech Stack

Frontend

AI

Content & Retrieval

Deployment

Technical Implementation

Streaming

Testing & Validation

Sanity Project Details

What I Learned