DEV Community: Dibyendu Chatterjee

I Built an Altar That Works With the Wifi Off

Dibyendu Chatterjee — Thu, 29 Oct 2026 15:28:52 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine

What I Built

On Día de los Muertos, families build an ofrenda: an altar with marigolds, candles, the food someone loved, the song that was theirs, a photograph. The idea is that the dead stay close as long as someone keeps telling their story.

The stories are the fragile part. They sit in a few heads and a few chat threads. And the obvious tool for turning scattered memories into writing, a cloud AI, asks you to paste your grandmother's last recipe into a company's server.

So I built ofrenda, a command line tool with zero dependencies. You answer eight questions about someone you miss: name, who they were to you, a food, a song, a scent, something they always said, one story you never want to lose, and optionally a photo.

A model running on your own laptop writes a short remembrance from your answers. Then the tool builds one HTML file: a papel picado banner that sways, three flickering candles, marigold petals drifting down, your remembrance typing itself out line by line, and an offerings tier for the food, song, flower, and phrase.

Demo

Code

dibyendu-coder / Ofrenda-CLI

🕯️ ofrenda

A ghost story where the ghost is someone you love.

Answer seven questions about a person you miss. A language model running on your own laptop writes their remembrance. You get a candlelit altar page you can keep forever.

No cloud. No account. No API key. No upload. Nothing leaves your machine.

Built for the Hacktoberfest Open-Source AI Challenge: Ghost in the Machine

The idea

On Día de los Muertos, families build an ofrenda: an altar with the food, the song, the flowers, and the stories of people who have died. The ritual says someone stays present as long as someone tells their story.

The stories are the fragile part. They live in a few heads and a few chat threads. If you hand them to a cloud AI to help you write them down, you upload your grandmother's last recipe to a company's server.

ofrenda keeps…

View on GitHub

How I Built It

Model: gemma3:4b, an open-weight model, served by Ollama on localhost. One environment variable swaps it (OFRENDA_MODEL=llama3.2).
Code: one Node file, no dependencies. It asks the questions, calls Ollama's /api/chat, and renders the altar as a single HTML file with the photo embedded as base64.
Prompts: two. One writes the remembrance, one runs the echo. Both are built around the same rule: use only what the person told you, and leave gaps empty.
Built with: Antigravity

Why Does Open Innovation Matter?

I could not have built this honestly with a closed API.

It ran with the wifi off. I turned the wifi off, ran the whole thing, and it worked. The only time the tool needs internet is the one-time model download.

[SCREENSHOT: terminal with wifi-off icon visible]

Nothing about a dead person left my machine. Answers, photo, chat, and output all stay in a folder on disk. That folder is gitignored so one careless git add . cannot publish a family's private stories.

Nothing can be switched off under it. The output is a plain HTML file with the photo inside it. No framework, no hosted viewer, no account. A memorial that depends on a company keeping an API alive is not much of a memorial. A file you can open in thirty years is.

I could change how it behaves. I swapped models with one variable, and I rewrote the echo's rules until it refused what it should refuse. With a hosted model, someone else's policy sits between me and that prompt.

It cost nothing to run. I rewrote prompts dozens of times with no metered calls.

My Agent Session

Prize Categories

Gemma

SlothWatch: I Made a Cloud Browser and a Local Gemma Do My Boring Web Checks

Dibyendu Chatterjee — Sat, 24 Oct 2026 21:25:12 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

What I Built

I keep re-checking the same web pages. A portal status. A deadline. A price. An announcements page. It takes two minutes each time and I forget half the time anyway.

SlothWatch is a lazy web-chore watcher. You tell it once, in plain English, what you care about on a page:

slothwatch add https://example.com/page --care "tell me if the deadline changes or results are announced"

After that:

A cloud browser (Browserbase, driven by Stagehand) opens the page on a schedule and pulls the text.
If the page is identical to last time, SlothWatch stops. No model call at all.
If something changed, a local Gemma model running on my laptop through Ollama reads only the change and decides if it matters to me.
If it matters, my phone gets a two-line push through ntfy. If not, silence.

Silence is the feature. A sloth does not check things twice.

Demo

Code

dibyendu-coder / slothwatch

SlothWatch 🦥

Built during Hacktoberfest Open-Source AI Challenge Week 3 (Oct 19-25, 2026)
Note: This project was conceived, designed, and built entirely inside the challenge window.
Submission Deadline: October 25, 2026, 11:59 PM PDT (October 26, 12:29 PM IST).
Landing Page: slothwatch.tech (or docs/ on GitHub Pages)

SlothWatch is a lazy web-chore watcher. You describe in plain English what you care about on a web page. A cloud browser (Browserbase + Stagehand) visits the page on an automated schedule. Visible text is extracted deterministically and hashed. If the page is unchanged, SlothWatch exits immediately—consuming 0 LLM calls, 0 tokens, and 0 API dollars.

When real text changes occur, a local open-weight model (Gemma via Ollama) evaluates the compact diff to determine if it satisfies your intent. Your phone receives a push notification via ntfy only when it genuinely matters—filtering out timestamps, visitor counters, ads, and cookie…

View on GitHub

Install:

npm install -g github:[[your-username/your-repo]]
slothwatch init
slothwatch doctor

You need Node 20+, Ollama with a pulled model, a Browserbase API key, and an ntfy topic.

How I Built It

Stack: TypeScript, Stagehand and Browserbase for browser sessions, Ollama running [[Gemma tag, for example gemma4:e2b]] for judgment, SQLite for state, ntfy for push. Built on a laptop with an RTX 2050 and about 15 GB RAM. No closed model API anywhere in the path.

The design rule that made it work: split the job. My first instinct was to let the model drive the browser. Small local models are not reliable at that. So the browser does the deterministic work (navigate, wait, extract text) and the model only reasons over plain text it is handed. [[One or two sentences from NOTES.md on what actually failed when you tried the other way, if you tried it. If you did not, delete this sentence.]]

Laziness, enforced in code:

Hash check first. Page text is normalized and hashed. Same hash as last run means no model call. [[X of Y runs skipped the model entirely, from stats.]]
Recipe replay. The first run of a watch can include steps like dismissing a banner or clicking a tab. Those steps are recorded, and later runs replay them without asking a model to rediscover anything. [[First run took X s with Y model calls, replay took X s with 0, from stats.]]
Strict judge output. Gemma must return JSON with matters, a two-line summary, and a reason, validated with zod. Invalid output is retried once, then recorded as a failed judgment. It is never silently treated as "nothing changed".
Read-only. It never submits forms, buys anything, or posts for you.

Benchmark. I wrote [[N]] labelled test cases (real changes, noise like timestamps and cookie banners, and a few tricky ones) and ran them through [[model names]]:

Model	Accuracy	Score	False Positives (Noise)	Invalid JSON / Retried	Avg Latency	Avg Tokens
gemma4:e2b	95.5%	21/22	1	0 / 0	5.31s	696
gemma4:e4b	95.5%	21/22	1	0 / 0	6.35s	550
llama3.2:3b	90.9%	20/22	2	0 / 0	1.81s	527

Why Does Open Innovation Matter?

Private by default. Page content, including pages behind a login, is judged by a model on my own machine. It never goes to a closed API.
Costs nothing per check. After setup, each judgment is free. A closed API would turn "check this page every hour" into a bill that grows with every watch.
Swap the brain. The model is one config line. That is how the benchmark above was possible: same cases, different open models, real numbers.
Honest limits. Small open models were [[describe the real weakness from your benchmark]]. Splitting the job let me keep the model local anyway. A bigger closed model might have judged better, but it would have cost the privacy and the zero bill, which were the point.

My Agent Session

The Lazy Payoff

Prize Categories

Best Use of Browserbase: Stagehand sessions fetch and render pages, and recipes turn a one-off run into a repeatable workflow.
Best Use of Gemma: Gemma runs locally through Ollama as the judge and was benchmarked against other open models.

PayRadar: a late-payment radar for freelancers that never uploads an invoice

Dibyendu Chatterjee — Sun, 18 Oct 2026 21:27:17 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

Freelancers rarely go broke from a lack of work. They go broke in week 7, when three invoices that were "Net 30" are still unpaid and rent is due.

PayRadar is a local-first tool that tells a freelancer three things:

Which open invoices will probably be paid late, and by how many days.
Whether the next 8 weeks of expected cash-in cover their monthly costs, with gap weeks flagged.
What to write to each client. It drafts a reminder whose tone matches the risk: a friendly nudge, a clear and polite note, or a firm one. The freelancer reviews it and copies it. Nothing is ever sent automatically.

You drop in invoice PDFs or photos. A local vision model reads them, a tabular foundation model learns each client's payment habits from the freelancer's own history, and everything stays in a SQLite file on the laptop.

Who would pay for it? The target is a solo freelancer or a tiny agency billing a handful of clients, especially one whose contracts or NDAs make uploading client invoices to a cloud SaaS uncomfortable. I think $19 to $29 a month is plausible, but that is a hypothesis. I have not tested it with a paying customer, so I am not claiming demand. The demo on the synthetic data below shows the product works. It does not show anyone wants to buy it yet.

Demo

payradar-640a.onrender.com

The hosted copy runs in a lightweight demo mode with synthetic data and precomputed results. The free server has no GPU and no room for PyTorch, so it cannot run Gemma or TabPFN live. The real app runs entirely on your own machine. The first load can take about a minute because the free tier sleeps.

Code

dibyendu-coder / PayRadar

PayRadar 📡

Local-first late-payment radar for freelancers.
Drop in invoices, and it tells you which ones will be paid late, whether you will have a cash gap in the next 8 weeks, and drafts the right chaser email for each one.

Built Oct 12-18, 2026 during Hacktoberfest Week 2 (Side Hustle Challenge).

🏆 Hackathon Window Note

Event: Hacktoberfest Open-Source AI Challenge Week 2 (Side Hustle)
Challenge Window: October 12 – October 18, 2026
Window Statement: This repository was created and completely built within the challenge window (Oct 12-18, 2026).
Post-Deadline Commits None. (Any commit made after October 18, 2026 23:59 UTC will be documented here with justification).

Note

SYNTHETIC DATA NOTICE: All benchmark tables, model evaluation figures, demo documents, and client histories in this repository and application are 100% synthetically generated for demonstration and benchmarking purposes. No real personal, business, or financial customer data was…

View on GitHub

How I Built It

Open pieces doing the real work

Gemma 4 (gemma4:e2b) through Ollama reads invoice images and returns JSON constrained to a schema (client, amount, issue date, due date, terms). The same model drafts the chaser emails.
TabPFN-3.5, run locally with the tabpfn package, predicts two things per open invoice: the probability of being paid more than 7 days late, and the expected days late.
scikit-learn supplies the baselines I measured against.
Streamlit + SQLite + PyMuPDF for the app, the store, and PDF-to-image conversion.

The pipeline

PDF or image goes to an image of at most 1024px.
Gemma extracts the fields. The user reviews and edits them in a form, then confirms.
Features are built per invoice: log amount, payment terms, issue month and weekday, client, project type, and the client's history (how many earlier invoices, their mean lateness, and how this amount compares to their median).
TabPFN scores every open invoice. Scores are cached in SQLite so the UI never waits on the model.
Expected pay date is the due date plus predicted days late. I sum expected cash-in per week for 8 weeks and compare it to monthly costs.

The hardware constraint shaped the design. I built this on a laptop with an RTX 2050 (4 GB of VRAM) and 15.6 GB of RAM. So Gemma never runs inside a page request. Extraction and drafting are explicit button presses with a timeout and a template fallback, and results are saved. The default model is the small e2b, switchable with one environment variable.

Measured results (all on synthetic data)

The demo data is generated from a seed: about 400 invoices across 25 clients with deliberately different habits (always late, pays at month end, big invoices slower, one very reliable client). I split by time, not randomly, and client-history features only ever see invoices issued earlier. A test fails if a feature for invoice N can see invoice N or anything later.

Extraction on 20 rendered invoices with known answers, using gemma4:e2b: all four fields (client, amount, issue date, due date) correct on 20 of 20, at about 19 seconds per document. These are clean synthetic documents. Real scans will do worse.

Prediction, averaged over 3 expanding time windows (mean ± std):

Model ROC-AUC Brier Delay MAE (days)
Per-client mean 0.855 ± 0.035 0.128 ± 0.025 8.02 ± 0.32
HistGradientBoosting 0.863 ± 0.062 0.144 ± 0.058 8.66 ± 1.20
TabPFN-3.5-Fast 0.854 ± 0.055 0.122 ± 0.027 7.36 ± 0.97
TabPFN-3.5 0.844 ± 0.020 0.124 ± 0.025 7.28 ± 0.92

I want to be straight about what this says. TabPFN had the lowest delay error and a slightly better Brier score, and it needed no tuning. But the gaps are inside one standard deviation, and its ROC-AUC was the lowest of the four. On this data, where each client has a strong habit, a simple per-client average is a tough baseline. So the honest claim is "comparable accuracy with no feature tuning, and a modest edge on predicting how many days late", not "TabPFN wins". Full tables are in results/benchmark.md.

Why Does Open Innovation Matter?

The product's whole pitch is that invoices are confidential. A client list with amounts and due dates is exactly what an NDA tells you not to upload. A closed vision API would have meant sending every invoice to someone else's server. With an open-weight model on Ollama, the document never leaves the laptop, and the privacy claim is something I can show in the code (the only network call in the extraction path is to localhost).

Cost works the same way. There is no per-document price. At about 19 seconds per invoice on a 4 GB GPU laptop, a freelancer with a few dozen invoices a month pays nothing per page, so a low monthly price can still leave a margin.

Open also meant I could adapt it to the hardware I actually had: swapping e2b for e4b is one variable, and if the TabPFN weights are unavailable the app falls back to scikit-learn instead of failing.

A caveat I would rather say out loud: the TabPFN-3.5 weights are released for research and evaluation. Selling PayRadar as a commercial product would need a commercial license from Prior Labs. That is why the app has a fully permissive fallback path (gradient boosting and per-client averages) that works commercially. So the open story is strong for Gemma and for the architecture, and more conditional for TabPFN. A closed API would have been better at one thing: extraction from messy, low-quality photos. I have not tested that against real scans, so I do not claim a win there.

Prize Categories

Best Use of Gemma: Gemma 4 (e2b) runs locally through Ollama for vision extraction and email drafting.
Best Use of TabPFN: TabPFN-3.5, run locally, predicts late-payment risk and days late inside a cash-flow forecasting pipeline, benchmarked against baselines on time-split data.
Best Use of Render: the hosted demo runs on Render in a lightweight mode.

Fieldnotes: an offline Gemma field guide that turns a park walk into a quest

Dibyendu Chatterjee — Mon, 12 Oct 2026 06:49:39 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

Fieldnotes is an offline field-guide that turns a short walk into a
quest. You pick a park and how many minutes you have, and a local
open-weight model writes three tasks grounded in real OpenStreetMap
features and species recorded nearby on iNaturalist. You photograph
what you find, a local vision model checks each photo, and the app
writes a short field-journal entry. It's for anyone who wants a reason
to put the phone away and look around a park. [I built the area pack for Eco Park, Kolkata.]

Demo

Screenshots and a screen recording coming soon. The README has the full setup.

Code

dibyendu-coder / Fieldnotes

🌿 Fieldnotes — Touch Grass Challenge

An offline-first outdoor nature quest assistant and field journal powered by local open-weight AI (gemma4:e2b). Fieldnotes plans short walking quests tailored to your local park, verifies your photo observations using on-device vision, and writes a calm, reflective field journal — completely on-device with Wi-Fi switched off.

📸 Screenshots

(Replace with field walkthrough recordings or UI screenshots)

+-----------------------------------------------------------------------+
|  🌿 Fieldnotes                 [Offline: runs locally]  [DEMO DATA]    |
|  Quiet, observation-first nature guide powered by local AI.            |
+-----------------------------------------------------------------------+
|  1. PLAN YOUR WALK                                                    |
|     Area Pack: [ demo [DEMO DATA]          v ]                        |
|     Time Available: [ 20 Minutes (Standard Walk) v ]                  |
|     [ Generate Quest ]                                                |
+-----------------------------------------------------------------------+
|  ACTIVE QUEST: Coastal Observation Walk                               |
|  Intro: Let us take a brief survey of the immediate coastal           |
|         environment and the unique life it harbors.                   |
|                                                                       |

…

View on GitHub

How I Built It

Model: Gemma 4 (e2b) running locally through Ollama, for both quest writing and photo checking.
Data: a one-time "area pack" download from OpenStreetMap and iNaturalist (open data). After that, the app makes no network calls.
Sunset and daylight are computed locally.
Stack: Python/FastAPI backend, plain HTML page.
Benchmarks on my laptop (RTX 2050, Windows, Ollama): gemma4:e2b ran at about 35 tokens/s for text and 42 tokens/s for vision. gemma4:e4b was about 9 tokens/s and froze my laptop while loading, so I switched to e2b.
Offline test: the smoke test passed with Wi-Fi switched off.

Why Does Open Innovation Matter?

It works where trails have no signal, because the model runs on the laptop. A closed API can't follow you there.
My location and photos never leave the device. Coordinates are only sent to the public data services once, when I download the area pack.
The data is open (OpenStreetMap, iNaturalist), and the model can be swapped or fine-tuned for a region.
Tradeoff, honestly: the small model is sometimes generic or unsure about photos.

Prize Categories

Best Use of Gemma (Gemma 4 e2b running locally).

You See the Swags. You Don’t See the Rejections.

Dibyendu Chatterjee — Tue, 01 Sep 2026 12:34:28 +0000

If you look at my LinkedIn profile, you might think I have figured things out.

You’ll see opportunities.

Programs.

Hackathons.

Certificates.

Rewards.

Swags.

Achievements.

You’ll see posts where I’m excited because I got selected for something. You’ll see pictures of packages that arrived at my door. You’ll see another opportunity, another badge, another small win.

And honestly, I’m grateful for every single one of them.

But there’s a part of the story that rarely gets posted.

The part where I applied and didn’t get selected.

Again.

And again.

There was a time when my inbox felt like a collection of rejection emails.

“Unfortunately…”

“We regret to inform you…”

“Your application was not selected…”

“Due to the high number of applications…”

You get used to reading those words.

Or at least, you think you do.

The truth is, rejection still hurts.

Especially when you’ve spent hours filling out an application.

Writing answers.

Building something.

Recording a video.

Making a resume.

Trying to explain why you deserve a chance.

And then you wait.

Sometimes for days.

Sometimes for weeks.

Only to receive a polite email telling you that you weren’t selected.

Nobody sees that part.

Nobody sees the applications that went nowhere.

Nobody sees the nights spent working on something that never got noticed.

Nobody sees the projects that didn't win.

Nobody sees the hackathons where I walked away without a prize.

Nobody sees the opportunities I really wanted but didn't get.

People usually see the result.

They rarely see the attempts.

And I think that's one of the biggest illusions social media creates.

We compare our behind the scenes with someone else's highlight reel.

We see someone getting selected and think, “They are so lucky.”

We see someone winning a hackathon and think, “They are naturally talented.”

We see someone receiving a package full of goodies and think, “Everything is working out for them.”

But we don't see the hundreds of applications behind that one selection.

We don't see the dozens of failures behind that one success.

I have applied to opportunities knowing there was a very high chance I wouldn't get selected.

I still applied.

Because what if?

What if this is the one?

What if this application changes something?

What if I learn something while applying?

What if I meet someone?

What if I get selected?

Sometimes the answer is no.

But occasionally, the answer is yes.

And those few yeses make the previous no's feel a little different.

The swags that arrive at my home aren't just swags to me.

They are reminders.

Reminders of an application I almost didn't submit.

A project I almost gave up on.

A late night.

A failed attempt.

A rejection that came before the acceptance.

So when you see me posting about another opportunity or another reward, please don't assume that everything has been easy.

It hasn't.

I have had my fair share of self-doubt.

I've questioned whether I'm good enough.

I've wondered why someone else got selected while I didn't.

I've looked at other students moving faster than me and wondered if I'm falling behind.

But I'm still applying.

Still building.

Still learning.

Still trying.

And maybe that's the real achievement.

Not the certificate.

Not the swag.

Not the badge.

Not even the selection.

It's being able to open an inbox full of rejection emails and still send the next application.

Because behind every visible achievement, there is usually an invisible pile of attempts.

And mine is pretty big.

So yes, you can celebrate the wins with me.

But remember that they didn't happen overnight.

You are just seeing the part of the journey that finally worked.

You didn't see all the times it didn't.

And honestly?

I'm learning to be proud of those attempts too.

I Built a Voice-First Learning Companion in 10 Days

Dibyendu Chatterjee — Sat, 15 Aug 2026 09:53:52 +0000

Building an AI voice agent sounded simple at first.

Give it a voice. Connect it to an LLM. Let users talk to it.

But as I progressed through the 10 Days of Voice Agents – Voice for Bharat Edition, I realized that a useful voice agent needs much more than the ability to listen and speak.

It needs to understand the user.

It needs to remember context.

It needs to know when to use external tools.

It should know when to continue helping and when a human should take over.

And most importantly, it should be able to measure whether it is actually helping the user achieve something.

Over the last 10 days, I built a voice-first Learning & Literacy agent focused on helping learners practice spoken English in a more natural and supportive way.

This is the story of what I built, the problems I faced, and what I learned while building it.

Why I Chose the Learning & Literacy Track

Many learners understand English but hesitate when it comes to speaking.

Sometimes the problem is not a lack of knowledge.

It is the fear of making mistakes.

People may worry about pronunciation, grammar, or simply saying something wrong in front of others.

I wanted to build something that gives learners a space to practice without feeling judged.

That became the main idea behind my project.

A voice-based learning companion that can:

Talk with learners naturally
Help them practice spoken English
Give exercises
Evaluate responses
Provide supportive feedback
Remember previous interactions
Support Hindi-English code-mixed conversations
Reach out for practice sessions
Know when a human teacher should step in
Track whether learning sessions are successful
Hand off specialised requests to another agent when needed

Instead of building just another chatbot with a voice attached to it, I wanted to explore what happens when a voice agent starts behaving more like a complete learning system.

Why Voice?

For spoken-English practice, typing is not enough.

A learner needs to actually speak.

Voice makes the interaction more natural.

The learner can respond as they would in a normal conversation instead of typing carefully constructed sentences.

It also makes the experience more accessible.

A learner can simply speak:

"Mujhe English practice karni hai."

And continue the conversation naturally.

This was especially important for my project because I wanted to support Hindi-English code-mixed conversations.

In India, many people naturally switch between languages while speaking.

Forcing users to speak in only one language can make an interaction feel unnatural.

So one of my goals was to make the agent respond in a similar register when appropriate.
The Core Idea

The final project can be viewed as a learning journey.

Learner speaks
↓
Voice agent understands the request
↓
Learner context and memory are considered
↓
Personalized learning interaction
↓
Exercise is provided
↓
Learner responds
↓
Answer is evaluated
↓
Supportive feedback
↓
Learning outcome is recorded
↓
If needed → Human help
↓
If specialist knowledge is needed → Agent handoff

The goal was not to make every feature independent.

I wanted the different parts of the project to work together.
Building the Voice Agent

The first step was creating the basic voice interaction.

A typical voice agent involves several components working together:

User speaks
↓
Speech-to-Text
↓
LLM / Agent Logic
↓
Memory + Tools + Agent Decisions
↓
Text Response
↓
Text-to-Speech
↓
User hears the response

For the voice output, I used Murf Falcon, the fast text-to-speech API.

The voice layer was important because the project was built around real conversations rather than text-based interactions.

But getting an agent to speak is only the beginning.

The next challenge was deciding how it should behave.

Giving the Agent a Clear Personality and Guardrails

For a Learning & Literacy agent, how the agent responds is just as important as what it responds with.

A learner can make mistakes repeatedly.

That is part of learning.

So one of the important guardrails I implemented was:

The agent should never shame a learner for giving a wrong answer.

Instead of saying:

"That's wrong."

The agent should respond in a supportive way.

For example:

"Good attempt. Let's look at this together."

The goal was to make the interaction feel encouraging.

Another important rule was that the agent should not make inappropriate assumptions about learners.

It should focus on helping them practice and improve without making claims beyond its role.

These instructions became the foundation for the rest of the project.

Adding Memory and Personalization

One of the biggest improvements came when I added persistent memory.

A normal conversation can feel disconnected when the agent forgets everything after the session ends.

If a learner practiced yesterday and comes back today, the experience should not always start from zero.

I integrated a database to store useful learner context.

This allowed the agent to remember information such as:

Learner identity
Preferences
Relevant learning context
Previous interactions

The frontend was also adapted to reflect learner information and session state.

This changed the experience from:

"Hello, how can I help you?"

every single time,

to something more contextual and personalized.

Memory made the agent feel less like a one-time interaction and more like a continuing learning companion.

Teaching the Agent to Use Tools

An AI agent cannot rely only on what the language model already knows.

Sometimes it needs to fetch information or perform a specific action.

For my Learning & Literacy track, I added tools that support the learning flow.

The agent can:

Fetch or select the next exercise
Evaluate a learner's answer
Use the result to provide feedback

One lesson I learned here was that creating the function itself is not enough.

The model needs to understand when it should use that function.

A tool can work perfectly in the backend and still fail to be useful if the description is unclear.

The tool description became an important part of the system.

The agent needed to know:

When to fetch an exercise
When to evaluate an answer
When not to call a tool unnecessarily

This was one of the first moments where I understood that building AI agents is not only about writing functions.

It is also about designing the decisions around those functions.

Adding Outbound Practice Calls

Most voice agents wait for the user to start the interaction.

For Day 6, I explored the opposite approach.

What if the agent could proactively reach out to the learner for practice?

For a learning agent, the use case was simple:

A learner chooses a practice time, and the agent can initiate a practice session.

This introduced a new challenge.

Outbound conversations need to begin differently from inbound conversations.

The user did not initiate the interaction.

So the agent needs to clearly communicate:

Who is calling
Why it is calling
How the user can stop future calls

While building this feature, I also faced practical challenges with telephony services, trial limitations, and account requirements.

I initially explored different options before moving toward a SIP-based setup using Linphone for testing.

This was one of the more challenging parts of the project because the idea was simple, but getting the communication setup working reliably required experimenting with different approaches.

It reminded me that real-world AI products are not only about AI.

Integration and infrastructure can sometimes take more time than the model logic itself.

Teaching the Agent When to Ask for Human Help

This was one of my favourite parts of the project.

An AI agent should not try to solve every situation by itself.

For my Learning & Literacy agent, I defined two situations where human support may be needed:

The learner is genuinely frustrated or upset.
The learner explicitly asks to speak with a teacher or human.

However, I also needed to make sure that the agent did not escalate unnecessarily.

A wrong answer is not a reason to involve a human.

Making mistakes is part of learning.

So the system needed to distinguish between:

"I got this answer wrong."

and

"I'm really frustrated. I don't understand this anymore. Can I talk to a teacher?"

When human help is needed, the agent does not immediately share information.

It first asks for permission.

The flow looks like this:

Learner needs help
↓
Agent detects the situation
↓
Agent explains what information can be shared
↓
Learner gives permission
↓
Human-help request is created
↓
Reference ID is generated
↓
Teacher or human can review the request

The request contains only useful information, such as:

Who needs help
What the learner is struggling with
What the agent has already tried
Urgency
Preferred language or follow-up method

The agent does not need to send the entire conversation.

This feature made the project feel more realistic.

Sometimes the best thing an AI system can do is recognize that another person is better suited to help.

Measuring Whether Learning Actually Happened

Building an agent is one thing.

Knowing whether it is useful is another.

For the Call Analytics Dashboard, I had to answer a simple question:

What does a successful call mean for my agent?

I decided that a successful learning session means:

The learner completes at least one exercise and receives feedback.

A failed session does not necessarily mean the system crashed.

It simply means the learning objective was not completed.

For example:

The learner ends the session early.
The learner does not complete an exercise.
The conversation stops before feedback is given.

Every session is recorded with an outcome.

The dashboard tracks:

Total Calls
Successful Calls
Failed Calls

The important part was that these numbers come from actual agent sessions.

They are not hardcoded values.

This changed the way I thought about AI agents.

It is easy to measure activity.

For example:

How many conversations happened?

But activity does not automatically mean success.

For a learning agent, a better question is:

Did the learner actually complete something useful?

That became the purpose of the analytics system.

Moving from One Agent to Multiple Agents

By Day 9, the main agent could already handle several tasks.

But one agent should not try to become an expert at everything.

For the Learning & Literacy track, I created a separate:

Maths Practice Specialist

The main Learning & Literacy agent continues to handle things such as:

Spoken English practice
English exercises
Grammar support
General learning conversations

But when the learner asks for maths help, the conversation is handed off to the Maths Practice Specialist.

For example:

Learner:

"I don't understand percentages."

The main agent responds:

"I'll connect you to our Maths Practice Specialist."

The specialist then continues the conversation.

The important part is that the learner does not have to repeat the entire problem.

Relevant context is passed during the handoff.

The flow looks like this:

Learner asks for help
↓
Main Learning Agent
↓
Does this require Maths expertise?
↓
YES
↓
Agent announces handoff
↓
Maths Practice Specialist
↓
Continues with existing context

The specialist can:

Explain concepts step by step
Give practice questions
Provide hints
Evaluate answers
Give supportive feedback

The same language style can also continue after the handoff.

For example:

"Mujhe fractions samajh nahi aa raha."

The specialist can continue naturally in a Hindi-English code-mixed style.

This was my first deeper experience with a multi-agent workflow.

The biggest lesson was simple:

Instead of making one agent responsible for everything, it can be better to give different agents clear and focused responsibilities.

The Overall Architecture

The final system can be represented like this:

                    ┌───────────────────┐
                    │      Learner      │
                    │   Voice / Browser │
                    └─────────┬─────────┘
                              │
                              ▼
                    ┌───────────────────┐
                    │ Speech-to-Text    │
                    └─────────┬─────────┘
                              │
                              ▼
             ┌─────────────────────────────┐
             │ Main Learning & Literacy    │
             │ Agent                       │
             │                             │
             │ • English Practice          │
             │ • Guardrails                │
             │ • Memory                    │
             │ • Learning Flow             │
             └───────┬──────────┬──────────┘
                     │          │
                     │          │
              ┌──────▼───┐  ┌──▼────────────────┐
              │ Learning │  │ Maths Practice    │
              │ Tools    │  │ Specialist Agent  │
              └──────┬───┘  └───────────────────┘
                     │
                     ▼
             ┌─────────────────┐
             │ Memory / SQLite │
             └────────┬────────┘
                      │
         ┌────────────┼─────────────┐
         │            │             │
         ▼            ▼             ▼
  Human Help     Call Analytics   Outbound
   Escalation      Dashboard       Calling

                      │
                      ▼
                ┌─────────────┐
                │ Murf Falcon │
                │     TTS     │
                └─────────────┘

The Difficult Parts I Faced

Not everything worked perfectly on the first attempt.

And honestly, that was probably one of the biggest learning experiences from this challenge.

Telephony Setup

I initially explored different telephony services for outbound calling.

Some required upgrades before allowing access to certain features.

Others had trial limitations or different onboarding requirements.

I had to experiment with different approaches instead of assuming that the first service would work perfectly.

Eventually, I explored a SIP-based setup using Linphone for the outbound calling workflow.

The lesson was:

Always have an alternative approach when a third-party integration becomes a blocker.

Tool Calling

Creating a function was relatively straightforward.

Making sure the agent calls it at the correct time was harder.

If the tool description is too vague, the agent might:

Call it unnecessarily
Never call it
Call it at the wrong point in the conversation

I learned that tool descriptions are part of the agent's reasoning interface.

The description needs to clearly explain:

What the tool does
When it should be used
When it should not be used

Human Escalation

The difficult part was defining when the agent should stop.

A learner making mistakes does not mean the system failed.

The agent should continue encouraging and teaching.

But genuine frustration or an explicit request for a teacher is different.

Finding this boundary helped me understand the importance of defining clear conditions for AI systems.

Defining Success

At first, it would have been easy to mark every completed conversation as successful.

But that would not tell me much.

A learner could talk for five minutes and still not complete a single exercise.

So I defined success around the actual learning outcome.

The learner needs to:

Complete at least one exercise.
Have the answer evaluated.
Receive feedback.

Only then is the session marked as successful.

This made the analytics more meaningful.

What I Learned About Building Voice Agents

After working on this project for 10 days, I learned that a voice agent is much more than:

Speech in → AI → Speech out.

A useful agent needs a complete system around it.

Here are some of my biggest takeaways.

Voice is only one layer

The voice may be the first thing the user notices.

But memory, tools, guardrails, decisions, and context determine whether the interaction is actually useful.

Memory changes the experience

Remembering previous interactions makes the agent feel more continuous.

The conversation no longer needs to restart from zero every time.

Tools need good descriptions

A working function is not enough.

The agent needs clear instructions about when and why it should use it.

AI should know its limits

Human escalation was an important reminder that an AI agent does not need to solve everything.

Knowing when to ask for help can be just as important as answering correctly.

Measure outcomes, not just conversations

Counting calls is easy.

Understanding whether users achieved something meaningful is more valuable.

One agent does not need to do everything

Specialist agents can have smaller, clearer responsibilities.

The main agent can focus on coordinating the experience and hand off specific tasks when needed.
How to Build and Run This Project
GitHub Repository

Repository: https://github.com/dibyendu-coder/murf-livekit-starter

Prerequisites

1.Python (v3.9 or higher)
2.Node.js (v18 or higher) and npm
3.LiveKit Cloud / Server credentials or local LiveKit Server instance
4.API Keys:
LiveKit API Key & Secret
Murf AI API Key
LLM Provider API Key (e.g., OpenAI / Gemini API key used in your agent backend)

Installation

Obtain the Project Files
Clone the repository or download the project files into a local folder.
Configure Environment Variables (.env.local)
Create a .env.local file inside the murf-livekit-starter root (or backend/frontend folders as required by your setup) containing:

env

LiveKit Configuration

LIVEKIT_URL=wss://your-livekit-domain.livekit.cloud
LIVEKIT_API_KEY=your_livekit_api_key
LIVEKIT_API_SECRET=your_livekit_api_secret

Next.js Public URL

NEXT_PUBLIC_LIVEKIT_URL=wss://your-livekit-domain.livekit.cloud

Murf & AI Keys

MURF_API_KEY=your_murf_api_key
OPENAI_API_KEY=your_openai_api_key # (or relevant LLM key)

Backend Setup Open a terminal and navigate to the backend folder:

bash
cd murf-livekit-starter/backend
Create and activate a Virtual Environment:

Windows (PowerShell):
powershell
python -m venv .venv
..venv\Scripts\Activate.ps1
macOS / Linux:
bash
python3 -m venv .venv
source .venv/bin/activate
Install Dependencies:

bash
pip install -r requirements.txt
Start the Agent Backend Worker:

bash
python agent.py dev

Frontend Setup Open a second terminal window and navigate to the frontend folder:

bash
cd murf-livekit-starter/frontend
Install Node Dependencies:

bash
npm install
Start the Next.js Development Server:

bash
npm run dev

Launch the Application Open your browser and go to http://localhost:3000. Click Connect / Start Call to initiate a voice session with your LiveKit + Murf AI voice agent.

Environment Variables

LiveKit Configuration

LIVEKIT_URL=wss://your-livekit-domain.livekit.cloud
LIVEKIT_API_KEY=your_livekit_api_key
LIVEKIT_API_SECRET=your_livekit_api_secret

Next.js Public URL

NEXT_PUBLIC_LIVEKIT_URL=wss://your-livekit-domain.livekit.cloud

Murf & AI Keys

MURF_API_KEY=your_murf_api_key
OPENAI_API_KEY=your_openai_api_key or GEMINI_API_KEY=your_gemini_api_key
Also make sure .env is included in .gitignore.
How to Run the Project
Activate the livekit server from the root folder :
.\livekit-server.exe --dev
Activate the backend : uv run python src/agent.py dev
Activate the frontend : pnpm dev

What I Built in 10 Days

Looking back, the project started as a simple idea:

Build a voice agent for learning.

By the end of the challenge, the project had grown into a system that could:

Communicate through voice
Follow learning and safety guardrails
Support Hindi-English code-mixed conversations
Remember returning learners
Personalize interactions
Fetch and provide learning exercises
Evaluate answers
Give supportive feedback
Make outbound practice calls
Ask for permission before escalating to a human
Create structured human-help requests
Track successful and failed learning sessions
Display analytics from real session data
Hand conversations to a specialist agent

The project changed significantly from the original idea.

And that is probably the biggest thing I enjoyed about this challenge.

Every day added a new question.

What should the agent remember?

When should it use a tool?

How should it contact the user?

When should it stop?

How do we know whether it is actually helping?

When should another agent take over?

Answering those questions slowly turned a simple voice assistant into something that feels much closer to a real product.

Final Thoughts

Ten days ago, I started with the goal of building a voice agent.

I thought the biggest challenge would be getting the AI to listen and respond.

It was not.

The bigger challenge was deciding how the agent should behave once the conversation starts.

What should it remember?

What should it do with that information?

When should it use a tool?

When should it continue helping?

When should it ask for human support?

And how do we know if the interaction was actually successful?

By the end of this journey, I realized that building an AI agent is not just about making it more capable.

It is also about giving it boundaries.

Knowing what it should do.

Knowing what it should not do.

And knowing when it should step aside.

This project is still a work in progress, but the last 10 days gave me a much better understanding of how voice agents can move beyond simple conversations.

I started by building a voice agent.

I ended up building the foundation of a voice-first learning companion that can remember learners, personalize practice, use tools, proactively initiate conversations, involve humans when needed, measure outcomes, and work with specialist agents.

And this is only the beginning.

Built as part of the 10 Days of Voice Agents – Voice for Bharat Edition using Murf Falcon.

VoiceForBharat #MurfAI #MurfFalcon #AIVoiceAgents #VoiceAI #LearningAndLiteracy #AIAgents #MultiAgent #ConversationalAI #BuildInPublic #10DaysofAIVoiceAgents

The Silent Side of Hackathons: When Your Name Is Never Called

Dibyendu Chatterjee — Wed, 03 Jun 2026 17:26:17 +0000

If you look at my social media profile, you might think my hackathon journey has been exciting. There are photos from events, team pictures, project demos, certificates, and countless posts about late-night coding sessions.

What you won't see are the moments after the winner announcements.

The moments when everyone gathers around the stage, waiting for the results.

The moments when I sit quietly in the audience, listening to names that are never mine.

I have participated in many hackathons. I have spent sleepless nights building projects, fixing bugs at 3 AM, creating presentations minutes before submission, and learning technologies that I had never touched before. Every time I join a new hackathon, I tell myself that maybe this time things will be different.

Maybe this time my project will be good enough.

Maybe this time my team will make it to the top.

Maybe this time I will finally hear my name being called.

But often, the results are the same.

Someone else wins.

I clap for them.

I smile.

And then I quietly leave the venue.

At first, it was easy to accept. I was a beginner, and there was always something new to learn. But after participating in many hackathons, the feeling became heavier. Watching other teams win again and again started making me question myself.

Am I not working hard enough?

Am I not skilled enough?

What am I doing wrong?

These questions are difficult because there is rarely a clear answer.

The truth is that hackathons are strange competitions. Sometimes the best technical solution does not win. Sometimes the team with the best presentation stands out. Sometimes the judges are looking for something different. Sometimes another team simply builds something extraordinary.

And sometimes, despite doing everything right, you still don't win.

That is probably the hardest lesson I have learned.

I used to think that effort always guarantees results. Hackathons taught me that effort guarantees growth, not necessarily trophies.

Every project I built taught me something valuable. I learned how to work under pressure. I learned how to communicate ideas. I learned how to collaborate with teammates. I learned how to present in front of judges. Most importantly, I learned how to keep going after disappointment.

People often celebrate the winners, and they deserve that recognition. But there are hundreds of participants who return home without medals, prize money, or certificates of achievement. Their stories are rarely told.

I am one of them.

I know what it feels like to refresh the results page again and again.

I know what it feels like to believe your project has a chance.

I know what it feels like to hear the final winner announcement and realize your journey ends there.

It hurts.

Not because someone else won.

But because of the dreams you secretly attached to that project.

The dreams of proving yourself.

The dreams of finally succeeding.

The dreams of showing everyone that your hard work was worth it.

Yet despite all of this, I keep coming back.

Not because I enjoy losing.

Not because failure feels good.

But because every hackathon gives me another opportunity to improve.

Every loss teaches me something that a victory never could.

Every rejected project reveals a weakness I can work on.

Every unsuccessful attempt makes me more experienced than I was before.

One day, maybe my name will finally be called on that stage.

Maybe I will stand where the winners stand today.

Maybe all these failures will become part of a success story.

But even if that day takes longer than expected, I refuse to stop trying.

Because hackathons were never just about winning.

They were about becoming the kind of person who keeps building even when nobody is watching.

And until the day my name is announced, I will continue doing exactly that.

Building.

Learning.

Failing.

Improving.

And showing up again.

Because sometimes the strongest participants in a hackathon are not the ones holding the trophies.

They are the ones sitting quietly in the audience, carrying their disappointment home, and still deciding to come back for the next challenge.

Building BackBench — An Emotionally Adaptive AI Learning Platform

Dibyendu Chatterjee — Tue, 19 May 2026 21:22:52 +0000

During this hackathon I kind of wanted to build something that felt different from the usual AI chatbots, quiz makers, and productivity tools. I mean not just another screen that delivers content or spits out answers right away. Instead of focusing on content only, I wanted to tackle a more real and human issue in education—silent student struggle, you know.

A lot of students don’t exactly “stop learning” they just… hesitate. They mess up again and again , lose confidence, avoid asking doubts where everyone can see, and slowly drift out of the whole learning vibe. Most platforms never catch that emotional part. They can track marks, timelines, “progress”, but they don’t really see when a student is quietly giving up.

That’s how BackBench happened. It’s an emotionally adaptive AI-powered learning platform made to support students in real time when they struggle silently , without turning it into a big performance.

The main idea behind BackBench is honestly pretty plain:

“Learning should adapt to the student, not the other way around.”

So the platform looks for patterns like repeated wrong answers, hesitation, inactivity, quick guessing, and focus drops. Once those signals show up, BackBench adjusts the learning experience automatically with things like Rescue Mode, adaptive AI explanations, emotional learning analytics, and anonymous doubt rooms.

One of the coolest pieces I built was Rescue Mode. When the system notices that a student is struggling repeatedly, the interface turns into a more calm, more supportive space. The AI simplifies the concepts, it changes teaching styles, and it gives guided explanations in a way that feels less overwhelming. The point isn’t to “correct” them harder, or make them feel judged. It’s more like helping them get confidence back, step by step, while learning.

Another feature I really enjoyed was the Anonymous Doubt Rooms. A ton of students avoid questions publicly because they worry about embarrassment or being judged. These rooms let students collaborate anonymously in real time, so they get a safe emotional space, while still getting AI-assisted help when they need it.

I built the whole project using MeDo.dev. One thing I learned here is that modern development isn’t just about writing code by hand. It’s also product thinking, user experience design, and figuring out the actual problem you’re solving. I used MeDo not only as a code generation helper, but kind of as a creative partner to sharpen the product direction, the interaction flows, the adaptive systems, and the emotional UX of the platform , step by step.

This hackathon taught me that the best projects aren’t always the ones with the most technical complexity. Sometimes the strongest ideas come from understanding human problems deeply, and then building experiences that actually feel meaningful.

BackBench isn’t just another AI tutor. It’s an attempt to make education more emotionally aware, more supportive, and more human-centered.

And yeah, this is only the beginning.