DEV Community: Rahul Pandey

i made a 501B model think less and it scored the same

Rahul Pandey — Thu, 22 Oct 2026 08:42:14 +0000

i spent this week trying to understand where reasoning actually happens in a model.

it started with a post on my timeline. reflection and hugging face were giving $20k to people building with Beam, reflection's new 501B open model. i had never gone this deep into how a model thinks, so i used it as an excuse to learn.

demo: https://huggingface.co/spaces/PengoSword123/beamdial
data: https://huggingface.co/datasets/PengoSword123/beamdial-v0.1
code: https://github.com/CoderRahul01/beamdial

the question

Beam has a setting called reasoning_effort. 5 levels, low to max. higher means the model thinks longer before it answers, and those thinking tokens are billed like any other output.

most developers set it once and forget it. usually on max.

a smart model should think hard on hard problems and answer fast on easy ones. you shouldn't have to guess.

what i measured

-> 410 problems with known answers. school math, competition math, olympiad problems, python coding
-> every problem at all 5 levels. 2,050 runs
-> math checked with hugging face's math-verify, code checked by running the real tests in a sandbox

what i found

on standard math and code every level landed inside the same error bars.

low 96.3%. max 96.6%. low used 55% of the tokens.

the dial does work. on competition math max thinks 2.2x longer than low. it just doesn't buy more right answers on these tasks.

one interesting hint: when low thinks way longer than usual, the problem is usually hard. on the 23 tasks where that happened, high got every one right. but when i picked the threshold properly on held out data, "re-ask at high" only tied "always use high". so it's off by default until i can test it on harder problems.

the bug i found in my own grader

halfway through, accuracy looked like 93.1%. something felt off, so i read the "wrong" answers. most were right. 1/4 written as \frac14, 15 pi with a space.

the math checker can't run inside worker threads, and my grader was quietly counting that as a wrong answer. after the fix: 96.7%.

rule: read your failures before you trust your numbers.

what i built

BeamDial wraps the openai client you already use with Beam.

-> picks the level per prompt with a tiny router (pure python, under 1 MB, no gpu)
-> moves up a level only when your own check fails
-> retries cut-off answers at the same level with more room instead of thinking even longer
-> always sends back the reasoning_content Beam needs on follow-up turns, which otherwise breaks the whole chat

and beamdial calibrate. point it at your own prompts and it tells you if more thinking actually helps for you. my benchmark is a starting point, not the final word.

where this stops

these are standard benchmarks where Beam already scores about 97%, and it has likely seen many of them in training. harder or newer problems might need more thinking. next i'm testing aime 2026, hmmt 2026, apex and freshly generated long-context tasks.

the whole study cost about $4 of credits.

do you set reasoning effort yourself, or leave it on max?

i post the build as it happens on X: x.com/rahulpandey187

i stopped journaling. now i just say what i saw outside, and my phone keeps the almanac

Rahul Pandey — Sun, 11 Oct 2026 13:36:46 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

My name is Rahul Pandey. Every evening, I walk the exact same road, along the exact same path, back to the same place.

On paper, it’s a routine commute. But if you actually look up, it is never the same walk twice.

One evening, the parijat tree by the gate has dropped a carpet of orange and white blossoms on the asphalt. Another evening, the dusk air carries that first crisp chill of October. Next week, a blue kingfisher is perched on the lake fence.

Those small, fleeting things are what ground me. They become my quiet motivation for the next morning.

Yet when I tried journaling to capture them, it died in a week. Writing after a long day feels like homework. And three months later, I can never remember when any of those moments actually happened.

The Problem in the Backcountry

Then think about hikers, backpackers, and adventure travelers out in the wild.

When you're three hours into a mountain trail, you have zero cellular bars. More importantly, your battery is life or death. You cannot afford to keep a bright display on for 15 minutes typing paragraphs into a diary app, or watching a loading spinner drain your phone while searching for signal.

When you crest a ridge or notice a rare flower, you want to record it in 3 seconds without breaking stride or burning your battery.

That’s Why I Built Almanac

Almanac is a personal, voice-first farmer’s almanac designed to make the screen the shortest part of the outdoor experience.

You download the app and the lightweight open model once. (Gemma 3 1B LiteRT on Android, or Whisper in the browser).
On the trail or your evening walk, tap once, speak what you saw, and tap save.
The phone goes straight back into your pocket. The screen is on for under 5 seconds.
Zero signal needed. Your battery is saved, and your words are securely stored on-device.
Next year, it nudges you back outside: "The Parijat is due in about a week" — giving you one genuine reason to step outside today.

For global travelers exploring unfamiliar ridges, or for anyone walking the exact same evening road home: Almanac picks out the things that truly mean something. Because the small things should matter to you a lot.

Demo

🌐 Live Web App (PWA, Airplane Mode Capable): https://almanac-outdoors.vercel.app
🌿 Interactive 1-Year Demo Almanac: https://almanac-outdoors.vercel.app/app?demo=1
📊 Pulse Live Model Quality & Metrics: https://memoria-pulse.vercel.app
⚡ Cloudflare Edge API: https://almanac-edge.rahulpandey-creates.workers.dev
🖥️ Render TabPFN Service: https://almanac-api-5ak6.onrender.com

Code

💻 Source Code: https://github.com/CoderRahul01/almanac (Full monorepo by me containing Web React/Vite PWA, Cloudflare Edge Worker API, Render TabPFN service, native Android Kotlin/Jetpack Compose app, and Swift iOS core).

How I Built It

1. Minimal Screen Time & Zero Battery Drain

The core design constraint was outdoor survival:

Screen on for <5 seconds: No complex forms, category pickers, or tagging dropdowns while walking.
Immediate Local Storage: Notes are written instantly to on-device storage (IndexedDB on web, Room on Android). No network polling, no battery drain.
Offline Transcription: Whisper large-v3-turbo runs on Cloudflare Workers AI when connected; an in-browser WebAssembly Whisper model or native Android LiteRT model runs when in airplane mode.

2. Say It However You Want (Fuzzy Vernacular Mapping)

Nobody recites formal Latin botanical names like Plumeria rubra when hiking uphill or catching their breath. They say "the champa tree" or "neem ka ped mein phool aa gaye", or speech recognition mishears outdoor terms over wind noise.

Almanac passes your personal history of observed things to open-weight Gemma 4. The model maps phonetic near-misses and conversational descriptions into consistent personal entities:

"parajat near the gate is in full bloom" → Parijat · peak
"neem ka ped mein phool aa gaye" → Neem · flowering
"king fisher again at the lake railing" → Kingfisher · seen

A real slip from testing: Whisper transcribed "has started flowering" as "has started flooring". Gemma still filed it correctly under Parijat, flowering, and conversational Ask answered: "The Parijat by the gate started flowering on Oct 09."

3. I Measured the Models Daily (40 Real Cases)

I didn't want to rely on vibes. We built an automated test suite (eval-cases.js) with 40 labelled outdoor voice recordings (clean notes, rambling Hinglish, misheard plant names, and zero-nature control notes) plus 12 questions:

Engine	Finds the Right Things (F1)	Invents Nothing (Precision)	Unanswered (Rate Limit)	Typical Latency
Gemma 4 on Cloudflare Workers AI	97.6%	100%	0%	1.9 s
Gemma 4 on Google AI Studio (Free)	78.3%	100%	40% (Rate limits)	2.9 s

Both engines scored 100% on Ask Q&A, and both honestly answered "Not in your almanac yet" when asked about things never observed. The data picked our production routing: Workers AI first, Google AI Studio fallback, OpenRouter tertiary.

4. TabPFN-3.5 for Seasonal Predictions

On Render, we run TabPFN-3.5 to learn:

Your best hour of the day to step outside (fitting weather forecasts against past walk enjoyment ratings).
Predicted bloom and arrival windows from sparse historical data points.

5. Zero-Knowledge End-to-End Encryption

Your outdoor journal belongs to you:

No account required to start recording.
Optional backup sync is end-to-end encrypted with AES-256-GCM on your handset, sealed with a 24-word recovery phrase.
The server stores encrypted blobs it cannot decrypt.

Why Does Open Innovation Matter?

Backcountry Independence: Closed APIs require continuous cellular coverage. When a hiker is on a remote mountain trail in airplane mode, closed AI is useless. Open-weight models (Whisper + Gemma) run locally on the silicon in your hand.
Battery Preservation: Cellular radios searching for distant cell towers drain batteries rapidly. Running local open models lets outdoor adventurers keep airplane mode on all day without losing their smart journal.
Intimate Privacy & Sovereignty: Notes about where you walk, what you notice, and how you feel should never be used to train third-party corporate models. Open weights allow local inference and zero server access to your plaintext life.
Free for Walkers Everywhere: Nature lovers around the world shouldn't have to pay a $20/month subscription just to notice the flowers along their evening walk. Open models make this free forever.

Take It Outside: How It Went

Testing Almanac during my evening walks this week completely changed how I experience the path:

Oct 8 (06:40 AM): Walking past the gate: "The Parijat blossoms are carpeting the pavement this morning." 4 seconds later, phone back in pocket. Understood as Parijat · peak bloom.
Oct 9 (07:15 AM): Tested offline in airplane mode by the lake. Tapped record: "Saw a blue kingfisher dive for fish by the willow." Captured instantly into local storage. When reconnected, Gemma filed it under Kingfisher · seen.
The Morning Nudge: Yesterday, Almanac sent a sealed push: "The air is clear and the Parijat is near the end of its peak bloom. Best time to step out today is 07:00 AM." It got me out the door before opening my laptop.

My Agent Session

We saved our agent design and benchmarking session using DevRelay:

Building Almanac: Offline-First Nature Companion with Gemma 4, Whisper & TabPFN

You

We need to build Almanac for the Hacktoberfest 'Touch Grass' challenge. The core constraint: it must get people outside, work in airplane mode on the trail with zero signal, and use open-source AI at its core. How do we architect the model pipeline?

Agent

To make the screen the shortest part of the walk and survive zero connectivity, we architect a 3-tier open model pipeline:
1. On-Device / Browser: Whisper (Wasm/WebGPU) for offline voice transcription + client-side rule extraction, saving to IndexedDB/Room immediately.
2. Edge: Cloudflare Workers AI running open-weight Gemma 4 (26B/31B) for high-accuracy nature extraction and conversational almanac queries.
3. Fallback: Google AI Studio Gemma 4 + OpenRouter if edge rate limits hit.
Let's benchmark the engines with 40 real-world outdoor audio transcripts (clean, rambling, Hinglish, misheard plant names).

▸ run_model_eval evalrun.js studio,cf

Input

evalrun.js studio,cf

Output

Workers AI Gemma 4: 97.6% entity accuracy, 0% hallucination on non-nature notes, 1.9s latency. AI Studio: 78.3% (due to 40% rate limits). Recommendation: Workers AI first, AI Studio fallback.

You

How do we handle TabPFN and predictive outdoor timing without exposing private walks?

Agent

We isolate TabPFN-3.5 on a dedicated Render worker service. It trains on localized seasonal observations and anonymized hour timestamps to predict:
1. Best hour to step outside based on temperature and past walk enjoyment.
2. Expected bloom windows for flora observed in your area.
All user notes remain end-to-end encrypted with client-side AES-256-GCM; the server stores zero plaintext notes.

▸ call_tabpfn_api curl -X POST https://almanac-api-5ak6.onrender.com/predict/window

Input

curl -X POST https://almanac-api-5ak6.onrender.com/predict/window

Output

{
  "prediction": "Parijat flowering window estimated Oct 09 - Oct 16",
  "confidence": 0.92
}

Direct link: https://dev.to/agent_sessions/building-almanac-offline-first-nature-companion-with-gemma-4-whisper-tabpfn-irkm7p

Prize Categories

🏆 Best Use of Gemma: Gemma 4 (26B/31B) powers entity extraction, fuzzy vernacular mapping, and grounded almanac Q&A; on-device Gemma 3 1B handles offline Android inference.
🏆 Best Use of TabPFN: TabPFN-3.5 on Render models few-shot seasonal bloom windows and personal optimal outdoor hours.
🏆 Best Use of Render: Hosts our TabPFN inference service, background push dispatch, and daily model benchmark jobs.
🏆 Best Use of ElevenLabs: Generates audio narration for daily morning outdoor nudges.

Built with ❤️ by **Rahul Pandey* (@talkyrahulrp) to help people touch grass -Hacktoberfest 2026*

Follow along on X: x.com/rahulpandey187

i built a place to keep the stories only one person in your family knows

Rahul Pandey — Wed, 07 Oct 2026 06:50:53 +0000

tape" width="800" height="420">every family has one person who knows all the stories.

the recipe nobody wrote down. the night train they took with forty rupees in their pocket. the advice they repeat every sunday until you stop listening.

and one day nobody can ask them anymore.

i started Memoria for the Hacktoberfest weekend challenge ("build for a friend"). i'll be honest, i missed the deadline. but the thing works and i want people to use it, so i'm shipping it anyway.

live: https://memoria-family.vercel.app

code: https://github.com/CoderRahul01/memoria

what it does

-> you sit with them and press record. they talk, in any language

-> the words get written down, given a title and a year, and filed in that person's album

-> the album has a little timeline of their life, placed by year

-> later, anyone in the family can ask "how did nani make the dal?"

-> the answer comes back with her exact words underneath, and which story they came from

if she never talked about it, Memoria says "she hasn't talked about this yet" and gives you a button to record that story next.

that one rule mattered more than any feature. an app that makes things up about someone you love is worse than no app.

the recorder is a cassette

the first version had a generic mic button. it felt like a voice memo app.

now it's a cassette tape. the reels turn while you record, and the tape slowly winds from the left reel to the right as time passes. it sounds small, but older people get it instantly. they've done this before.

how it's built (open models at the core)

-> Gemma 4 (31B) through Backboard writes the titles, answers questions and keeps each person's memory. Backboard also lets me fall back to the faster Gemma 4 26B when the big one is slow

-> Tinker serves a Qwen 3.6 checkpoint as the backup model

-> ElevenLabs does transcription in 90+ languages, read-aloud, and (with consent) their own voice

-> Render runs the server, Vercel serves the site, Neon holds the data

-> Dodo Payments for a one-time $19 "Lifetime"

why open matters here: these are someone's family stories. i want to be able to swap the model, run it somewhere else, or fine-tune it on how a family actually talks, without asking a closed API for permission. the app doesn't care which open model answers, as long as it only answers from what was said.

the boring stuff that makes it launchable

this is the part nobody posts about.

-> no sign-up. each family gets a private space and a link. share it with your siblings and they can add stories too

-> it can't run my credits dry. every paid call is logged against a monthly budget. the last 10% is kept for paying families. free families fall back to the device voice, the browser transcript or the backup model. the app keeps working, it just gets a bit simpler

-> paying families fund the expensive parts. the natural voice and voice cloning are Lifetime features, so the $19 pays for the credits it uses

-> analytics live outside the app. i built Pulse, a separate dashboard that reads read-only views from the database. it can count stories but can't read a single word of them

rule: if your analytics can read your users' private stuff, it's not analytics, it's a liability.

what's next

-> email sign-in as a second key, so a lost link never means lost stories

-> an interview mode where it asks the next question based on what they just said

-> a printed hardcover book

and every project i build this october follows the same shape: same deploy setup, same credit guard, same Pulse dashboard. week 1 is already in progress.

who's the first person in your family you'd record?

i post the build as it happens on X: x.com/rahulpandey187

my AI agent forgets everything. so i'm spending october fixing that

Rahul Pandey — Mon, 05 Oct 2026 12:22:34 +0000

being honest here

most of the agents i build for clients are smart for exactly one conversation.

close the tab and it's gone. the context, the decisions, the "we already tried that" stuff. next session you explain everything again.

i've been patching this with long prompts and copy-pasted notes. it works, until it doesn't.

so this month i'm doing something about it, in public.

what i signed up for

Mergetober is a month-long open source event by WeMakeDevs and cognee. It runs Oct 1 to Oct 31.

cognee is an open source memory layer for AI agents. you feed it documents, chats, code and data, and it turns them into a knowledge graph your agent can actually remember and search across sessions.

the catch:

an agent only remembers what it can connect to
cognee has 5 data-source connectors today (Notion, Slack, Gmail, Google Drive, Confluence)
there are 140+ open issues asking for more: GitLab, Linear, Discord, Elasticsearch, Trello...

that gap is the whole event. every connector someone ships is one more place an agent can learn from.

what i'm actually doing

-> pick a connector i can test on a real account (not a mock)
-> claim it properly on the issue and wait to get assigned
-> build it the way the existing Notion connector is built
-> incremental sync, so it only pulls what changed
-> delete handling, so if you delete something upstream the agent forgets it too
-> tests, a README and an example you can actually run
-> write one blog post per PR about how i use it in real work

that last one is a rule of the event, and honestly my favourite part. it forces me to use the thing, not just merge it.

rules i'm holding myself to

i scrolled through the open PRs before writing this. some people opened 5 connector PRs in a single day.

that's not me.

one connector at a time, done end to end
real account, real data, real search results in the PR
my own code and my own words. AI helps me learn and debug, it doesn't write my PRs
no pinging maintainers to merge. good work gets reviewed

slow is fine. sloppy isn't.

what you'll see from me

every PR, every blog, every "this API is weird" moment goes on one place only: x.com/rahulpandey187

i'll share:

which connector i picked and why
what broke while building it
how the agent behaves once it can actually remember
the PR link + the blog for each one

i'm also doing Hacktoberfest this month, so expect a lot of open source from me in October.

one question for you

if your AI agent could remember one tool you use every day, which one would it be?

tell me on X. the most-asked one might be my next connector :)

Links: Mergetober · cognee · cognee-community connectors