DEV Community: Subarna Maity

Repost : Breach Week Dropped 10 'Patch Now' Posts. A Local Gemma Boiled Them Down to 2 Lines, and Code Checked Every Number It Wrote.

Subarna Maity — Fri, 23 Oct 2026 16:57:38 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady Submission 🦥

Sourish Panda

Oct 23

Breach Week Dropped 10 'Patch Now' Posts. A Local Gemma Boiled Them Down to 2 Lines, and Code Checked Every Number It Wrote.

#devchallenge #hf26challenge #temporal #security

25 min read

---> With @t-rexbytes and @soumyadeepdey

We Searched eBay for 237 Recalled Products. Only 45 of 12,562 Listings Named the Model.

Subarna Maity — Sat, 17 Oct 2026 07:26:20 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

Built by @dronzer2code and @soumyadeepdey @t-rexbytes.

Picture a Thursday afternoon at a kids' consignment shop (a composite, not a real store). A parent drops a bag at the intake desk: a toy fire truck, a wooden learning tower, a sleep machine, some toddler pyjamas. The staffer has about two minutes per item. That morning the U.S. Consumer Product Safety Commission posted its weekly batch of a dozen or so recalls. Nobody at the desk has read them.

Here's the part of one notice that matters, from September 17, for the Melissa & Doug Lights and Sound Fire Truck (about 84,000 sold at Target):

Affected units will have one arrow pointing to the number 25 and the second arrow pointing to the number 4 or 5 or they will have one arrow pointing to the number 26 and the second arrow pointing to the number 3.

The arrows are two date dials under the battery box. The brand and the price tag can't answer it. You have to turn the toy over.

For the shop this is a legal question. CPSC's Resale Round-up campaign told thrift stores and online resellers directly that re-selling recalled products violates federal law. On October 6, Consumer Reports, via News4Jax, warned that banned and recalled baby gear is still listed on Craigslist, eBay and Facebook Marketplace. eBay said its product-safety systems (AI models, keyword rules and manual review) stopped 21 million potentially unsafe listings worldwide in 2025. And a 2026 Buchalter analysis notes that CPSC has now sent its 100,000th takedown notice for recalled or banned products sold online.

We built RecallDesk for that desk. Every Thursday it reads the new recalls, checks a shop's stock and eBay store against them, and prints one page: what to pull, and what to flip over.

Where the model number hides

Before any model code, we pulled every recall CPSC posted from July 18 to October 15: 151, all on Thursdays, 48 for children's products. Adding children's recalls back to 2023 (shops sell old baby gear too) made 591.

Then, with plain code and no model, we asked where the identifying number appears. In 73 of the 151 notices, the model, item, style or SKU number is only in the description, never in the headline or product name (24 of 48 for children's). "Woodure Toddler Kitchen Step Stools Recalled" gives you the brand. Whether your stool is affected is further down: "The model number is engraved on the underside of the bottom step."

On the resale side, SerpApi's eBay engine searched for 237 recalls (the 151 recent ones plus 86 older children's ones; 235 paid searches, since two recalls shared a cached query): 14,068 results, 12,562 unique listings. We counted titles containing a recalled identifier, ignoring case and dashes. There were 45, across 13 recalls. For children's recalls, 2 of 6,901, both for one brand of pool dive sticks.

Second-hand sellers are the least likely to type one: 17.1% of all listings were pre-owned (20.2% for children's recalls), but only 9% of the 45 exact hits were.

That count shaped the product. A keyword filter can flag the 45 but can't clear the other 12,517, because a title without a model number proves nothing. For most same-brand items the useful answer is "go look at the label", plus where the label is.

What I Built

RecallDesk is a weekly recall desk for resale businesses. A shop loads its stock once (CSV or paste) and, optionally, its eBay seller name. Every Thursday at 2:00 PM Eastern, after CPSC posts, it turns each new recall into a structured spec (brand, model numbers, date codes, sold window), compares every item and live listing against the specs that could apply, applies the shop's past corrections, and prints one card: DO NOT SELL for match, CHECK LABEL for check with the place to look, and nothing for no_match.

A trimmed excerpt of a real card for the demo shop, from a run with the fine-tuned model on Tinker:

=== RECALL CARD card_2026-10-15_little-loop  Little Loop Kids Consignment  (week of 2026-10-15) ===
  6 new recalls this week | 60 items checked | model: qwen3.5-4b-recalldesk-lora
  DO NOT SELL (8)
    x Papablic Convertible Bedside Sleeper with Play Yard (model PAPA03001AI)  -> CPSC 27027
        why: Listing matches recalled brand, product type, and explicitly states the recalled model number.
        remedy: Stop use, destroy product, email photos for refund
    x Love To Dream Portable Sleep Machine with Night Light (model LTD-SM23)  -> CPSC 26798
  CHECK LABEL (7)
    ? Woodure Wooden Toddler Learning Tower  -> CPSC 26658
        look for: model no. WD1764, WD1357, WD1720; sold 2024-07 to 2026-06. The model number is
                  engraved on the underside of the bottom step.
    ? Sweet Cubby Kids Loungewear Set (shirt + shorts)  -> CPSC 26752
        look for: lot/date code Batch No.: 272779; sold 2025-06 to 2026-05. "Batch No.: 272779" and
                  the size are printed on the sewn-in neck label and on the seam label.
  SHOP MEMORIES APPLIED (1)
    * beeeda90-bb74-4681-b1fa-bfdacd9ef8e6: Shop correction: 'HARPPA Nordi Foldable Toddler Tower' is
      NOT the recalled product for CPSC recall 26651. Ours is the 2022 one - consignor's receipt s...
  model usage: 179 classify calls for 60 items | ... | est. $0.0348 at Tinker list price
  workflow: check-little-loop-2026-10-17-e610   memory: backboard

One line on that card is wrong, and it's wrong in the cautious direction. The Love To Dream recall covers model LTD-SM23 with date code 7W15TN, so our own rule 3 says the item should be CHECK LABEL, not DO NOT SELL. The shop would pull a sleep machine that might be fine.

The dashboard is Next.js with BoardUI components. Most owners only need the printable card. The rest is for setup and for answering "why was this flagged?"

Any inventory row opens beside the recall it was compared with. "Not a match? Correct this" is how a shop teaches the system about its own stock.

The recalls page puts CPSC's text, identifiers highlighted, next to the extracted spec:

The eBay monitor is for flippers who sell online:

Who would pay? Kids' consignment shops, which take in second-hand children's goods from strangers daily. Seasonal consignment events, with thousands of items and volunteer staff in one weekend. eBay and Poshmark flippers buying returns pallets blind. We priced it at $15 a month per shop and $49 per consignment event. A full re-check of the 60-item demo shop took 179 model calls and cost $0.035.

It isn't legal advice or a substitute for cpsc.gov/Recalls. It reads text and can't see a label in a photo, which is why CHECK LABEL exists. U.S. CPSC recalls only.

Demo

One-click deploy of the full stack (agent runtime plus dashboard) from the repo's Render Blueprint:

Or run it locally. Every key is optional, and anything missing falls back to a labelled substitute.

git clone https://github.com/SoumyaEXE/recalldesk && cd recalldesk
cp .env.example .env            # TINKER_API_KEY / SERPAPI_API_KEY / BACKBOARD_API_KEY, all optional

python -m uv venv --python 3.12 agent/.venv
python -m uv pip install --python agent/.venv/Scripts/python.exe -r agent/requirements-tinker.txt
agent/.venv/Scripts/python -m agent.dev --schedule        # Temporal :7233 (UI :8233), worker, API :8000
agent/.venv/Scripts/python -m agent.run_check little-loop # run one shop's check and print its card
agent/.venv/Scripts/python -m agent.demo_crash            # kill the worker mid-check, watch it recover

cd web && npm install && npm run dev                      # dashboard on :3000

On macOS or Linux the interpreter is agent/.venv/bin/python.

Code

SoumyaEXE / recalldesk

Before you list it, check it. Weekly CPSC recall check for resellers and kids' consignment shops — a 4B open model fine-tuned on Tinker, Temporal, SerpApi, Backboard.

RecallDesk

Before you list it, check it. A weekly CPSC recall check for kids' consignment shops and eBay resellers.

Reselling a recalled children's product is a federal violation in the US, and recalled baby gear keeps turning up on resale sites. CPSC posts about a dozen recalls every Thursday, and in 73 of the 151 recalls since 2026-07-18 the model number that decides it is only in the description's fine print. eBay sellers almost never type it into a title.

RecallDesk reads every recall, checks a shop's inventory and eBay listings against it, and prints one card a week DO NOT SELL for items that are the recalled product, CHECK LABEL (with exactly where the label is) for items only a person holding it can decide. Then it goes quiet until next Thursday.

The rule of the build: the big model teaches, the small model reads, code decides.

…

View on GitHub

There are four directories. ml/ holds data collection, specs, eBay search, labelling, training and evaluation. agent/ holds the Temporal workflows, the API and shop memory. web/ is the dashboard. shared/schema.py is the single contract the other three code against. Every figure here traces to ml/results/*.json, data/*/…stats.json, docs/charts/chart_data.json or a transcript under agent/demo_logs/. License: MIT.

How I Built It

Rules first, model second

We wrote the decision rules before choosing a model. Our three verdicts are defined in ml/LABELING.md:

match: the item is the recalled product. Brand and line agree, and either a recalled identifier is visible or the recall covers every unit of that exact product.
check: same brand and line, but whatever decides it (model number, date code, sold window, variant) isn't visible.
no_match: a different product, an excluded model, or only an accessory, part or "compatible with" item.

Seven ordered rules apply them. Rule 7 did the most work: an unbranded listing with no recalled identifier is a generic look-alike, so no_match, unless it names something only the recalled product has, like a licensed character. The model outputs one small JSON object, and plain code maps it to the card.

A large open model, Qwen3.5-397B-A17B, wrote specs for all 591 recalls (100 seconds, $2.21), and code drops any identifier not actually in the notice. It also labelled 3,300 (recall, listing) pairs, and kept calling unbranded look-alikes "check": 1,134 checks on the first pass, 872 after we added rule 7 to the prompt. Still too many, so we stopped asking. A short function checks the rule and rewrites the training target when the teacher breaks it:

def is_generic_lookalike(spec: dict, title: str) -> bool:
    words = set(norm(title).split())
    if brand_words(spec.get("brand", "")) & words:      # names the brand -> not generic
        return False
    if id_hit(spec, title):                              # shows a recalled identifier -> not generic
        return False
    return len(distinctive_words(spec) & words) < 2      # fewer than 2 distinctive product-name words

It fired on 391 pairs: 357 in training and 34 in the test split. Every change is logged in data/labeled/targets.jsonl.

The 397B teaches once; a 4B works every Thursday

At $2.20 per 1,000 checks, the 397B is too expensive to run for every shop every week. So with Tinker we fine-tuned Qwen3.5-4B (LoRA rank 32) on its corrected labels: 3,007 pairs, 2 epochs fixed in advance, 376 steps at batch 16, learning rate 4.91e-4 decaying linearly, loss on the JSON only. Our code assembles batches and calls forward_backward and optim_step; Tinker's GPUs do the rest, and save_weights_for_sampler returns the tinker:// path the weekly workflow samples. About 38 minutes; held-out loss 0.724 → 0.228. With a ~400-token prompt (the teacher's is ~1,060), the student costs $0.19 per 1,000 checks.

Tinker spend at list prices: specs $2.21, discarded first labelling run $6.80, second run $7.47, training $2.39, evaluation $0.23, about $19.10 total. SerpApi: 235 of the free plan's 250 searches.

Scoring it without fooling ourselves

We split by recall: 217 for training, 20 held out (12 children's), with no recall or listing shared. One try per pair at temperature 0; invalid JSON is wrong. The evaluation harness (bootstrap CIs, exact McNemar) and the Temporal crash-test pattern come from @soumyadeepdey's earlier open-source TrailTruth project (repo).

One caveat shapes everything below. We built a blind labelling tool but didn't finish human labels in time, so the answer key for the 293 test pairs is the teacher's verdict with rule 7 applied by code, not a person's. The scores show how faithfully a 4B learned the teacher's rulebook on unseen recalls, not how often it's right about the world.

On 293 held-out pairs	Accuracy (95% CI)	False alarms (of 255)	Risky pairs flagged (of 38)	$ / 1k checks
Qwen3.5-4B, base	0.512 (0.457–0.570)	120	34	$0.22
Qwen3.5-4B, base + teacher's full rulebook	0.625 (0.570–0.679)	106	36	$0.37
Identifier/keyword rules, no model	0.864 (0.823–0.901)	22	22	$0
Teacher, Qwen3.5-397B-A17B	0.884 (0.846–0.918)	34	38	$2.20
Qwen3.5-4B + RecallDesk LoRA	0.918 (0.884–0.949)	12	26	$0.19

The teacher's 38 of 38 is automatic, because every gold "check" is one of its own. Its 34 false alarms are exactly the pairs rule 7 overrode. The base 4B returned invalid JSON on 14 pairs, and every other model on none.

Fine-tuning moved the 4B from 0.512 to 0.918 (+40.6 points, 95% CI +33.8 to +47.1, exact McNemar p < 0.001). Against the teacher the gap (+3.4, p = 0.20) isn't significant: same reading, about a twelfth of the cost. For a shop the false-alarm column matters most. The base model sent 120 of 255 harmless listings to CHECK LABEL, which would bury the real ones. The LoRA sends 12.

The price: the LoRA flags 26 of the 38 risky pairs; its base flagged 34 (on risky pairs alone the base is ahead, 11 to 3, p = 0.057, not significant). Nine of the twelve misses are one recall, a Sviyatp pool drain cover that, per CPSC, "has no labeling or product identification on the product itself". The listings are unbranded white drain covers, which rule 7 reads as no_match, and that's what the student said. Our coded rule kept them as "check" only because the titles share "dual suction outlet" with the product name printed on Sviyatp's packaging. Two more misses are unbranded baby toys; one is a 14-pack of finger lights against a recall of 28- and 30-packs. So it's a genuine trade: fewer false alarms in exchange for trusting rule 7 on products with no label at all. On the 180 children's pairs the LoRA scores 0.933, the base 0.428.

A concrete false alarm: for TheKiddoSpace's recalled LED hover ball, eBay returned "Rechargeable Hover Soccer Ball LED Lights Air Power Indoor Outdoor Toy Kids 3+", no brand anywhere. The base said check because the listing "matches the brand". The LoRA said no_match: "Generic look-alike: no brand and no recalled identifier in the listing." Per-request latency on Tinker's hosted sampler is 2.3–2.8 s for every model, so the win is cost, not speed.

Finding the listings (SerpApi)

Each call is engine=ebay, ebay_domain=ebay.com, _ipg=50, with the teacher's first query as _nkw. Responses are cached under a hash of their parameters, so nothing is paid for twice, and a 429 halts the run. A shop's own store uses _sasl=<seller>&_saslop=1 (the engine has no _ssn). Seller names stay in the raw cache; published data uses a hash.

Thursday, every Thursday (Temporal)

recalldesk-weekly-sweep is a Temporal Schedule for Thursdays, 14:00 America/New_York. It next fires October 22 and 29 at 18:00 UTC, then November 5 at 19:00 UTC once the clocks change. Every step is an activity with its own failure policy: CPSC fetches back off, then fall back to a committed snapshot; classification heartbeats with six calls in flight, so a stuck call gets retried; malformed JSON is retried, then answered by the rules with a note on the card; a SerpApi 429 is never retried, and the step records source=snapshot (SerpApi quota/429, not retried).

Does a dead worker lose work? We started a check of the demo shop's 60 items (176 comparisons) and killed the worker process 12.4 seconds in, 19 items done. For three seconds nothing ran, while Temporal still showed the workflow running with six activities in flight. A fresh worker replayed the history and published the card at 33.2 seconds. The history: 60 item activities, each scheduled once; the 20 finished before the kill were kept, the 6 in flight got a second attempt, none completed twice. This rehearsal used the rules classifier plus 1.2 s of artificial latency per item, to make sure the kill hit mid-run.

Then we did it again with the real LoRA on Tinker (workflow check-little-loop-2026-10-17-1a16-crashdemo). We killed worker pid 12764 at 73.2 seconds with 18 of 60 items done and 6 calls in flight, and a fresh worker started at 76.5 seconds. The card was published at 170.9 seconds. Again there were 60 activities, each scheduled once: 18 kept, 6 retried, none completed twice, 344 history events. The chart below is this LoRA run; the Temporal UI screenshot after it is from the rehearsal.

The dashboard's runs page badges the same run as resumed:

With injected faults on the LoRA worker (fake CPSC 503s, hung calls, broken JSON), ten items needed retries and three were answered by the labelled fallback, and the card noted it. A manual sweep for recalls since October 8 fetched 14 live from CPSC and fanned out to one child workflow per shop, producing two cards in 83 seconds.

Remembering a shop's corrections (Backboard)

No model can know that this shop's fire truck reads 24/11 on its dials. The staffer holding it can. Their "Not a match? Correct this" note is stored as a memory in the shop's own Backboard assistant (explicit memory endpoints only). Next run, load_shop_memories retrieves it by meaning, not exact text. The memory is kept away from the model, which still answers check; code then changes the verdict to no_match and lists the correction on the card. In the recorded run (LoRA on Tinker, Backboard memory 063081d1-…), CHECK LABEL went from 7 items to 6 and DO NOT SELL stayed at 8.

An override needs a matching recall number and at least 60% word overlap with the remembered item, so a note about one tower can't clear another. The same run also applied an older HARPPA tower correction (memory beeeda90-…). One honest wrinkle: semantic search was flaky on long item names. The full "Melissa & Doug Lights & Sounds Fire Truck Activity Board" query didn't return its own correction, while a shorter brand + name query did. So each run now searches with brand, name and model only, and also lists the shop's memories once, putting both through the same overlap guard. In the recorded run the fire-truck correction came through the list.

Where it broke

A 402 froze training. When our Tinker credit ran out mid-run, the pending request never resolved, and the loop waited on it with no error. Every wait now has a 300-second timeout, and weights are saved at the end of each epoch.
The teacher wouldn't follow rule 7. Two prompt revisions only reduced the violations, so in the end code enforced the rule on the training targets.
Backboard's free credit covered memory only. We had planned to run extra open-weight baselines through Backboard's chat, and that wasn't included, so those rows aren't in the table.
Entire recorded nothing. Its hooks are installed per repository, and our coordinating session was started outside the repo folder, so no checkpoints exist and we didn't enter that category.
The first labelling run crashed on its first write because the output folder didn't exist yet. That took one line to fix.
Some recalls can't be read by anyone. Six notices end with "the following model numbers are included:" and put the list only on cpsc.gov, which refuses automated requests.

Why Does Open Innovation Matter?

A small shop's margins can't carry a frontier-model bill per item per week. Open weights let us train a 4B for this one job, and training is where the false alarms went; a closed API would have left a choice between the expensive model and the noisy prompt.

No lock-in either: the adapter exports as PEFT weights for a shop's own vLLM or llama.cpp server, selected with one environment variable, and with no keys RecallDesk still runs on its rules classifier and labels its cards accordingly. Specs, splits, predictions and the eval script are in the repo, so anyone can re-score our 293 pairs or add the human answer key we didn't get to.

Privacy: everything RecallDesk reads is public (CPSC's open API, public eBay search results via SerpApi). Seller usernames never leave the gitignored raw cache, and the demo shops are invented. By default, though, labelling, training and inference run on Tinker's hosted GPUs; local means the exported adapter.

My Agent Session

A coordinating session ran five parallel coding agents (ML and data, Temporal backend, dashboard, charts and screenshots, writing). Its first output wasn't code but CONTRACT.md: every data shape, every API route, and the rule that code, not the model, decides what reaches the card. With that pinned, the dashboard was built on real-data fixtures while training was still running, and the parts fit on first connection.

My First User

No paying shop has used RecallDesk yet, and we won't dress up a hypothetical as a customer. From the first kids' consignment owner we want to learn how long recall checks take today, whether CHECK LABEL is clear without a walkthrough, what price they'd name before seeing ours, and what would stop them. We suspect liability and data entry, which would put a POS import ahead of any model work.

So here's the offer, open until winners are announced: if you run a resale shop or flip on eBay in the U.S., send us your item list or seller name and we'll run it through RecallDesk this Thursday and send you the card, free. We'll add the first real reply to this section, with permission.

Prize Categories

Best Use of Tinker: Qwen3.5-397B-A17B writes specs and labels; Tinker's training API fine-tunes Qwen3.5-4B (LoRA rank 32), and its sampler serves it every Thursday. On 20 unseen recalls: 0.512 → 0.918 accuracy (p < 0.001), 120 → 12 false alarms of 255, $0.19 vs $2.20 per 1,000 checks, about $19.10 all in. Caveats (teacher-built answer key, 26 of 38 risky flagged) are spelled out above.
Best Use of SerpApi: 235 cached eBay-engine searches across 237 recalls, 12,562 unique listings, and the 45-of-12,562 finding the product is built around; plus a weekly per-store scan via _sasl + _saslop=1, with 429s never retried.
Best Use of Temporal: a DST-aware Thursday Schedule, per-activity retry and fallback policies, heartbeats on classification, and a worker-kill test: 60 activities, 20 kept, 6 retried, zero duplicates.
Best Use of Backboard: per-shop correction memories, retrieved by meaning and applied by code behind a recall-number and word-overlap guard, so one line from a staffer changes next week's card.

If you run a kids' consignment shop or a seasonal sale, or you flip returns on eBay, comment or message us. We'll run your store through RecallDesk free this Thursday and send you the card. For everyone else: has a recall ever made you turn something over to read the label, and what was it?

Built by @dronzer2code and @soumyadeepdey @t-rexbytes.

FarbeDex: a colour picker that only works outside (and never needs the internet)

Subarna Maity — Sun, 11 Oct 2026 05:35:12 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

I pick colours for a living, and almost all of them come from other screens: palette generators, screenshots, a hex code a client pasted into chat. Meanwhile the best palettes I've ever seen were outside my window the whole time. Moss on a wet wall. A late sky fading from peach to blue. Rust on a gate nobody has painted in years.

So I built FarbeDex, an Android app that turns what you see outside into colours you can keep.

It takes almost no time. Point your camera at the view, the sky, a cloud, even the clouds below you from a plane window, and tap once. The palette appears right away, with how much of the scene each colour covers. Pick your two favourites, Gemma names them a moment later, and your phone goes back in your pocket. You go back to the view.

Your favourites join a collection of 151 colours, from everyday greys to Mythical vivid violets, each one a little colour pal with its own face. And every catch hands you a palette you can drop straight into your next design.

Most apps that get you outside give you a job: run this far, find this bird, finish this quest. FarbeDex gives you none. No checklist, no target, no streak. When something makes you stop and look, that's your catch.

And it all runs completely offline, with two open AI models living on your phone.

Demo

Home: your collection at a glance

Catch, then pick your two favourites

A Legendary reveal

Your FarbeDex fills up

Gemma naming colours in airplane mode
🎬 Demo video:
📱 APK: https://github.com/Dronzer2code/FarbeDex/releases/tag/v4.0

Code

Dronzer2code / FarbeDex

FarbeDex

Catch real colours from real places. FarbeDex is an offline Android app for designers and front-end developers. You go outside, photograph a scene with the in-app camera, and FarbeDex extracts its palette and how much of the scene each colour covers. Every colour lands in the FarbeDex: a fixed index of 151 colour entries (#001–#151) that you complete over time.

Built for the DEV Hacktoberfest Open-Source AI Challenge, Week 1: Touch Grass.

Screenshots and demo video: TODO (added after the field test) APK download: TODO (GitHub release v1.0.0)

How it works

Camera only. There is no way to import a picture. Live checks (hand-held motion, a fresh capture) and the Referee make sure a catch is a real outdoor scene, not a screen or a print.
Deterministic colour maths. The photo is reduced to up to 7 colours with k-means in OKLab, and each colour's share of the scene…

View on GitHub

How I Built It

The maths sees, the models judge and name

The biggest decision was what not to give to AI. Colour extraction is plain maths: k-means clustering in OKLab, a colour space built around how people perceive difference. The same photo always gives the same palette. Every colour lands in one of 151 entries: 7 neutrals plus 16 hue families × 3 lightness × 3 chroma bands.

I wanted the collection to be completable, so a test sweeps the whole sRGB cube and checks that all 151 entries can actually be reached. Rarity follows one idea, "how often does nature show this colour": greens and greys are everywhere, vivid blues and violets are not. That gives 42 Common, 42 Uncommon, 37 Rare, 21 Legendary and 9 Mythical.

The models do only what models are good at: judging a scene, and naming things.

Gemma 3 270M, the namer that fits in your pocket

The star of FarbeDex is Gemma 3 270M, running on the phone through llama.cpp as a 253 MB file. When you register colours, Gemma names each new one, gives the whole palette a one-word mood (COOL, EARTHY, SERENE…), and writes a one-line tip on using the palette in an interface.

A 270M model has big ideas and small habits, so it never writes free text into the app:

Structured output only. Every request uses a JSON schema compiled into a grammar, and the schema pins exactly which colours Gemma must name. It can't skip one or invent one.
Gemma writes the poetry, the app writes the facts. Early on, Gemma named a pale blue "Dark Charcoal". Now Gemma writes only the evocative part ("Rooftop Morning"), and the app adds the true colour word ("…Blue"). Colour words are banned from Gemma's half, so a blue is always called blue.
Validate, retry once, then fall back. Every name is checked for length, uniqueness and a blocklist. A failed name gets one retry with the reason attached; after that a plain name like "Vivid Pale Blue" takes over, so you're never left waiting.

My first Legendary indigo came back as Ocean Waves. Here it is in the Tailwind export, next to the plain fallback names of colours Gemma hadn't named yet:

More names Gemma gave my catches: [2–4 more names from your catches]

MobileCLIP, the referee that keeps it honest

The second open model is MobileCLIP-S0 on ONNX Runtime. It answers two questions: is this a real outdoor scene, and what's in it?

The phone never runs a text encoder. I embedded every label on my laptop ("a computer monitor", "potted plants on a balcony", "the blue sky"…) and ship them as a small file, so on the phone one image embedding and a few dot products do the job. It rejects only confident screens and home interiors, so balconies, rooftops and pot plants all count. And the subject it names must match the photo's real colours.

Yes, I tried to cheat by photographing my own laptop. It didn't work:

Pick two, and sometimes get lucky

Catching shouldn't be mindless, so you choose two favourites per catch, and only those join your collection. Rare, Legendary and Mythical picks get a special reveal, and each has a 1-in-400 chance to be shiny: gold rim, sparkles and its own chiptune jingle, all original and generated in code.

Two AI teammates, one source of truth

I built FarbeDex in about two and a half days with two AI tools, each with its own job.

GitHub Copilot is the editor. While Claude builds, Copilot keeps the documentation updated in parallel, so the docs never fall behind the code. When each Claude task finishes, Copilot curates the commit message, turning a long session into one clean line of history.

Specs go in; tested, documented commits come out. The repo ships with 279 passing tests.

We Touched Grass

I'm a graphic designer and a front-end developer, which mostly means I'm a person at a desk. My whole working day happens on a laptop. When the work stops, the scrolling starts: my "free time" was Instagram Reels until my thumb got tired.

I didn't notice what it was doing to me until it had already done it. I went out less, talked to people less, and slowly got bad at simple things: starting a conversation, being somewhere without a screen, just being outside. Climbing back out took real, slow work, and I did most of it alone. Nobody hands you a plan for that.

So I know exactly where the problem sits. People like me know the outdoors exists. What we lack is a reason to step out that doesn't come with pressure. Every reason the internet offers is another task: 10,000 steps, a streak, a specific bird. When you're already drained, one more task is exactly what keeps you at your desk.

FarbeDex is built for that version of me. For a designer, colour is the reason. There's no target and no wrong answer. You walk to the corner, notice the light on a wall, and tap once. That moment becomes something real for your work, and a little colour pal you actually want to show people. Everything runs on the phone, so you can leave the Wi-Fi, and the feed, behind.

Then comes the moment later, back at the desk. You remember that evening and think: those colours were so good, I wish I had them in my design. Now you do, in five formats:

Palette image: a swatch card with a one-word mood, ready for a moodboard.
HEX codes: copy any colour with one tap.
CSS: custom properties to paste into a stylesheet.
Tailwind: a colors block for your config.
JSON: design tokens for Figma plugins and design systems.

A built-in contrast table even tells you which pairs pass WCAG for text, so the sky you saw can become a readable website.

Here's what happened when I used it myself:

[Your first walks with FarbeDex: balcony, roof, the street near home. How long you stayed out, how many colours you caught, your rarest find, a shiny if you got one, and what it felt like to look up instead of down. 2–3 photos.]

[Your FarbeDex counter before and after, e.g. "0 / 151 → 2 / 151".]

Why Does Open Innovation Matter?

It works where colour lives. Parks, rooftops and planes often have no signal. Both models (276 MB together) run fully on the phone, in airplane mode.
Your photos stay yours. Nothing is uploaded, there's no account, and the only permission FarbeDex asks for is the camera.
It costs nothing per catch. No API bill, no rate limit, and it will work exactly the same next year.
You can see inside it. Every prompt, label and threshold is in the repo. Don't like the names? Change the prompt. Want a different referee? It's one config line.

A big closed vision model might label scenes a little better. But it would need every outdoor photo uploaded, a signal in the middle of a park, and money for each catch. For noticing colours on a walk, two tiny open models are exactly enough.

What Didn't Work

The quantized referee saw screens everywhere. The int8 version of MobileCLIP called a beach, a lake and a savanna "a screen". Switching to the fp16 file (23 MB) fixed it completely.
My own balcony counted as indoors. On the phone, every shot from my balcony and roof was rejected as "inside". I flipped the logic: accept by default, and reject only when the model is sure it sees a screen or a home interior.
A sky called "fern". One prompt per subject wasn't enough. Several prompts per subject, plus a rule that the label must match the photo's real colours, fixed it. No green, no fern.

Prize Categories

Best Use of Gemma: Gemma 3 270M runs fully offline on an Android phone, naming colours, giving palettes a mood and writing design tips through validated, schema-constrained JSON.

Team

Subarna Maity - Me
*Sourish Panda - @t-rexbytes *

I set out to build a colour picker that makes you go outside. What I ended up with is an app that, most days, just asks you to look up. 🌿

The groove is the 40 milliseconds a grid throws away, so we built a beat studio that gives them back, in a browser tab

Subarna Maity — Sun, 04 Oct 2026 20:41:51 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

At 90 BPM a sixteenth note lasts 166.7 ms. Swing at 50% pushes every second one of them late by 41.7 ms. That is the entire difference between a hi-hat that marches and one that bounces, and it is shorter than one frame of most phone videos.

A plain step grid snaps every hit onto those 166.7 ms boundaries and throws the 40 ms away. We built one that keeps them, and then asked a harder question.

The single most common mistake in a multi-sponsor project is using each sponsor as a logo. A drum machine that stores a JSON blob somewhere is not a database project. A text box that calls an API once is not a sound-design project.

So the first question was: what can each piece do that the others physically cannot?

Piece	The one thing only it can do here
The browser	Play audio with sample-accurate timing and run open-source models on the user's own device
Tiger Data	Keep workspaces, audio, 384-d embeddings and a full-text index in one Postgres, and answer "find me a warm, dusty kick" with a single SQL statement
ElevenLabs	Produce a sound that exists in no sample pack, from a sentence
Entire	Keep the agent's reasoning next to the commit it produced, so the build itself is reviewable

My friend Anuv makes beats, and his ideas don't wait for him to be at his setup.Krisp is what we built him.

What I Built

Krisp is a minimal, hardware-feel beat studio in a browser tab. You tap a beat on pads, give it feel with velocity and swing, let an open-source model humanize it or offer four variations, describe a sound and get it, find any sound again by its vibe, arrange beats into a song, mix it through a real master chain, and export a WAV, or MIDI for your DAW.

 pads, keys --> zustand store --> scheduler -------> Tone.js engine --> speakers
 (tap, paint,   one Workspace,    pure timing math,  channels, reverb,
  velocity,     undo history      100 ms look-ahead  comp + limiter    same engine
  live REC)          |                                   |        --> Tone.Offline --> WAV
                     |--> Magenta.js (GrooVAE, MusicVAE, DrumRNN)   in the browser
                     |--> all-MiniLM-L6-v2 (384-d embeddings)       in the browser
                     |
                     '--> /api (Vercel) --> Tiger Data: Postgres + pgvector + full-text
                                       '--> ElevenLabs Sound Effects (server-side key)

One thing up front, because it changes how you read every number below

Every number in this post was measured, and here is where:

Audio and AI timings: headless Chrome on a Windows laptop with no GPU, against a local dev server. A laptop with a GPU will be faster. I haven't measured that, so I don't claim it.
Database latency: India to a Tiger Data service in US-East. The deployed functions run in Vercel's iad1, next to the database.
Search measurements used synthetic test sounds that the test script generated, uploaded and deleted again.
Where something broke, the error is quoted verbatim in What Didn't Work.

Demo Video

The walkthrough

Six stops, in the order you meet them.

1 · The dashboard

Every idea is a card. Its colour strip is the first six beats inside it, so a sketch is recognisable before you open it. Under the cards, a hand-lettered guide teaches the whole app with real controls you can touch.

For Anuv: his ideas stop living in voice memos and stray files. They are all one click from Home, and he never needs a manual.

2 · Creating a workspace

Start empty, or from one of eight original patterns, each with its own tempo and swing.

For Anuv: a groove is playing seconds after he opens the app, as a starting point he reshapes by hand, not a blank grid.

3 · The full workspace

One screen: the transport on top, the Rack on the left, the Beat Editor in the centre, the Song underneath, Sounds and AI on the right. Saving is automatic. The LED goes amber, then green.

For Anuv: nothing hides behind menus or modes. Beat, song and sounds are all in view while the idea is still warm.

4 · Left side: the Rack and the Beat Editor

Each Rack row is one sound, with tune, mute, solo and a preview pad, and it is the row header of the grid beside it: same height, same order. Click to place a hit, drag to paint a row, Shift-click for a ghost note or an accent, or hit REC and play it in from the keyboard.

For Anuv: he programs the way he already thinks, and the grid keeps the velocity and swing that make it his.

5 · Right side: Sounds and AI

Four tabs: Presets, Create (ElevenLabs), Upload and Library. The Library sorts sounds into playlists and searches by name or by vibe. The AI tab next to it holds Humanize, Variations, Continue and Morph (Stage 4 below).

For Anuv: an evening of digging through sample packs becomes one sentence and one search box.

6 · The mixer

Press M. Every sound gets a strip with EQ, reverb send, pan and fader. Then a reverb return, and a master with a compressor, gain-reduction LEDs and a limiter.

For Anuv: a sketch leaves Krisp already mixed, so the WAV he sends sounds finished, not like a demo.

Demo

Live: https://krisp.vercel.app

Password : subaru2007

Saving and the sound library sit behind a studio passcode, because ElevenLabs credits cost money. The video shows them. Everything in the next list works without it.

Verify this in 60 seconds, no keys, no clone

Open https://krisp.vercel.app → New workspace → Lo-fi → Create. If it asks for a passcode, click Not now. You can still make music; it just won't save.
Press Space.
Shift-click pads in the snare row. They cycle ghost → soft → normal → accent. Turn Swing.
AI tab → Humanize. The first run downloads the model. Then tiny dots appear inside the pads: that's microtiming.
Export → WAV.

The check I would make if I were judging this

Drop that WAV into Audacity and zoom into the hi-hats. At 90 BPM with 50% swing, every second hat must land about 41 ms late, and the even ones must not move.

When I ran exactly this against Krisp's own export, the odd hats landed 0.24 of a step late (40 ms against the formula's 41.7 ms) and the even hats moved by less than 0.02 of a step. That is how you know the WAV is the same engine you were listening to, and not a re-implementation of it.

Code

Dronzer2code / Krisp

Krisp — a beat workspace that runs in the browser

Krisp is a calm, hardware-feel beat workspace built for one producer friend. Make beats by hand on a step sequencer, arrange them into a song, mix them with channel strips and a master chain, and export WAV and MIDI for your DAW. Open-source AI (Magenta.js) runs in your browser to humanize and vary your own beats. ElevenLabs creates the exact one-shot or loop you describe, and a Tiger Data hybrid search finds any sound again by its name or by its sound ("warm dusty kick").

Built for the DEV Hacktoberfest Weekend Challenge: Build for a Friend (October 2026).

Live demo: URL added after deploy (the sound library and saving are behind a studio passcode)
Docs for agents and contributors: CLAUDE.md → docs/

What it does

Area	Features
Workspaces	Create (empty or from 8 original preset beats), open, rename, duplicate, delete;

…

View on GitHub

My Agent Session

Krisp was built with Claude Code and captured with Entire (see Stage 7):

Overview · krisp/krisp · Entire

Entire captures your AI agent sessions and stores them alongside every commit.

entire.io

Created inside the challenge window (first commit 2026-10-03). Open-source work Krisp stands on is credited in the README: Magenta.js, Tone.js, transformers.js with all-MiniLM-L6-v2, @tonejs/midi, pgvector, and Petaluma Script for the hand-lettered guide.

Run it yourself

git clone https://github.com/Dronzer2code/Krisp && cd Krisp
cp .env.example .env    # TIGER_DATABASE_URL, ELEVENLABS_API_KEY, APP_PASSCODE, DAILY_SOUND_LIMIT
npm install
# paste db/schema.sql into the Tiger Data SQL editor once
npx vercel dev          # app + /api on http://localhost:3000
npm run test            # 146 unit tests
npm run build && bash scripts/check-secrets.sh   # no key may appear in dist/

How I Built It

The architecture, actually explained

Stage	Tech	Where it runs	What it does
1	zustand store	browser	One Workspace object, undo history; a whole knob drag is one undo step
2	Scheduler	browser	Pure timing math on Tone.js's Transport, 100 ms look-ahead
3	Audio engine	browser	Channels → reverb → master EQ → compressor → limiter → ceiling clipper
4	Magenta.js	browser	Humanize, Variations, Continue, Morph on the user's own beat
5	ElevenLabs	server	Text → one-shot or seamless loop
6	Tiger Data	server	Workspaces (JSONB), sounds + embeddings (pgvector), full-text, playlists
7	Entire	git	Agent sessions as checkpoints, linked to the commits they produced

The rule behind every row: anything musical happens in the browser and never waits on the network. The network is for saving, the library, and making new sounds. Nothing else.

Stage 2: the scheduler is three pure functions

A drum machine in a browser is a timing problem before it's anything else. All of Krisp's timing is three functions you can unit-test without a sound card:

export const stepSeconds = (bpm: number) => 60 / bpm / 4;

export function swingDelay(stepIndex: number, swing: number, stepSec: number) {
  return stepIndex % 2 === 1 ? (swing * stepSec) / 2 : 0;
}

export function hitTime(tickTime: number, stepIndex: number, step: { offset: number }, swing: number, stepSec: number) {
  const t = tickTime + swingDelay(stepIndex, swing, stepSec) + step.offset * stepSec;
  return Math.max(t, tickTime - 0.4 * stepSec); // a humanized hit may lead, never into the previous step
}

The Transport never loops. The loop region is applied by one more pure function, wrapTick, so drum hits and audio clips are triggered by the same per-tick code and can't drift apart.

Stage 3: the export is not a second renderer

The obvious way to make a WAV is to write an offline renderer. That gives you two engines, and they will eventually disagree. createEngine(context) accepts any audio context, so the export opens a Tone.Offline context and runs the same graph and the same scheduler through it:

const rendered = await Tone.Offline(async (ctx) => {
  const engine = createEngine(ctx);     // identical graph: channels, reverb, master chain
  engine.sync(ws);
  await engine.ready;                    // reverb impulse response
  bindScheduler(ctx.transport, { getWorkspace: () => ws, trigger: engine.trigger /* … */ });
  ctx.transport.start(0);
}, seconds + 2, 2, 44100);

A 16-bar song at 90 BPM is 42.7 s of music. With a 2 s reverb tail, it rendered in 16.1 s on that GPU-less laptop, about 2.7× faster than real time. It also followed the arrangement exactly: audio in bars 1–4, then RMS 0.000000 after.

Stage 4: the AI never writes the beat for you

Most AI music tools generate the song. Krisp's AI only touches the beat you made, and every result is one undo away.

Tool	Open checkpoint	What it is allowed to change
Humanize	`groovae_2bar_humanize`, trained on the Groove MIDI Dataset of real drummers	Velocity and timing of steps that are already on. It can't add or remove a note
Variations	`drums_2bar_lokl_small`	Four candidates near your beat; wildness sets similarity and temperature
Continue	`drum_kit_rnn`	Writes bar 2 from bar 1
Morph	`drums_2bar_lokl_small`	Nine steps from one beat to another

Before Humanize, every hit sits dead on the grid:

After, the same hits, nothing added or removed. Velocities changed (brightness), and each dot is a microtiming offset the model put there:

Variations are previewed by holding a candidate. It plays in place of your beat for as long as you hold it, without touching the beat or the undo history.

Two facts I only learned by running the models are now built into the converter:

MusicVAE and DrumRNN return quantized notes with velocity unset. It reads as 0, so Krisp defaults it to 100.
GrooVAE returns unquantized notes: a start time, not a step. Krisp recovers the step as round(startTime / stepSeconds) and keeps the remainder as an offset clamped to ±½ step. That remainder is the groove.

And one I had to design around: Snare and Clap share drum pitch 38. A model's snare notes would wipe out the clap row, so model notes go to the first slot with a pitch and the rest keep their own steps.

Stage 5: ElevenLabs, the sound that isn't in any pack

Digging through sample packs is search over things that already exist. ElevenLabs makes the thing that doesn't. That's a different job, and it changes the workflow: the producer says what they hear, and the sound turns up on a pad.

Krisp shapes the request and shows you exactly what it sends, so nothing is hidden behind a magic box:

ONE_SHOT  "<your words>, single drum one-shot, isolated, dry, no music"
LOOP      "<your words>, seamless drum loop, <bpm> bpm"        + loop: true

POST https://api.elevenlabs.io/v1/sound-generation?output_format=mp3_44100_128
     xi-api-key: (server only)   { text, duration_seconds, prompt_influence: 0.5, loop }

Everything ElevenLabs is actually doing here

Feature	Where	Why it's load-bearing
Sound Effects API	`api/sound-generate.ts`	The only way to get a sound that exists in no pack
`loop: true`	loop requests	Seamless loops that drop straight onto an AUDIO lane
`duration_seconds` 0.5–2 s	one-shots	Drum hits, not soundscapes
Server-side key	Vercel function	`scripts/check-secrets.sh` fails the build if a key leaks into `dist/`
Daily limit	`sound_usage` table	40 credits per second of sound; the counter ("38 left today") keeps the demo alive through judging

The two generated sounds in this library came back at 1.3 s each. In the browser they are trimmed of silence, faded, peak-normalised to −1 dBFS, embedded, and filed next to uploads:

Stage 6: Tiger Data, one Postgres with three kinds of memory

Producers search two ways. Sometimes it's a word: "808", "clap". Sometimes it's a feeling: "warm", "dusty", "like old vinyl". Keyword search fails the second; vector search fumbles the first. The usual answer is a Postgres and a vector database and glue code between them.

Tiger Data made that a non-problem. One service holds the workspaces as JSONB, the audio as bytea, the meaning as pgvector and the spelling as tsvector. The fusion is a single statement, reciprocal rank fusion with k = 60 (columns abridged):

WITH v AS (            -- meaning: cosine distance on all-MiniLM-L6-v2 embeddings made in the browser
  SELECT id, row_number() OVER (ORDER BY embedding <=> $1::vector) AS r
  FROM sounds WHERE ($3::text IS NULL OR kind = $3)
  ORDER BY embedding <=> $1::vector LIMIT 50
), k AS (              -- spelling: Postgres full-text
  SELECT id, row_number() OVER (ORDER BY ts_rank(tsv, q) DESC) AS r
  FROM sounds, websearch_to_tsquery('english', $2) q
  WHERE tsv @@ q AND ($3::text IS NULL OR kind = $3)
  ORDER BY ts_rank(tsv, q) DESC LIMIT 50
)
SELECT s.id, s.name, s.kind,
       coalesce(1.0/(60+v.r),0) + coalesce(1.0/(60+k.r),0) AS score,
       v.r IS NOT NULL AS vector_hit, k.r IS NOT NULL AS keyword_hit
FROM sounds s LEFT JOIN v ON v.id = s.id LEFT JOIN k ON k.id = s.id
WHERE v.id IS NOT NULL OR k.id IS NOT NULL
ORDER BY score DESC LIMIT 20;

Every result carries a vector badge, a keyword badge, or both, so you can see why a sound came up. In testing, "dusty kick" found the uploaded kick with both badges. "warm lo-fi bass drum" (words that were never its name) found it by vector, plus a keyword hit on its warm tag.

The library is organised the way producers organise folders: playlists, where a sound can live in several at once, sit above everything still unsorted (you saw them in the walkthrough).

This is the whole schema on Tiger Data, live:

Everything Tiger Data is actually doing here

Feature	Where	Why it's load-bearing
pgvector (`vector(384)`, HNSW cosine)	`sounds.embedding`	Search by vibe
Full-text (`tsvector`, GIN)	`sounds.tsv`	Search by exact word
One-statement RRF	`api/search.ts`	Both searches, one round trip, ranked together
JSONB	`workspaces.data`	A whole Workspace (rack, beats, song, mix) in one row
bytea	`sounds.audio`	Audio lives with its metadata. No second storage service
Two-table join	`playlists` + `playlist_sounds`	Folders where a sound can be in many

Stage 7: Entire, the build is reviewable too

A project built with an AI agent has a second artefact besides the code: the reasoning that produced it. Entire keeps it. We ran entire enable --agent claude-code --import-history. That installed Claude Code hooks, and it also imported the session that had started before Entire was set up, so nothing from the first hour was lost. Since then, every git push origin uploads the session checkpoints next to the commits they produced. The repository itself lives on Entire (origin), mirrored to GitHub for Vercel.

Entire's own numbers for this repo: 36 commits, 7 checkpoints, 15 messages, 591.3k tokens (996 in, 590.3k out), about 84.5k tokens per checkpoint.

That is what made the "verbatim errors" table below possible. Every failure, and the reasoning that fixed it, is on record rather than reconstructed from memory.

Everything Entire is actually doing here

Feature	Where	Why it's load-bearing
Claude Code hooks	`.claude/settings.json`	Sessions captured as they happen, no copy-pasting transcripts
`--import-history`	first setup	Recovered the session from before Entire existed in the repo
Checkpoints on `git push`	`entire://…/et/krisp/krisp`	Reasoning travels with the code
Analytics	Entire dashboard	Measurable build: commits, checkpoints, tokens

What My Friend Said

[[HANDOVER: what Anuv did on camera — the tune he made, what he exported, what he took into his DAW. Facts only.]]

[[QUOTE: Anuv's own words, exactly as he said them]]

Why Does Open Innovation Matter?

Four concrete reasons, each one measured or tested:

The beat never leaves the device. I ran the whole AI test suite with every /api request blocked in the browser. Humanize, Variations, Continue and Morph all passed.
It costs nothing to run. No per-request inference bill for the AI. The only metered call in Krisp is ElevenLabs, and it's explicit and capped per day.
Open means inspectable. I only found out that GrooVAE returns unquantized start times, and that the VAE leaves velocity at 0, because the checkpoints and the library are open. A closed API would have handed back a groove I couldn't explain.
Once loaded, it is always there. The cold cost is real: the drums VAE loaded in 14.2 s and the first Humanize took 15 s, both on a GPU-less machine. Both happen once. [[MEASURE: warm Humanize time on Anuv's laptop]]

What this unlocks

The same pattern (a pure scheduler, one engine for live and offline, open models in the tab, one Postgres for meaning and spelling) works for any instrument, not just drums. A melodic step sequencer, a sample chopper or a practice metronome with real feel is a new rack, not a new architecture.

What Didn't Work

Hazards, with the errors verbatim

What happened	The error	The fix
Postgres refused my search column	`generation expression is not immutable`	`array_to_string` isn't immutable, so `tsv` became a plain column that each INSERT fills
A new Tiger service failed TLS for over an hour	`SELF_SIGNED_CERT_IN_CHAIN`	The cert came from Tiger's own CA, `ca.timescale.com`. I trusted that CA with verification on, instead of `rejectUnauthorized: false`
Disabling the limiter crashed the mixer	`RangeError: Value must be within [-100, 0], got: 1e-7`	Tone ramps start from 1e-7 when a param is 0, so dynamics params are now set directly, not ramped
Solo didn't silence the other channels	no error: the kick still read −32 dB with the snare soloed	`Volume.mute` was undone by the next volume ramp. Mute is now its own gate stage
The limiter didn't limit	no error: on +6.4 dBFS, off +7.2 dBFS	WebAudio's compressor adds makeup gain and lets transients through; even a raw −12 dB / 20:1 left a 0 dB sine at −4.2 dBFS. A soft clipper now holds peaks at exactly −6.0 dBFS
TypeScript gave up on transformers.js	`Expression produces a union type that is too complex to represent.`	A typed wrapper around `pipeline()`
Vercel blocked deploys after a teammate's merge	status Blocked	Hobby only deploys private-repo commits authored by the owner. An owner-authored commit unblocked it
Saving felt slow in local dev	~3 s per request	2.16 s TLS connect + 0.26 s per query, India → US-East, new connection per request. Production sits next to the DB and keeps its pool

What's real and what's simplified

Real	Simplified, on purpose
Magenta models running in the browser	The default kit is synthesised; your own sounds replace it
ElevenLabs generation, server-side	No time-stretching. A warning dot marks loops that don't fit the bar grid
pgvector + full-text hybrid search	A shared studio passcode instead of accounts; it's built for one person
The WAV export is the playback engine	4/4 only, 16 or 32 steps
146 unit tests + browser end-to-end checks	A daily generation cap to protect credits

Prize Categories

Best Use of Entire: the repository lives on Entire; every agent session is captured, imported from before setup, and linked to the
Best Use of Tiger Data: one Postgres holding JSONB workspaces, bytea audio, pgvector meaning and full-text spelling, fused in a single SQL statement with visible match reasons. commits it produced.
Best Use of ElevenLabs: generated one-shots and seamless loops inside the workflow, with the exact prompt shown and a daily cap.

Team

TEAM: Subarna Maity (@dronzer2code) · Sourish Panda (@t-rexbytes)

References

Magenta.js — https://github.com/magenta/magenta-js · Groove MIDI Dataset — https://magenta.tensorflow.org/datasets/groove
ElevenLabs Sound Effects API — https://elevenlabs.io/docs/api-reference/text-to-sound-effects/convert
pgvector — https://github.com/pgvector/pgvector · Tiger Data — https://www.tigerdata.com
Entire — https://docs.entire.io

Built within the challenge window by Subarna Maity and Sourish Panda with AI pair programming, captured on Entire. All preset beat patterns are original to this project. The sounds used in the search measurements were synthetic test tones, generated, uploaded and deleted by the test script. Anuv's name and words appear with his permission.

The System Never Checked If You Slept. Ours Reads Your Pulse First.

Subarna Maity — Fri, 25 Sep 2026 20:43:24 +0000

Let's Start With Hackathon Scene

Main video is made by the Organizers of Hacktropica this my edited version on it

Right now, while you read this, your face is changing colour.

Not by much. Every heartbeat pushes a pulse of blood into the capillaries under your skin. Haemoglobin soaks up green light a little more than the tissue around it, so your cheeks darken and fade, darken and fade, roughly once a second. The change is far too small for anyone looking at you to notice.

A phone camera notices it anyway.

You have never seen it happen. It has happened every second of your life.

Keep that in mind, because this story runs on three clocks, and they do not run at the same speed:

CLOCK 1  THE SCAN      10 seconds    a face, a front camera, a pulse
CLOCK 2  THE BUILD     36 hours      four people, one repo, one prize
CLOCK 3  THE READING   5 months      one line in the docs we never read

The first clock is the product. The second is how it got made. The third is the reason this post exists, and it does not show up until near the end. When it does, it changes what the first two mean.

The System never checked if you slept

In Solo Leveling, a mysterious System hands Sung Jinwoo the same daily quest every single day. 100 push-ups, 100 sit-ups, 100 squats, a 10 km run. Didn't sleep? The System doesn't care. Nearly died yesterday? The System doesn't care.

Every fitness app is that System. It asks how you feel, lets you tap a smiley face, and then serves the plan it was always going to serve.

But your face is already telling the camera how you feel. Heart rate and breathing rate are sitting right there in the colour of your skin.

So the question is not "what workout should I do today?" It is:

What if the System measured you before it assigned the quest?

That is SoloFit ("Arise. Train. Conquer."), a React Native app that reads heart rate from the phone camera, turns it into a readiness score, and only then asks an LLM to write your week. Our team, Kolkatar Rosogollas, built it in 36 hours at HackTropica 2K26, and it won MLH: Best Use of Presage SDK.

Here is the whole pipeline, with the real numbers from the code:

1 scan -> 2 vitals -> 1 score -> 3 bands -> 7 days -> 7 rules

One thing up front, because it changes how you read everything below

SoloFit has two ways of getting your vitals, and you need to know which is which before any number in this post means anything.

Measured. The video goes to Presage, comes back as heart-rate and breathing-rate samples, and the result is tagged source: "presage_api". The scan screen shows a green badge.
Estimated. If anything fails (no API key, no video file, upload error, a 100-second timeout), the service does not throw. It returns randomly generated vitals: heart rate 65-95, breathing 13-20, tagged mock_no_key, mock_no_video or mock_api_fallback. The scan screen shows an amber badge that says "Estimated Vitals."

The fallback was a deliberate hackathon decision. The code comment says so plainly: Fallback is expected in some environments; keep this non-error in UI/logs. A demo that crashes on bad venue Wi-Fi is not a demo.

What that fallback does downstream is a different story, and the third clock is where I tell it. For now, remember the amber badge.

And one line that applies to the whole post: Presage's own documentation says its metrics are for general wellness and informational purposes only, have not been cleared by the FDA, and may not be used for medical diagnosis or treatment. SoloFit is a fitness app. It is not a medical device, and nothing below is medical advice.

What I Built

CLOCK 1 · THE SCAN · 10 SECONDS

[Daily Quest] Look at the camera for ten seconds. Reward: a plan that knows how you slept.

Most fitness apps give you a plan built from your goals and your equipment, and the plan is the same on the morning after a full night's sleep as it is the morning after an all-nighter. The README puts it in one line I still like: "Most fitness apps ask the user how they feel. Solofit measures it."

The app runs on three beats:

Measure, don't assume. A 3-second countdown, then a 10-second front-camera recording. Presage reads your heart rate and breathing rate from that video.
Reason, don't template. Those vitals, plus your profile (age, goals, equipment, injuries, diet, allergies, budget, cuisine), become a readiness score and a prompt. Gemini 2.5 Flash writes a 7-day workout and diet plan.
Adapt, don't repeat. Scan again tomorrow and the plan is rebuilt from tomorrow's body, not yesterday's.

Every screenshot in this post is the real app, running on web through Playwright with no API keys. That matters later, so remember it.

The rest of the app keeps the Solo Leveling mechanics: quests with XP, a global leaderboard, a community feed, a posture screen that uses MediaPipe, and exercise GIFs so nobody has to guess what a "Bulgarian split squat" is. It doesn't keep the Solo Leveling look. It had it for part of the weekend, and why it lost it is the best story from the event. That's in Hackathon Experience.

The stack: React Native on Expo SDK 54, Firebase Auth and Firestore for profiles, the Presage Physiology REST API for vitals, Gemini 2.5 Flash for plans, Groq for fast meal swaps, MongoDB Atlas behind an Express server for the leaderboard and community, and Zustand for state.

Three engineering decisions worth stealing

1. Never trust an LLM's JSON. Validate it against hard rules, repair it, then fall back to code.
The LLM does not get the last word on your workout. A validator does (more on the seven rules below). If the plan breaks a rule, the exact violation goes back to the model in a repair prompt. If the repair still breaks a rule, a deterministic plan builder takes over. The user always gets a valid week.

2. Route each call to the model that fits its latency budget.
Generating a whole week can take a while, and nobody minds. Swapping one meal while you stand in the kitchen has to feel instant. So meal swaps go to Groq's llama-3.1-8b-instant in JSON mode at temperature 0.2, and only fall back to Gemini if that fails. Plan generation goes the other way: Gemini first (with a backup key), then Groq's openai/gpt-oss-120b if Gemini is down.

3. Tag every data point with where it came from.
Every vitals object carries a source string. That one field is what lets the UI tell measured data apart from estimated data. It is also what made the problem in the third clock easy to find.

Demo

ipsamurai / solofit

Solofit — Real-Time Adaptive AI Fitness Coach

A mobile fitness application that uses live biometric data to generate and continuously adapt personalised workout and diet plans — in real time, for every individual.

What is Solofit?

Solofit is a real-time adaptive AI fitness coach built as a React Native mobile application. It combines clinical-grade biometric analysis, large-language-model reasoning, and a social community layer to deliver hyper-personalised fitness and nutrition plans that update every time a user's physiological state changes.

Most fitness apps ask the user how they feel. Solofit measures it.

A 10-second face scan through the phone camera captures…

View on GitHub

Verify this in 60 seconds, no keys

The whole Presage pipeline in one function. presageService.js L223-L278. Read the try block top to bottom: read file, get upload URLs, upload chunks, complete, poll. The catch at the bottom is the fallback.
The readiness score. readinessEngine.js L23-L65. Start at 100, subtract. Then look at lines 62-65, which come up again in the third clock.
The LLM guardrails. geminiService.js L40-L128 for the validator, and L1491-L1603 for the generate, validate, repair, fall back chain.
Why the scan is 10 seconds. ScanScreen.js L36. One line, one comment. Remember it.

Partner Technologies

Presage (the prize)

What rPPG actually is. Remote photoplethysmography. A pulse oximeter clipped to your finger shines light through your skin and watches how much comes back as blood surges with each heartbeat. rPPG does the same thing with ambient light and a camera, from a distance. The colour change is tiny and buried in noise from movement, lighting and compression, so the hard part is not the idea. The hard part is pulling a clean signal out of a shaky phone video, and that is the part Presage does for you.

What we used, exactly. The prize is called "Best Use of Presage SDK." What SoloFit ships is the Presage Physiology REST API at https://api.physiology.presagetech.com, called from React Native with plain fetch, not the on-device native SDK. The header comment in our service says the flow was "reverse-engineered from official Python client v1.6.0." In other words, Saikat read Presage's Python client, worked out the HTTP calls it makes, and wrote them again in JavaScript, because there was no drop-in React Native package for what we needed. None of the rest of us could have built this part.

The flow has five steps:

Step	Call	What it does
1	`POST /v1/upload-url`	Send the file size and `hr_br: { to_process: true }`. Get back a job `id`, an `upload_id`, and a list of presigned S3 URLs.
2	`PUT` to each S3 URL	Upload the video in 5 MB chunks. Keep each chunk's `ETag`.
3	`POST /v1/complete`	Send the job id, upload id and the `{ ETag, PartNumber }` list. Processing starts in the cloud.
4	`POST /retrieve-data`	Poll every 2.5 s. `201` means still processing, `200` means results are ready. Give up after 40 tries (100 s).
5	Parse	Response looks like `{ hr: { "0.5": 72, "1.0": 74, ... }, rr: { ... } }`.

Here is the heart of it, trimmed from the source:

const { id, urls, upload_id: uploadId } = await requestUploadUrls(fileSize, apiKey);
const parts = await uploadChunks(fileBuffer, urls);
await completeUpload(id, uploadId, parts, apiKey);

for (let attempt = 0; attempt < MAX_POLL_ATTEMPTS; attempt++) {
  await new Promise((r) => setTimeout(r, POLL_INTERVAL_MS));
  const res = await fetch(`${BASE_URL}/retrieve-data`, {
    method: "POST",
    headers: { "x-api-key": apiKey, "Content-Type": "application/json" },
    body: JSON.stringify({ id }),
  });
  if (res.status === 200) return parseResults(await res.json());
  if (res.status === 201) continue; // still processing
}
throw new Error("Presage processing timed out");

In plain English: ask Presage where to put the video, put it there in pieces, tell Presage the pieces are all in, then knock on the door every two and a half seconds until the answer is ready or 100 seconds have passed.

And the parse step, which matters more than it looks:

const hrVals = Object.values(data.hr).filter((v) => typeof v === "number" && v > 0);
hr = Math.round(hrVals.reduce((a, b) => a + b, 0) / hrVals.length);
// ...same for rr...
return {
  heart_rate: hr ?? 72,
  breathing_rate: rr ?? 15,
  stress_level: inferStress(hr),
  source: "presage_api",
};

In plain English: take every positive sample in the time series, average them into one number, and label the result as live Presage data. Stress is not measured at all. It is inferred from heart rate alone: above 90 is high, above 72 is medium, anything else is low.

Hold on to three details from that block: every sample is averaged, nothing looks at confidence, and there is a ?? 15. All three come back later.

What Presage made easy. The server-side processing. We never touched a colour channel, a bandpass filter or a face tracker. Upload a video, get a heart-rate time series back. For a team with a 36-hour clock, that is the entire reason the feature exists.

What fought back. The upload. Presage's API has a 10 MB file limit, and a phone's front camera fills that quickly. The fix is in ScanScreen.js:

const RECORD_DURATION_S = 10; // Reduced to keep file under 10MB API limit

Plus quality: "480p", mute: true (rPPG does not need audio), and a maxFileSize cap of 9 MB. The in-code estimate is 2-4 MB for ten seconds, which fits in a single 5 MB part. The scan had been longer (a leftover comment in the same file still says recording (15s)). We cut it to ten to make the upload fit.

That trade looked free at 2 a.m. It is the most important line in this post, and the third clock explains why.

Gemini 2.5 Flash, and the guardrails around it

The prompt is heavy. Gemini gets the full user profile, the vitals JSON, the readiness score and band, the list of injuries to work around, the equipment, the allergens (marked STRICT), and the full exercise catalog with the instruction use ONLY these, EXACT names.

A prompt is a request, though, not a guarantee. So every plan goes through validateAndNormalizeWorkoutPlan(), which throws on the first violation:

#	Rule	Error it throws (verbatim)
1	Exactly 7 days	`workout_plan must contain exactly 7 days.`
2	2-4 exercises per day	`Day 3 must contain 2-4 exercises, got 5.`
3	Only exercises from the catalog	`Day 2, exercise 1 uses unknown exercise: "..."`
4	No duplicates within a day	`Day 4 has duplicate exercise "..."`
5	No exercise on consecutive days	`Day 5 repeats "..." from the previous day. Consecutive-day repeats are not allowed.`
6	At most 2 rest or recovery days	`workout_plan has too many recovery/rest days (3). Max allowed is 2.`
7	Enough variety across the week	`workout_plan has low variety: 3 unique exercises, need at least 4.`

Rule 7 is smarter than it looks. The variety floor is max(4, min(10, availableCount)), where availableCount is how many catalog exercises this user can actually do with their equipment and injuries. A bodyweight-only user with a bad knee cannot be asked for ten unique exercises. The comment in the code explains that without the cap, "every plan, including the deterministic fallback, [would] fail validation." Somebody found that out the hard way.

The chain around the validator:

Gemini (primary key -> backup key) -> Groq gpt-oss-120b if Gemini fails
  -> parse (with truncated-JSON recovery)
  -> validate
     -> fail: repair prompt containing the exact error -> parse -> validate
        -> fail: deterministic fallback workout -> validate
  -> tune for equipment + injuries -> validate again
  -> diet normalisation -> allergen check

That is one generation, one targeted repair, then code. Not a retry loop that hopes for the best.

The elimination test. Could Gemini be replaced by a template? Try this user: vegan, peanut allergy, resistance bands only, knee injury, high stress this morning. A template needs a branch for every combination of those. The LLM handles the combinatorics. The validator makes sure the combinatorics came back in a shape we can trust. An LLM without guardrails is just a template with extra steps. The guardrails are the engineering.

Groq

Meal swaps only, and only for speed. llama-3.1-8b-instant, temperature: 0.2, JSON mode with a retry without it, 550 max tokens, and Gemini as the fallback. One job, done fast.

MongoDB Atlas

The social half of the System. An Express server exposes GET /leaderboard, sorted { xp: -1, streak: -1 } (XP first, streak as the tiebreak), and GET/POST /community/posts, sorted newest first, with image upload. If Atlas can't be reached, the server swaps in an in-memory collection seeded with demo users, so the screen never goes blank on stage.

Why none of this is decoration

Feature	Where	Why it isn't decoration
Camera vitals	Presage REST API	Replace it with a "how do you feel?" slider and you've rebuilt every other fitness app. The scan is the thesis.
Plan generation	Gemini 2.5 Flash + Groq fallback	Combinations of constraints no template covers
Plan validation	`validateAndNormalizeWorkoutPlan`	Turns a probabilistic output into a guaranteed shape
Meal swap	Groq `llama-3.1-8b-instant`	Latency budget different from plan generation
Leaderboard + community	MongoDB Atlas + Express	Quests without a scoreboard are just chores

One scan, all the way down

Let's go down through the layers of a single scan. This is a worked example with plausible values, not a saved reading from the event. Every arithmetic step uses the real formula from readinessEngine.js.

Layer	Value	Where it comes from
Video	10 s, 480p, muted, ~3 MB	`ScanScreen.js`, in-code estimate 2-4 MB
Upload parts	1	3 MB fits in one 5 MB chunk
Job	`id` from `/v1/upload-url`	Presage
HR samples	a time series keyed by seconds	`/retrieve-data`, status `200`
Averaged HR	88 bpm	mean of every positive sample
Averaged RR	20 br/min	mean of every positive sample
Stress	`medium`	inferred: 88 > 72
Source	`presage_api`	green badge
Readiness	68	decomposed below
Band	`light workout`	45-74
Plan	7 days, lighter volume	Gemini, told `RECOMMENDATION: light workout`
Validation	passes 7/7	or repair, or fallback

Now the score by hand, for a 19-year-old with no declared health issues:

start                                             100
HR 88 > 80        excess 8, min(8 x 2, 30)        -16
HR > 0.7 x (220 - 19) = 140.7 ?   no                0
RR 20 > 18        (20 - 18) x 3                    -6
stress medium                                     -10
health issues     none                              0
                                                  ----
readiness                                          68   ->  "light workout"

The step to watch is the breathing term. Two breaths per minute of RR moved the score by 6 points. RR is also the number the third clock is about.

Hackathon Experience

CLOCK 2 · THE BUILD · 36 HOURS

HackTropica 2K26. Asansol Engineering College, West Bengal. April 4-5, 2026. Thirty-six hours.

The theme was Nature, and how beautiful it is. We built an app that stares at your face. That sounds like the opposite of the brief until you remember the first paragraph of this post: a pulse is the oldest rhythm in nature, and it's been running under your skin your whole life. We just pointed a camera at it.

That's the event, not the product. The app demo is up in the Demo section. This is what the 36 hours looked like from outside the laptop.

The crew

We called ourselves Kolkatar Rosogollas, after Kolkata's most famous syrup-soaked sweet, and then spent a weekend building an app that writes people diet plans. I'll let that sit.

Saikat Das built the Presage connection: the upload, the S3 chunks, the polling, the parsing. It was the one piece nobody else on the team could have done, and the whole app stands on it.
Me (Subarna Maity) built the UI of the Expo app, including the redesign you're about to read about, plus the posting features and a steady stream of bug fixes as everything else broke around him.
Soumyadeep Dey Community posting, the onboarding flow that collects everything the prompt needs (age, goals, equipment, injuries, diet type, allergies, budget, cuisine), and the diet planning feature. So finding #6 further down is in my part of the code.
Sriz Debnath helped where he could, and spent much of the rest of the 36 hours on dedicated fieldwork into the food and the swag. He reports that both were excellent. Every team needs one person who remembers there's an event going on outside the terminal.

A note on reading the repo: every commit is under Saikat's account, so the log tells you when things happened, not who did them. Treat it as the team's diary, not a scorecard.

The costume came off

The biggest challenge of the weekend wasn't rPPG, or S3, or getting JSON out of an LLM. It was a question.

We had started out building the app to look like the System itself: glowing blue quest windows, notification panels, the whole anime interface, as if the phone were Jinwoo's status screen. We loved it. It was the reason the name was SoloFit.

Then the judges walked up to our table, looked at it, and asked something like: if a serious fitness person opens this app, will they stay once they see an anime theme?

We didn't have a good answer, because there isn't one. Someone who trains every morning wants their numbers, not a costume. A theme that makes four hackers grin can make the person the app is actually for close it in five seconds. That hurt more than any stack trace that weekend, because it wasn't a bug. It was taste, and the judges were right.

So we tore it down in the middle of the hackathon and rebuilt it as minimalist neo-brutalism. Every colour is now in theme.js: a warm grey #E8E8E6 background, near-black #111111 text, one loud orange #FF5A1F for anything you're meant to press, hard 1.5-2px borders, no glow anywhere. Heart rate, breathing and stress each get a single flat colour of their own. It's quiet enough for a serious lifter and blunt enough to still feel like a System.

What we kept were the bones: quests, XP, the leaderboard, "Arise." What we threw away was the paint. That rebuild is where Version 2.0.0 : Massive UI overhaul in the log below comes from.

The anime was for us. The app was for them.

The diary

Here's the diary, in IST:

Time	Commit
Apr 4, 19:10	`Solofit: AI-powered adaptive fitness coach app`
Apr 4, 19:15	`chore: Secure API keys and move Firebase config to environment variables`
Apr 4, 19:18	`Updated gitignore`
	five and a half hours of silence
Apr 5, 00:54	`Presage implementation successful along with custom fallbacks`
Apr 5, 01:00	`feat: dynamic exercise GIFs with chevron dropdown in WorkoutScreen`
Apr 5, 03:05	`Harden env safety and stabilize AI workout generation`
Apr 5, 03:23	`Version 2.0.0 : Massive UI overhaul`
Apr 5, 06:20	`3.0.0 : Leader board and Community sections live using Mongo DB`
Apr 5, 10:10	`2.6.0 : UI changes and AI model fallbacks added`
Apr 5, 12:13	`version 3.0.1 : Minor changes`

Look at the gap. Between 19:18 and 00:54 there isn't a single commit. Those were Saikat's five and a half hours, and the commit that ends them is the one the whole project depends on. Our Devfolio "challenges" field sums up those hours in one line: "Implementing presage SDK into React Native App but we nailed It." The Devfolio stack also lists C++ and CMake, which are not what you reach for when calling a REST API.

The rest of the log reads like a team that has stopped being scared. Six minutes after Presage lands, someone ships exercise GIFs. By 03:23 the costume is off and the neo-brutalist UI is in. By 06:20 the leaderboard is live on Mongo. At 10:10 the version number goes backwards, from 3.0.0 to 2.6.0, in a commit that adds AI model fallbacks. That tells you exactly how much sleep was involved.

Look at the 00:54 commit message again, too. Not "Presage implementation successful." It's "Presage implementation successful along with custom fallbacks." The fallback was born in the same minute as the feature. Hold on to that.

Then judging, and the prize: MLH: Best Use of Presage SDK. We also applied to the MongoDB and Gemini tracks. We didn't win those. We won the one the whole app was built around.

[Warning] What I'd fix before anyone trains on this

CLOCK 3 · THE READING · 5 MONTHS LATER

[Warning] Vitals estimated. Intense quests should be locked. They aren't.

Five months later, sitting down to write this post, I opened Presage's current SmartSpectra docs to get one sentence right about breathing rate. This is the sentence:

Confidence is 0 until the full 30-second window is reached.

Our scan is 10 seconds long.

Go back through everything above with that in mind. The upload limit that pushed the scan down to ten seconds. The parse step that averages every sample and never checks confidence. The readiness formula where two breaths per minute cost six points. None of it was wrong at 2 a.m. It was just built without the one number that tells you whether the breathing data can be trusted at all.

These are findings, not apologies. Each one comes with its fix.

1. Breathing rate from a 10-second window. Presage's docs say breathing-rate confidence stays at 0 until 30 seconds. (Those docs describe the SDK. I haven't confirmed whether the REST endpoint behaves exactly the same.) Either way, a breathing rate from ten seconds of video is a number the provider itself says not to lean on, and our score leans on it.
Fix: On scans under 30 seconds, drop RR from the score completely. When breathing matters, scan for 30 seconds and keep the file under the limit by dropping to a lower resolution.

2. Averaging blindly. parseResults() averages every positive sample. The docs list what degrades the signal: large head, body or camera motion, chewing gum, flickering light like a TV screen. Pulse is valid for roughly 40-110 BPM.
Fix: If the API returns confidence with each sample, weight by it or throw away low-confidence samples. Either way, clamp HR to the valid range before it gets near a score.

3. A hard-coded breathing rate labelled as live data. Look at rr ?? 15 again. If Presage sends back heart rate but no usable breathing rate, the app puts in 15, and the object still says source: "presage_api". The green badge lights up for a number nobody measured.
Fix: Return breathing_rate: null and track source per field, not per object.

4. The fallback can unlock an intense workout. This is the big one. Here are lines 62-65 of readinessEngine.js:

// For estimated/mock vitals, keep readiness randomized strictly within 55-85.
if (isFallbackSource) {
  score = Math.floor(Math.random() * (85 - 55 + 1)) + 55;
}

In plain English: if the vitals were made up, make the score up too, somewhere between 55 and 85. But 75 and above maps to "intense workout". From 55 to 85 there are 31 possible scores, and 11 of them are 75 or higher. So on a fallback scan, there's roughly a 35% chance the app prescribes an intense session from vitals that were never measured.

I didn't have to imagine this. I ran into it while taking the screenshots for this post. Chromium's fake webcam produced no usable video file, so the app returned mock_no_video, and the random roll landed on 78:

The amber badge was there, just like it's supposed to be. So were a green 78, a "READY" pill and an upper-body hypertrophy session, all built on a heartbeat nobody measured. It gets worse downstream: the vitals JSON goes into the Gemini prompt under the heading REAL-TIME BODY DATA. The source field is inside that JSON, but nothing tells the model what mock_api_fallback means.

The fallback was a good decision for a demo. The mistake was letting it go further than the badge.

Fix:

if (isFallbackSource) {
  return {
    readiness_score: null,
    recommendation: "light workout",
    details: "Vitals were estimated, not measured. Intense quests stay locked until a real scan.",
  };
}

In plain English: when we don't know, say so, and pick the safe option. Estimated vitals can unlock a light day. They should never unlock a hard one. Tell the model the same thing in the prompt, and make the amber badge impossible to miss.

5. Two different definitions of "high stress". Real vitals call stress high above 90 bpm. The mock generator uses above 85. That's small, but it means the same heart rate gets a different stress label depending on where it came from.
Fix: One inferStress() function, used by both paths.

6. The allergen check flags instead of blocking. validateNoAllergens() does a substring match on meal names and, on a hit, renames the meal [ALLERGEN WARNING] .... It doesn't reject the plan, and it can't catch a peanut-based satay that never uses the word "peanut."
Fix: Send a flagged plan back through the same repair loop the workout validator uses, and match against ingredients, not just names.

The magic trick was real. The part we never checked was how long it needs to run.

What this unlocks

Solo Leveling was the costume, and the judges already made us take it off once. What was left underneath is one idea: measure the user before you personalise anything. Nearly every "personalised" app personalises from a profile you filled in once. A camera-based vital sign is a profile that updates every morning.

Study planners that check fatigue before scheduling the hard chapter.
Focus timers that stretch the break when your resting heart rate is up.
Rehab programmes that won't progress the load on a day your body says no.
Shift and driver apps that ask for thirty seconds before a long night, not a checkbox.

Every one of them gets the same three lessons from this post: tag the source, respect the window, and never let an estimate unlock the risky option.

FAQ

How do you measure heart rate from a phone camera in React Native?

Record a short front-camera video with expo-camera (recordAsync with quality: "480p" and mute: true), then send it to an rPPG service. With the Presage Physiology REST API: POST /v1/upload-url, PUT the chunks to the presigned S3 URLs, POST /v1/complete, then poll POST /retrieve-data until you get a 200. The response is a heart-rate time series you can summarise.

What is rPPG and how accurate is it?

Remote photoplethysmography reads the tiny colour changes in skin caused by blood flow. Presage's docs give a valid pulse range of about 40-110 BPM and list large motion, flickering light and chewing gum as things that degrade it. Its metrics are for general wellness, not FDA cleared, and not for diagnosis.

How long does a Presage scan need for breathing rate?

Presage's SmartSpectra docs say breathing-rate confidence is 0 until a full 30-second window, and HRV statistics need 60 seconds. A 10-second scan is fine for a pulse estimate. It isn't enough to trust breathing rate.

How do you stop an LLM from returning an invalid workout plan?

Don't rely on the prompt. Validate the parsed JSON against hard rules in code (for us: 7 days, 2-4 exercises per day, catalog-only names, no duplicates in a day, no consecutive-day repeats, at most 2 rest days, a variety floor). On failure, send the exact error back in a repair prompt. If the repair fails too, build the plan deterministically.

Why does my Presage poll keep returning 201?

201 from /retrieve-data means the job is still processing, not that it failed. Keep polling. We used a 2.5-second interval and gave up after 40 attempts (100 seconds), which is when Presage processing timed out fires and the fallback kicks in.

Credits

Team Kolkatar Rosogollas: Saikat Das (the Presage pipeline), Subarna Maity (the UI, posting and bug fixes), Soumyadeep Dey (posting, onboarding, diet planning), Sriz Debnath (morale, food reconnaissance, swag).
HackTropica 2K26 and Asansol Engineering College, for the 36 hours and a Nature theme we stretched as far as it would go.
The judges, for the question that killed our anime UI. It was the best feedback we got all weekend.
MLH, for the Presage track and the prize.
Presage, for an API that turned "read a pulse from a face" into five HTTP calls, and for docs honest enough to tell us where we went wrong.
Solo Leveling by Chugong, for the System we wanted to argue with.

SoloFit is a general-wellness fitness app. Its vitals are not medical measurements, and nothing in it or in this post replaces advice from a doctor. This post was drafted with AI assistance; the code, the build and the mistakes are ours.

Feels Great To be part of this project, its a awesome experince to build this project and completed it on time

Subarna Maity — Sun, 12 Jul 2026 18:18:31 +0000

DEV Weekend Challenge: Passion Edition Submission

Soumyadeep Dey

Jul 12

We Taught a Snowflake Warehouse to Judge World Cup Conviction and Write the Verdict Back to Solana

#devchallenge #weekendchallenge #snowflake #100daysofsolana

15 min read