DEV Community: Kshitij

Japan turning footsteps into Electricity, I turned them into Music.

Kshitij — Mon, 12 Oct 2026 02:31:19 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

I built Footwork, a music experience where going for a walk becomes part of the creative process. Instead of asking you to stare at a screen, it turns the way you move - your pace, pauses, turns, and loops into a short instrumental called a Soundprint.

Using it is intentionally simple:

Tap Start walking.
Put your phone away and explore.
End the walk and listen to what your movement created.

Every walk has its own rhythm. You might slow down to cross a road, make a sudden turn, speed up along an open stretch, or loop back toward somewhere you have already been. Footwork turns moments like these into musical decisions, such as changes in energy, transitions between sections, and brief pauses.

Before any AI processing happens, Footwork removes the location data. It sends only an anonymous timeline of movement events to Gemma, Google's open-weight model. Gemma interprets the shape of the journey and creates a structured musical plan. After that plan is checked and validated, a separate renderer turns it into the final piece of music.

In other words, the walk itself becomes the prompt. You do not have to sit down afterward and explain your experience to an AI. The way you moved already tells the story.

Footwork is for walkers, runners, and anyone curious about hearing an ordinary journey in a completely different way. It is not meant to score your fitness, recommend a route, or give you another reason to keep checking your phone. It is simply a creative reward for putting the screen away and spending some time outside.

Demo

Try the public app: footwork-m6g0.onrender.com

Fast judge path

Open the app and choose Hear an example.
Press play on the Soundprint.
Watch the traced path, movement ribbon, and active moment move together.
Jump to the turn near 0:18 and the pause near 0:36 to inspect two different movement-to-music ideas.

The example is prepared in the browser so anyone can evaluate the complete interaction without spending generation credits. It is clearly labeled Example walk and is not presented as a live Gemma or ElevenLabs Music call.

For the real experience, start a walk, keep the tab open, put the phone away, and move for at least 90 seconds of active time. Ending the walk starts the privacy-cleaning, arrangement, and rendering pipeline.

The source is public on GitHub: github.com/kernelKain/footwork

Architecture notes: Docs/architecture.md

flowchart LR
    A[Walk captured in browser] --> B[API cleans the trace]
    B --> C[Detect turn, pause, pace change, return]
    C --> D[Remove coordinates and original clock]
    D --> E[Gemma plans the arrangement]
    E --> F[Validate constrained JSON plan]
    F --> G[Eleven Music renders audio]
    G --> H[Soundprint follows one audio clock]
    C --> I[Route Sketch fallback]

How I Built It

Footwork is built as a series of small, clearly defined steps. I deliberately avoided sending the entire walk to one large, mysterious AI process. Each part of the pipeline has one job, and the next step only begins when the previous one has produced a valid result.

1. Record the walk without demanding attention

The browser records the journey while the user is outside. The recording screen has large controls, pause and resume options, a visible 90-second readiness target, and a hold-to-end button that helps prevent accidental taps.

Once the walk starts, the user should be able to put the phone away and focus on being outside.

When the walk ends, the backend cleans the recorded trace and looks for a few meaningful types of movement:

sharp turns
pauses
changes in pace
returns near an earlier part of the route

These events become the raw material for the music.

2. Keep the movement, remove the location

Privacy was an important design decision from the beginning. Gemma does not need to know the name of a street or even which country the walk happened in to understand its rhythm.

Before the data is sent to any model provider, the server removes the coordinates and original timestamps. What remains is an anonymous movement timeline containing event IDs, event types, normalized timing, and the approximate duration of the finished piece.

This allows the AI to understand how the walk unfolded without receiving the location where it happened.

3. Ask Gemma to arrange the journey

I use google/gemma-4-E2B-it, running inside a public Hugging Face ZeroGPU Space, as Footwork's musical director.

Gemma is not asked to describe the route or invent a story about where the user went. Its job is much more focused: it studies the anonymous movement events and creates a structured instrumental arrangement.

For example, it can decide that a sharp turn should introduce a new section, a pause should create a musical hold, or a faster stretch should increase the energy. This is where the shape of the walk begins to become the shape of the music.

4. Check Gemma's plan before generating audio

Gemma must return its arrangement as a small, well-defined JSON plan. Footwork validates that plan before making a music-generation request.

The validator checks that Gemma only refers to events that actually occurred during the walk. It also rejects unsupported moods, unknown values, and styles outside Footwork's intentionally limited instrumental vocabulary.

Only a valid arrangement is allowed to continue to Eleven Music. This makes the pipeline more predictable and avoids wasting a music-generation request on a malformed plan.

5. Keep the visuals and music on the same clock

Once the Soundprint is ready, the audio player becomes the single source of time for the whole experience.

The traced path, movement ribbon, selected event, playback progress, and current Soundprint moment all follow the position of the same audio element. There is no separate animation timer running beside it, so the explanation does not slowly drift away from the music.

The hardest parts

Some of the most difficult work happened around the details that users should never have to think about:

Protecting privacy: I needed to preserve the shape and rhythm of a journey while removing the actual place where it happened.
Handling failure honestly: a demo track, a Route Sketch, and a generated Soundprint must always be clearly distinguishable.
Making retries safe: if a music request times out, Footwork cannot blindly send it again because the original request may already have consumed paid generation quota.
Keeping everything synchronized: the path, movement events, and explanations must remain aligned when the user plays, pauses, seeks, or leaves the tab.
Designing for outdoor use: important actions need large touch targets, strong contrast, minimal reading, and reduced-motion alternatives.

The result is a pipeline where each part can be inspected, tested, and replaced independently—while the experience itself remains simple: take a walk, put the phone away, and come back to a piece of music shaped by the journey.

Results

Footwork's result is more than an audio file. It is a synchronized explanation of how the walk became the piece:

Soundprint — the finished playable artifact, pairing the instrumental with the walk's detected moments so movement and music remain readable as one result.
Traced path — the cleaned route outline preserves the shape of the journey while model providers receive anonymized movement events instead of coordinates.
Movement ribbon — a compact, time-ordered view of turns, pauses, pace changes, and returns. During playback, it shows which movement event is shaping the current moment.
Journey pattern — the walk's larger structure—where motion changed, stopped, accelerated, or returned becomes the compositional plan rather than incidental activity data.
Synchronized playback — the traced path, movement ribbon, and active Soundprint moment all follow one audio clock, keeping the explanation aligned with what the listener hears.
Graceful fallback — if model planning or music generation fails, the same event timeline still produces a clearly labeled Route Sketch.

Gemma on Hugging Face

Gemma is the decision-making center of Footwork, not a decorative text generator.

Model: google/gemma-4-E2B-it
Space: kernelKain/footwork
Runtime: public Gradio Space on free Hugging Face ZeroGPU (zero-a10g)

The Space exposes one narrow /arrange job. It receives turns, pauses, pace changes, returns, normalized timing, and duration but never coordinates or the original clock. Thinking output is disabled, and the model is asked for one bounded JSON arrangement.

A checked call returned a validated plan from the pinned model:

{
  "status": "valid",
  "model_id": "google/gemma-4-E2B-it",
  "arrangement": {
    "mood": "warm_cinematic",
    "style": ["instrumental", "warm", "cinematic", "rising"],
    "event_refs": ["turn-1"]
  }
}

The response is accepted only when its mood is warm_cinematic, its style stays within the closed instrumental vocabulary, and every referenced event exists in the submitted timeline. Invalid or unavailable responses remain blocked and fall back to Route Sketch. The Space never invents a successful plan, and only a validated arrangement can be sent to Eleven Music.

This separation is the architecture: Gemma interprets the journey, Footwork validates that interpretation, and Eleven Music renders the accepted plan.

Why Does Open Innovation Matter?

Open innovation matters here because the model controls the part of Footwork that makes the project more than a GPS visualizer: how a journey becomes musical structure.

The planner is inspectable

I can show the exact model, pin its revision, inspect the prompt and response contract, and prove which movement event IDs were accepted. With a closed route-to-music service, the most important creative decision would be hidden behind an endpoint.

The privacy boundary belongs to the project

Footwork decides what the model is allowed to know. Gemma receives turn-1 at normalized time 0.31; it does not receive a street, a coordinate, or a wall-clock timestamp. Because the planning layer is open and separately hosted, that boundary is visible in the code and testable in isolation.

The model can move or change without rewriting the product

The planner speaks a small JSON contract. I can tighten the vocabulary, compare a compatible checkpoint, or move inference closer to the device while keeping the capture, validation, renderer, and Soundprint interface intact. Footwork does not run fully offline today, so I am not claiming that benefit yet but the open-weight core keeps that route available.

Open inference keeps experimentation accessible

The checkpoint runs on a free Hugging Face ZeroGPU Space. That made it possible to test a real, pinned model and expose the planning service publicly without building the experience around paid planning calls. Specialized audio rendering is used only after the open planner has made and passed the decision.

A closed end-to-end service could produce audio, but it would weaken the two ideas I care about most: an enforceable location boundary and a replaceable, inspectable relationship between movement and music.

While shaping the submission, I compared the approach with two other outdoor-first community entries: WanderInk by @subhamduary and Paper Trail by @synthreaper. Both reinforced the value of making the device secondary to the outside experience, while Footwork takes a different path: movement is not context for generated content; it is the score itself.

My Agent Session

The agent-assisted development sessions for Footwork are tracked with Entire:

View the Footwork agent sessions on Entire

Together with the repository, these sessions provide a reviewable trail of the agent-assisted work behind the build.

Prize Categories

Best Use of Gemma — Gemma is the musical director. It converts an anonymous movement timeline into the constrained arrangement that defines the generated piece; without that plan, the studio path does not proceed.
Best Use of Render — Render hosts the public experience
Best Use of ElevenLabs — Eleven Music renders the validated instrumental plan into the final audio
Best Use of Entire — Provide a reviewable trail of the agent-assisted decisions, implementation steps, and verification work behind the project.
Best Use of Copilot — For debugging purposes after code review by Code Rabbit.

Arun plans every evening like a research paper. So I built him a planner that is allowed to say it doesn't know.

Kshitij — Mon, 05 Oct 2026 04:21:23 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

Arun is the friend who plans. I am the friend who says "anything is fine" and then has opinions later.

He has a system. Ten Google Maps tabs. A screenshot of opening hours that might be from last year. A chatbot paragraph that reads finished. Last month he asked one for drinks in Lisbon, for two, around nine. It came back in seconds: three bars, a note about the lighting, a walking time nobody had measured. One of those bars was shut that weekday. Arun still went, because the answer had sounded sure of itself.

What got me wasn't that the answer was wrong. It was that there was no way to tell which parts anyone had actually checked.

So I spent the weekend building him a planner that is allowed to say "I don't know."

Happen takes one evening, written the way you'd text a friend, and returns one plan where every claim is either sourced or visibly blank.

You type something like:

Drinks in Lisbon on Friday at 9 for two.

Then three things happen.

Describe. You write the evening in your own words.
Review. Happen turns it into a short brief: place, date, time, stop order, group size. If something essential is missing or ambiguous, it asks exactly one question. Every field is editable before anything gets searched.
Plan. You press Check live places. Happen searches live listings, checks the opening hours for that day against your time, and picks at most two stops.

The landing. One box, three starter evenings, and three promises: live listings, feasibility first, no hidden guesses.

That middle step matters more than I expected, and the screenshot below is the reason I trust it.

Review, mid-build. The brief on the left had resolved "at 9" to 9:00 AM and "Friday" to Wednesday. Nothing had been searched yet, so the mistake cost nothing.

I'm showing that screenshot because it caught a real bug, not because it flatters the product. That build read "at 9" as 9 in the morning and "Friday" as the coming Wednesday, then carried both forward without complaint. A planner built on "never claim what you didn't check" was quietly inventing two of its own fields.

So the parser now treats a bare time as ambiguous and stops to ask, rather than picking one:

What time should the evening start? — 09:00 or 21:00

Nothing is billed until Arun answers. That one fix is most of the difference between a confident plan and a true one.

Each stop then shows its time, why it was chosen, the listing details, and one line most planners never show you:

Couldn't confirm.

If the listing doesn't say whether a place seats two, Happen doesn't guess. If that day's hours are missing, the stop doesn't get the benefit of the doubt. Missing evidence never counts in a place's favour.

A finished plan. One stop, open at 9, live listings retrieved at 4:30 AM GMT+1. Couldn't confirm: price, current crowd level, or seating for two.

Note the date in that screenshot is the Wednesday, carried over from the same mid-build parse. The lookups, the hours check, the evidence, and the blank line are all real; only the date was wrong, and only for that one run.

View sources opens what was checked, the planned day's hours, the links, and a full audit with the time each fact was retrieved. If Arun wants to know why Ágave made the cut, the answer is a link rather than a paragraph about the ambience.

And if the plan isn't right, he types the change and chooses Review this change. The current plan stays on screen until he presses Apply.

What it deliberately doesn't do

I cut a lot on purpose, and I'd rather say so than let a screenshot imply otherwise:

One evening, two stops at most.
No travel time. A second stop shows its own hours and a directions link, not a made-up arrival.
No live crowd levels, reservations, or price conversion. Those three are exactly what this Lisbon plan refused to invent.
No accounts, no saved history, no database. Close the tab and it's gone.

Handing it over

I sent Arun the live URL and asked him to plan the Lisbon evening himself. He typed the prompt, answered the 9 PM question, changed nothing else, and pressed Check live places.

He read the Ágave card twice. The thing he pointed at first wasn't the five-star rating. It was the last line.

"Okay. So it's open at 9, and it's not pretending to know the price. That's actually useful. I keep getting paragraphs. I wanted the gap."

He got the distinction without me explaining it. Why it fits is what the listings supported: hours covering 9 PM, an official site, a Maps rating. Couldn't confirm is what they didn't: price, how busy it is right now, seating for two. He said he'd still go, and that he'd text the bar about a table, which is the decision Happen is supposed to leave with him.

What confused him was there being only one stop. He expected a second place, because the landing page says up to two. I gave him the honest version: a second stop only appears if a second place also clears the hours check. Happen won't pad the evening to look complete. His response was that this belongs on the result screen, not just in my explanation. He was right.

Demo

Try it: happen-web.onrender.com

Type an evening, or tap one of the three starters. Drinks in Lisbon on Friday at 9 for two is the one in the screenshots.

No login. The first search spends live SerpApi credits; asking for the same evening again within 15 minutes doesn't.

Code

kernelKain / happen

Find the right place and time for your plans with recommendations backed by reviews, busyness data, and your preferences.

🌙 Happen

Evidence-First Evening Planning That Is Allowed To Say It Doesn't Know

Built for a Friend: Handcrafted for Arun, a friend who plans every evening the same way — ten browser tabs, a screenshot of opening hours that might be from last year, and a chatbot paragraph that reads finished. Last month one of those answers sent him to a bar that was shut that weekday, because the text had sounded sure of itself. So I built him a planner that is allowed to say "I don't know."

Challenge Categories Submitted:

🏆 Best Use of Gemma (Google open-weight model, run locally and gated in code)

🏆 Best Use of SerpApi (the only place-data provider, with a hard per-plan budget)

🏆 Best Use of Entire (agent sessions checkpointed throughout the build)

🏆 Best Use of Render (one-click blueprint, both services defined in render.yaml)

📌 The Problem

Every evening…

View on GitHub

MIT licensed. Python backend (FastAPI), React and TypeScript frontend.

How I Built It

Here's the path one evening takes:

"Drinks in Lisbon on Friday at 9 for two."
   │
   ▼
Brief  ── place, date, time, stop order, group size
   │       (one question if something essential is missing or ambiguous)
   ▼
SerpApi ── Google Maps listings, official sites, community text
   │       (hard cap: 8 billed requests per plan, 15-minute cache)
   ▼
Python checks ── open on that day at that time?
   │              does the listing support what was asked for?
   │              missing evidence = no credit
   ▼
One plan, at most two stops
   ├── why each stop was chosen
   ├── one "Couldn't confirm" line per stop
   └── sources + audit with retrieval times

SerpApi is the only data source. Every fact on screen traces back to a Google Maps listing, an official website, or community text that SerpApi fetched. The server counts billed requests and refuses the ninth one before it's sent, so a single plan can never cost more than eight. A retry that actually reaches the network stays counted; a cancelled one costs nothing. Identical searches are cached for 15 minutes.

Python makes the call, not a model. Scoring is plain code you can read. It checks feasibility (is this place open at 9 PM that day?) and evidence (does anything actually say this place fits?). Where two sources agree on hours, that's agreement, not a conflict. Where they disagree, you see it.

Every claim carries its receipt. Each fact has a source link and a retrieval time. The "Couldn't confirm" line is generated from what was missing, not picked from a template. On this plan it's price, crowd, and seating for two, because the listings never said those things.

The open model, and the result I didn't expect

This is the part I want to be straight about, because it's the most interesting thing that happened all weekend.

The original design had Gemma reading the messy text, the community reviews, and the user's prompt, then handing structured evidence to the Python scorer. I picked Gemma 3 270M because it runs on CPU on the same box as the API, with no GPU and no hosted inference. It runs locally through llama.cpp, from a public quantized build, pinned to an exact revision and verified by SHA-256 before it loads. Temperature 0, fixed seed, so the same input gives the same output.

Then I did the thing you can really only do with an open model: I measured it myself, on my own task, before letting it anywhere near the product.

I wrote a held-out set (30 review excerpts, 24 planning prompts) and set gates the model had to clear before any of its output could reach a user:

Gate	Required	Gemma 3 270M	Gemma 3 1B
Valid JSON on review extraction	95%	6.7% (2 of 30)	100%
Right dimension and polarity on reviews	80%	4.4%	67.8%
Valid JSON on planning prompts	95%	91.7%	100%
Essential planning fields correct	90%	66.7%	79.2%

The 270M model mostly couldn't produce valid JSON for my schema. So I tried to fix it the way open weights let you: 100 training examples, no text shared with the held-out set, and a short QLoRA adapter trained on a free Colab T4. It came back at 6.7% parse and 3.3% accuracy — worse than the untuned model. My rule was that an adapter ships only if it beats baseline by five points without adding invalid output, so it didn't ship. With about eight hours left, I asked my agent whether to pivot the whole project. It said no: the product was already built to refuse when evidence is thin, so a weak model makes it quieter, not wrong.

The 1B model was a real step up. About 1.3 GB of memory, under a second to load. But it still got review polarity wrong about a third of the time, and for a product whose entire promise is "we only tell you what we checked," a model that's wrong a third of the time about whether a review is positive is worse than no model.

So it ships gated off, and the gate is code rather than a promise. There's a checked-in quality report (ml/reports/model-quality.json). At startup the backend hashes the model file and compares it against the hash in that report; model-derived claims are only permitted if the report for that exact file says it passed. Right now it says it didn't, so a deterministic parser reads the prompt instead, and the health endpoint openly reports model_claims_enabled: false.

I wanted to write "Gemma powers Happen." Better headline. But building a tool about not overclaiming and then overclaiming in the write-up felt wrong. What I can say is that the open model shaped the design: it's the reason there's a schema, a validator, a held-out set, and a gate at all. The moment a Gemma build clears those gates, turning it on is a config change.

How I worked

I built this over the weekend in Cursor, with Grok 4.7 High as my coding agent. I wrote the product contract first — what it does, what it must never claim, what the request budget is — and checked every change against it.

I used Entire to checkpoint the agent sessions into the git history, so the reasoning sits next to the code it explains. The public log is here: entire.io/gh/kernelKain/happen/sessions.

The weekend on Entire. 31 checkpoints across 62 commits, 8 PRs merged, about 5.7 million tokens. The spike is October 4th, when the live planner and the evidence-first UI landed.

What the test suite covers right now:

466 backend tests, all on mocks and captured fixtures. They never spend a SerpApi credit.
68 frontend unit tests.
22 Playwright tests at 1280px and 390px, with automated accessibility checks (axe) on each one.
A secret scanner across the whole tree, plus a check of the production bundle for keys, hosts, and test fixtures.

One bug I'm quietly pleased about catching: a typed follow-up answer was occasionally cleared before it was sent. It failed roughly 1 run in 28, and never when the test ran alone on an idle machine. I only found it by running the suite dozens of times and tallying failures. After the fix it passed 40 runs in a row, and the test now asserts the typed answer actually arrives in the request body, so it can't creep back in.

Why Does Open Innovation Matter?

Because I could check the model instead of trusting it. With open weights I downloaded the exact file, pinned its hash, ran it on my own laptop, and scored it against my own task. A closed API would have given me a good demo and a changelog. It wouldn't have let me prove, file by file, that the thing running in production is the thing I tested — and it certainly wouldn't have told me this precisely that the model wasn't ready.

Because "no" was an option. With a hosted model, the model is the product, and if it isn't good enough you're stuck. Here it's one swappable piece behind a gate. When the 270M failed I tried the 1B by changing a few lines of config. When a larger Gemma clears the bar, it's the same change. No vendor, no new contract, no new API key.

Because it costs nothing to run. The model shares a CPU with the API. There's no per-token bill, which matters for something I want to hand to a friend and leave running.

Because the evidence trail is open too. The scoring, the request cap, the quality report, and the gate are all in the repo. If you don't believe a plan, you can read exactly how it was made.

And because the open path was measurable in a way a hosted one wouldn't be. Gemma ran locally on a CPU, fine-tuned on a free Colab T4, and scored against gates I wrote myself. That's the whole reason the model section above exists at all.

What did open cost me? Quality, at this size. A large hosted model would probably have cleared my gates on day one. But I'd have had no way to keep checking it, and no way to know when it quietly changed underneath me.

My Agent Session

I made the product calls: two stops at most, missing evidence never helps a place, eight requests per plan, and shipping the model gated off rather than pretending otherwise. The agent wrote and tested most of the code against that contract.

The session I'd point a judge at is the one where the model failed. It walks through running the QLoRA notebook on a free Colab T4, getting 6.7% parse and 3.3% accuracy back, and then asking, with eight hours left, whether the project should be thrown away. The answer was to keep the refusal behaviour, stop tuning, and ship the deploy.

The full checkpoint trail is on Entire: entire.io/gh/kernelKain/happen/sessions.

Prize Categories

I'm entering four.

Best Use of Gemma. Gemma is the open-weight model at the centre of this build. I download a pinned public quantized build of Gemma 3 270M, verify its SHA-256 before load, run it locally through llama.cpp on CPU with no hosted inference, and QLoRA fine-tuned an adapter against it on a free Colab T4. Every one of those steps is only possible because the weights are open.
Best Use of SerpApi. SerpApi is the only data source in Happen. Google Maps listings, official websites, and community text all come through it, and every fact in a plan links back to the SerpApi result it came from, with a retrieval time. The server enforces a hard budget of eight billed requests per plan — the ninth is refused before it's sent — and caches identical searches for 15 minutes.
Best Use of Entire. Agent sessions were checkpointed with Entire throughout the build. The public log is here. This weekend: 31 checkpoints, 8 merged PRs, and the reasoning sitting next to the commits it produced.
Best Use of Render. The live app is happen-web.onrender.com. Frontend as a static site, backend as a Python web service in Singapore, both defined in render.yaml and deployed from main only after checks pass. The SerpApi key lives in the Render dashboard, never in the repo. The backend build downloads the pinned Gemma file and verifies its hash on Render, with no Hugging Face token and no hosted inference.

Built for Arun, who now has one fewer reason to open ten Maps tabs and still end up at the usual place.

Your most polite customers are the ones you’re about to lose.

Kshitij — Sun, 30 Aug 2026 17:43:23 +0000

Think of the last time you said an app was “fine.”

You still have it on your phone. You still open it when you’re tired. You would not recommend it to a friend. You also would not write to support. Support is for a missing order. “Fine” is quieter than that. “Fine” means you order less, then you stop.

This review is that moment:

“It’s not bad but I’m comparing every food order to getting downstairs 10 floors. The convenience tax is getting loud.”

— 3 stars

In plain English: skipping 10 floors of stairs used to be worth the extra fee. Now walking is starting to win. The person still has manners. They even left stars in the middle.

A normal sentiment tool reads “not bad,” sees 3 stars, and calls it okay. Maybe even positive. Green on the dashboard.

What happens next is not a complaint. It is fewer orders. Then the app comes off the phone.

That is the whole product: the review that looks okay and means goodbye.

What Resonance is

Resonance reads a sheet of reviews and tells you what people felt, not what they performed. It scores eight emotions. It notices when the words and the feelings disagree. It names the groups hiding in the 3- and 4-star pile. It writes a Hidden Ask — the need nobody filed as a ticket. Then it waits. It will not invent a product roadmap until you say yes.

Try it live: https://graphics-newsletters-usr-simplified.trycloudflare.com
Code: https://github.com/kernelKain/resonance
Watch a run: https://www.youtube.com/watch?v=QnLD8gl54OU

Two minutes. Then keep reading — the rest is what the clip doesn’t show.

kernelKain / resonance

Resonance

Customer emotion archaeology powered by Plutchik's Wheel

Resonance does not classify reviews as positive, negative, or neutral. It scores each review on Plutchik's eight emotions, flags cognitive dissonance, maps an unmet Maslow need, clusters the emotion vectors in a TrueForge sandbox, names psychological archetypes, writes Hidden Asks — then pauses for a human before any product-roadmap recommendation is emitted.

Built for The Agent Harness Hackathon on the TrueForge harness (filesystem MCP, Exa web search, Daytona sandbox, ask_user_question).

Live demo: https://graphics-newsletters-usr-simplified.trycloudflare.com

Always-on Oracle Always Free ARM VM in Mumbai. Nothing runs on a laptop. The URL is a Cloudflare Quick Tunnel (*.trycloudflare.com); it can change if that tunnel process restarts.

Screenshots

Zomato demo run (Plutchik profile → HITL approval → roadmap).

Upload a CSV → watch the Plutchik wheel fill in real time → approve the analysis → export a dark-theme PDF.

Sample analysis

Product	Dataset	Report

…

View on GitHub

Built for The Agent Harness Hackathon (WeMakeDevs × TrueForge × Qodo). The live URL is a Cloudflare tunnel on a small always-on VM. If that process restarts, the hostname can change. The repo and the video stay.

Drop a CSV. Watch the wheel fill. Approve or decline. Download a PDF.

The job I gave the agent

A product manager already has a spreadsheet. Their dashboard says “mostly positive.” They need the people hiding in the polite ratings.

The job, in order:

Read the file. Don’t make me paste reviews into a chat.
Learn what the product is, so “I love it” on food is not “I love it” on a bank.
Score every row: eight emotions, whether the words fight the feeling, one unmet need.
Group similar emotion patterns with real math — not a guess.
Name the groups. Write the Hidden Asks.
Ask me before any “you should build X.”

The eight emotions are Plutchik’s, not happy / sad / meh. Politeness is not joy.

This is the job: a shape of feeling, not a thumbs-up.

TrueForge: the harness in the middle

A chatbot answers. An agent has to reach a file, run code somewhere safe, and stop before it does something you can’t undo. I didn’t want to spend the week building that machinery. TrueForge is the layer between the model and everything it touches.

You (browser)  →  workbench  →  TrueForge agent
                                   ├ filesystem (the CSV)
                                   ├ web research (what is this product?)
                                   └ sandbox (the clustering script)

What the harness handled:

Files — the agent reads the upload by name and writes results. It doesn’t get the rest of the disk.
A specialist — one helper looks up the product. Scoring and the wait stay on the main agent.
A sandbox — grouping runs in a sealed machine, not on my laptop.
A pause — after Hidden Asks, TrueForge stops and asks. Roadmap items stay empty until Approve. Decline ends there.

I wrote the screen, the parser, and the PDF. The harness wrote the loop, the tools, the sandbox, and the wait.

TrueForge paused on ask_user_question — waiting for Approved or Decline.

The wait is not a fake button. The session actually stops.

What broke

Stage 1 — the model vanished. The first provider ran out of quota. I moved the agent to OpenRouter: one model first, a second if the first turn fails. A run stays on one model until you approve. Unsexy. Why the demo still works.

Stage 2 — the sealed room had no door. I told it to run the clustering script on the scored file. The sandbox had never seen that file. I had to copy the script and the data in. After that, it was just a tool.

Stage 3 — the answer got cut in half. Long replies died mid-stream. The UI said done. The wheel was empty. I stopped asking the model to “be complete” and started only drawing what I could actually read.

Stage 4 — the first model got tired. Same job, second model, more room. Not a second opinion.

Stage 5 — the small computer had opinions. The app had to listen for the real visitor IP behind the tunnel. The harness on that machine had to listen on all network interfaces or Docker couldn’t talk to it. The public URL is a tunnel. The only open door is SSH.

That’s the messy middle. That’s also why a stranger can click the link.

Qodo from the first PR

Every real change went through a pull request. Qodo read it before it landed. That was the way the repo grew, not a polish pass at the end.

Example: PR #4 — the approval step. The dangerous part of this product is emitting a roadmap. Qodo caught a High bug: if the pause data failed to parse, Approve would fake a local replay, mark the run done, and never send the answer back to TrueForge. The wait would look real. The harness would never hear you.

I stopped inventing a fake replay id. Approve only continues when the pause is a real TrueForge interrupt.

I don’t merge because the bot is green. I merge when the note is fixed, or the thread says why we skipped it.

All merged work: the PR list.

One Zomato run

I uploaded zomato_reviews.csv. Fifty reviews. No extra chat window — the file is the input.

First, the polite ones that don’t mean polite.

Same quote as the opening. Now you can see the tag.

Then the group that still opens the app.

They stay because discovery still works. One bad bill and they don’t.

Then the need that never became a ticket.

Not “make the bot nicer.” A person who can undo a wrong charge.

Then it waits. The roadmap side is blank on purpose.

Approve continues. Decline stops here. I tried both.

Only after Approve:

Same ask, now as something you could ship. That sentence did not exist before a human said yes.

Go try it

Open the live link at the top, or clone the repo, and hit Load demo dataset. Approve once. Decline once. The interesting screen is the one where the roadmap is still empty.

PawSpective: See the World Closer to How Your Dog Sees It

Kshitij — Mon, 17 Aug 2026 04:44:53 +0000

This is a submission for Weekend Challenge: Dog Days Edition.

Have you ever bought a bright red toy that looked impossible to miss only to watch your dog struggle to find it on green grass?

Humans and dogs do not experience color in the same way. That inspired me to build PawSpective, an application that combines a canine-vision approximation, AI-assisted scene understanding, deterministic visibility calculations, and playful narrated videos.

The central idea is one connected experience:

Capture a scene → explore a canine-vision approximation → discover which objects remain visible → turn the moment into a fictional story.

I also wanted PawSpective to be entertaining without pretending it can read a dog’s mind. It does not claim to determine exact canine vision, gaze, thoughts, feelings, smell, or intent.

What I Built

PawSpective helps dog owners explore how objects in an everyday scene may appear under a canine-inspired color transformation.

A user can:

Create a lightweight dog profile.
Open the Live Dog Lens.
compare Human View with a canine-vision approximation.
Record or upload a short video.
Let Google Gemini identify visible objects and scene events.
Review, rename, or remove incorrect AI detections.
Calculate the relative dog-visible contrast of an object.
Explore possible attention cues through the Curiosity Map.
Compare alternative toy colors in the Toy Color Lab.
Generate and download a fictional narrated Story Reel.

Live Dog Lens

The Live Dog Lens uses a browser-based WebGL transformation to provide an immediate Human/Dog Vision comparison.

The transformation preserves more blue and yellow differentiation while reducing red and green differentiation. A comparison slider makes the effect easy to understand, and a dog-height guide helps users frame the scene from a lower point of view.

This is deliberately described as a canine-vision approximation, not an exact reconstruction of what a particular dog sees.

AI scene analysis

After a user uploads a 5–15 second video, PawSpective sends a normalized, silent version of the clip to Google Gemini.

Gemini returns structured scene evidence, including:

Visible object labels
Approximate bounding boxes
Timestamps
Object categories
Confidence values
Visible evidence
Coarse motion levels

The response must pass a strict Pydantic and JSON Schema contract before the application accepts it.

Users can then remove incorrect events, rename objects, and select which object should be analyzed. These corrections become the source of truth for visibility calculations and Story Reel generation.

Visibility Lab

The Visibility Lab measures how strongly a selected object differs from its nearby background.

The backend:

Seeks to the event timestamp.
Samples pixels inside the object’s corrected bounding box.
Samples a surrounding background region.
Applies the same canine color transformation used by the frontend.
Converts colors into CIE Lab.
Measures foreground/background color and luminance separation.
Produces a relative dog-visible contrast score.

The result is presented as a product score and qualitative visibility band—not as a scientific probability.

For example:

Dog-visible contrast: High

The blue toy remains visually distinct from the surrounding grass after the canine-vision transformation.

Curiosity Map

The Curiosity Map highlights objects that may visually stand out in the scene.

It combines:

AI-inferred motion
Measured dog-visible contrast
Apparent object size
A small optional profile-relevance bonus

Every result explains why it appeared. PawSpective calls these possible attention cues, because the application does not track a dog’s gaze or know what the dog is actually paying attention to.

Toy Color Lab

Toy Color Lab compares six screen colors against the measured background around a selected object.

It keeps the surrounding background unchanged, simulates alternative colors for the object, applies the canine transformation, and ranks the colors by approximate dog-visible contrast.

The preview is illustrative—it does not claim that a physical toy will look exactly the same or guarantee a dog’s response—but it makes the red-versus-blue-on-grass problem immediately understandable.

Story Reel

The final step turns the reviewed scene into an 8–10 second vertical video.

Google Gemini uses only the user-reviewed scene timeline to generate grounded story and animation direction. ElevenLabs creates the fictional dog narration, and FFmpeg combines the video, narration, captions, overlays, music, watermark, and disclaimer into a downloadable 9:16 MP4.

The result remains clearly labeled Just for fun.

Demo

The application is available at:

👉 https://pawspective.onrender.com/

Demo note: [My Gemini API daily usage limit ran out, so I couldn’t publish the complete demo.]

The interface and deployed application can still be explored through the link above. The complete demonstration will be added to the repository once the provider usage limit is available again.

Code

The complete source code is available on GitHub:

kernelKain / pawSpective

See the world closer to how your dog sees it. PawSpective combines canine-vision simulation, AI-powered object visibility analysis, curiosity mapping, and playful narrated story reels.

PawSpective

See the world closer to how your dog sees it. PawSpective combines a canine-vision approximation, reviewed AI scene analysis, deterministic visibility scoring, color comparison, curiosity mapping, and downloadable fictional Story Reels.

What it does

Opens a live Human/Dog Vision comparison or accepts a 5-15 second video.
Detects visible objects with Gemini and lets the user correct the result.
Measures foreground/background contrast with OpenCV and CIE Lab color.
Shows possible attention cues in a timestamp-aligned Curiosity Map.
Compares six screen colors in Toy Color Lab.
Creates an 8-10 second animated dog-height POV reel with fictional narration.
Supports a SHA-256-bound controlled demo for offline rehearsals.

PawSpective labels deterministic calculations as Research-grounded model interpretation as AI-inferred, and fictional output as Just for fun. It does not claim exact canine vision, gaze, thoughts, emotions, smell, intent, or behavioral diagnosis. See the AI disclosure for data flow, limitations, and provider details.

…

View on GitHub

Repository: https://github.com/kernelKain/pawSpective

The repository also contains the product contract, AI disclosure, deployment guide, exported schemas, automated tests, Docker configuration, and release checklist.

How I Built It

PawSpective uses a Next.js frontend and a Python media-processing backend.

Frontend

The frontend uses:

Next.js 16
React 19
TypeScript
WebGL for the live color transformation
Canvas and video APIs for previews and overlays
Vitest and Testing Library
Playwright for end-to-end smoke testing

The browser handles the live Dog Lens, comparison slider, dog profile, recording and upload flow, object-correction interface, visibility results, Toy Color Lab, Curiosity Map, job polling, and Story Reel download.

Backend

The backend uses:

FastAPI
Pydantic
Google Gemini
OpenCV
NumPy
ElevenLabs
FFmpeg and FFprobe
SQLite

FastAPI validates uploaded media, normalizes videos, coordinates AI requests, performs deterministic calculations, and manages background Story Reel jobs.

SQLite stores job metadata and progress, while generated media is stored on a writable backend volume. Story rendering runs through a bounded background worker so the frontend can submit a job, poll its progress, and download the completed reel.

Google Gemini

Gemini is used for three related tasks:

Scene analysis

Gemini examines the short clip and returns visible objects, approximate bounding boxes, timestamps, motion levels, and supporting visual evidence.
Grounded story generation

Gemini receives the reviewed event timeline instead of being allowed to invent details directly from the raw video.
Animation direction

Depending on the configured model and access, Gemini Omni or Veo can create a canine-vision-inspired artistic edit using the source clip or reference frames.

One of my most important engineering decisions was requiring structured model output. Gemini responses are validated against strict schemas, and malformed coordinates, unsupported fields, impossible timestamps, or ungrounded results are rejected.

ElevenLabs

ElevenLabs generates the complete fictional dog narration for the Story Reel.

Only the finalized narration text is sent to ElevenLabs—the original video is not. The selected voice is intentionally described as a fictional dog voice, rather than an attempt to recreate a real animal’s voice or internal thoughts.

The application also supports narration failure handling and cached demo output so a temporary voice-service problem does not break the entire user journey.

Deterministic computer vision

I did not want an AI confidence value to become a fake scientific visibility score.

Instead, the Visibility Lab and Toy Color Lab use deterministic OpenCV and NumPy calculations. They sample pixels from the video, apply the documented color transformation, convert the colors to CIE Lab, and measure relative foreground/background contrast.

This creates a clear boundary between what the model inferred and what the application calculated.

Video composition

FFmpeg performs the final media work:

Video normalization
Portrait 9:16 composition
Captions
Narration synchronization
Curiosity overlays
Visibility result cards
Music
Watermarking
Final MP4 encoding

Using a fixed template made the rendering pipeline more reliable and kept the project focused on one polished experience instead of becoming a general-purpose video editor.

Reliability and responsible AI

PawSpective separates its output into three categories:

Research-grounded: deterministic transformations and calculations
AI-inferred: visible-object and motion interpretation from Gemini
Just for fun: fictional narration and story framing

The application includes recovery paths for camera denial, unsupported browsers, malformed AI output, dark footage, empty detections, narration failures, rendering errors, portrait and landscape uploads, and missing profile photos.

It also supports a controlled rehearsal mode. Cached analysis and media are bound to one exact video using a SHA-256 fingerprint, preventing saved bounding boxes from being incorrectly applied to an unrelated upload.

This separation became one of the most important lessons from the project: an AI product can be playful while still explaining where every result came from.

Prize Categories

Best Use of Google AI

Google Gemini powers PawSpective’s structured scene analysis, grounded story generation, and optional video-animation workflow.

Gemini is not used as an unrestricted narrator. Its output is schema-validated, reviewed by the user, and checked against visible scene evidence before it enters later stages of the application.

Best Use of ElevenLabs

ElevenLabs converts the grounded fictional script into narration for the downloadable Story Reel.

Voice generation is integrated into a complete media pipeline rather than presented as an isolated text-to-speech example. The narration is synchronized with captions, scene events, overlays, music, and the final vertical video.

What I Learned

The hardest part was not adding more AI. It was deciding which parts should not depend on AI.

Gemini is useful for interpreting a scene, but object visibility is better represented by deterministic calculations over measured pixels. Storytelling can be imaginative, but it must remain grounded in events the user has reviewed.

That led to the design principle behind PawSpective:

Use AI for interpretation and creativity, deterministic code for measurable calculations, and clear labels wherever uncertainty remains.

PawSpective started as a fun question about how dogs experience color. It became an experiment in building an AI-powered product that combines delight, practical insight, transparency, and responsible storytelling.

I Built a Tool to Detect Delayed Access Revocation

Kshitij — Sun, 09 Aug 2026 20:16:36 +0000

This weekend, I built TimeTrap.

It started with a simple question:

What if access is revoked, but something that depends on it stays active?

That small delay can become a real security problem.

🔗 Try TimeTrap live

💻 View the source code

The problem

Imagine a user cancels a premium subscription at minute 10.

The subscription is revoked immediately, but a cached permission remains active until minute 60. The system allows a five-minute grace period.

So the timeline looks like this:

Time	What happens
Minute 10	Subscription is revoked
Minute 15	Allowed grace period ends
Minute 60	Cached access finally expires

The user keeps premium access for 50 minutes after cancellation. Of those, 45 minutes violate the rule.

Each part of the system may look correct by itself. The subscription was revoked, and the cache eventually expired. The problem is the gap between those two events.

That is what TimeTrap finds.

What TimeTrap does

You describe the objects in your authorization flow—subscriptions, caches, sessions, or tokens—and add events such as:

issue
refresh
revoke
expire

Then you choose a rule, such as “cached access must end within five minutes of subscription revocation.”

TimeTrap checks the timeline and shows:

whether the scenario is safe;
the exact time the violation begins and ends;
how long stale access remains active;
the events that caused the problem;
a suggested correction.

For the example above, TimeTrap reports the violation as [15, 60) and suggests revoking the cached permission when the subscription is revoked.

You can see the complete flow here:

How it works

The analyzer is written in Go and is fully deterministic. The same input always produces the same result.

Instead of checking every minute, it checks only the moments where something can change: an issue, refresh, revocation, expiry, or rule deadline. It then joins failed periods into one clear violation interval.

The rest of the application uses:

React and TypeScript for the interface;
Go for the API and analyzer;
PostgreSQL for saved scenarios and results;
Zerops for deployment.

Saved results use their own URLs, so they can be refreshed or shared. Applying a fix creates a corrected copy instead of replacing the original unsafe result.

What I learned

Authorization is not only about who can access something. It is also about when that access should stop.

TimeTrap is currently a design-time tool. It checks the scenario you provide; it does not inspect or change a live production system. The model is intentionally small and uses integer-minute timing, but it makes this easy-to-miss problem visible.

OpenAI Codex helped me with planning, implementation, testing, documentation, and deployment. I still reviewed the code and verified the behavior through tests, builds, API checks, and browser testing.

I built TimeTrap for the WeMakeDevs challenge and deployed it on Zerops.

If this kind of authorization problem sounds familiar, try the demo and let me know what you think.

👉 Open TimeTrap

⭐ Explore the GitHub repository