Meena

Posted on Oct 19

FirstStep: Coding Practice Tool That Never Gives You the Answer

#devchallenge #hf26challenge #ai #opensource

Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle Submission 💸

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

When I started competitive programming, I would look at a problem, watch a tutorial, type the solution I had seen, and feel productive. A week later the same idea appeared with a small twist and I was blank. I had never practised the part that actually matters: deciding the first concrete move, then debugging against an input small enough to trace by hand.

LeetCode, HackerRank and the rest are excellent at giving you volume and a green checkmark. ChatGPT will happily write the full solution. None of them forces the starting move or hands you the tiniest counterexample instead of the answer. I wanted something a beginner (or a rusty intermediate) would actually open when they sit down to practise, not another place to collect green checkmarks.

How it works

You pick a problem.
Before any code is accepted, you must write one sentence about how you would start: a tiny example by hand, a try-everything loop, or one thing you would keep track of. I will think about it and solve the problem is rejected. I would keep track of the best sum of a block that ends at the current number unlocks the editor.
You write code. If it is wrong, the tool returns a minimal failing input you can step through on paper, plus a short plain-language reason (all-negative numbers, single element, self-pair trap, and so on). It never gives you the solution.
At the end of a session, you see which failure shapes keep beating you.

Here is a real result from my demo. My solution clamped the running sum at zero, so it passed the positive samples. FirstStep answered:

Found a failing input (smallest I could reduce it to: 4 runs)
1
-5
Expected: -5      Your program: 0
Your program was run on numbers that are all below zero.
Check what it does when no choice is positive.

One number was enough to expose the bug.

Who would pay (the side-hustle angle)

Students and career-switchers already pay for LeetCode Premium, coaching, and course platforms as a small investment toward interviews or contests. FirstStep is narrower: it sells the habit of starting concrete and debugging against a hand-traceable input, plus a private report of the shapes that keep beating you.

The paid unlock today is the harder level 3 problems and a full learning report (week-by-week trend, the first steps you wrote, and a practice plan ranked by your recurring mistakes). The next natural products are more problems, pattern filters, and later, twisted variants that measure whether the skill actually transfers. None of that needs a large team or a big model bill. The expensive part is writing good problems and good twists, not inference.

Demo

Live: https://hacktoberfeste6c4p8.tech
Also on Render: https://firststep-jtuf.onrender.com
Health: https://hacktoberfeste6c4p8.tech/api/health

Free tier works fully. To see the paid problems and full report, use the unlock code: JUDGE-DEMO-2026

What the video shows:

The first-step gate rejecting a vague sentence, then accepting a concrete one
Wrong code, and the smallest failing input found in a couple of seconds
The session report showing the failure profile

The home page: free problems to start, and the level 3 paid pack locked.

The paid plan and unlock box:

Code

https://github.com/meenanoolu/firststep

meenanoolu / firststep

Practice tool that forces a concrete first step before coding, then returns the smallest failing input you can trace by hand. Open-weight optional, rules always work.

FirstStep

A practice tool for people who keep solving the same coding problems by memory and then freeze on anything new.

Live demo: https://hacktoberfeste6c4p8.tech
Also on Render: https://firststep-jtuf.onrender.com
Repo: https://github.com/meenanoolu/firststep

Before you write any code you must state one concrete first move.
If your code is wrong, the tool returns a tiny input you can trace by hand that breaks it, plus a short plain-language reason.
It does not dump a full solution.

Why this exists

Many beginners (and some of us who already know the patterns) pass samples by recognition, then blank when the same idea appears in a slightly different shape.

FirstStep forces the starting move and shows the smallest counterexample instead of a green checkmark or a full answer. The goal is a habit: start with something concrete, then debug against a hand-traceable input.

What it does

First-step gate
You write one sentence about how you would…

View on GitHub

Key files:

app/firststep.py - first-step classifier (rules + optional model)
app/counterexample.py - oracle + shrinker for the minimal failing input
app/runner.py - sandboxed execution
app/pro.py - paid unlock + full learning report
tinker_ft/ - fine-tune script and measured results
render.yaml - Render Blueprint

How I Built It

FastAPI + SQLite for sessions, failure profiles and spaced reviews
AST-limited Python runner with time and memory caps
Brute-force oracle + budgeted search + shrinker for the minimal counterexample
Optional Ollama / OpenAI-compatible client with leak checks
Single static front-end, Docker, Render Blueprint (render.yaml)
Fine-tune path with Tinker (tinker_ft/)
Tests for the classifier, finder, limits and API

The model path is optional. When it is on, a small open-weight model helps classify the first-step sentence and rephrase explanations. When it is off, or the model is wrong, deterministic rules take over. The tool never relies on the model being available.

Partner technology

Render

FirstStep’s public product runs on Render as a Docker web service defined in render.yaml (Blueprint).

Auto-deploy from main
Health check: GET /api/health
Persistent disk at /data for SQLite (DB_PATH=/data/firststep.db) so sessions, failure profiles, and reports survive restarts
Env-based config: LLM_PROVIDER=none on the public demo (fast, free, no model bill), rate limits, and demo unlock codes
Custom domain hacktoberfeste6c4p8.tech pointed at this service

Render is the production front of the product: the sandbox, the counterexample finder, and the learning store all run there. Local Ollama/Gemma is optional on a developer machine; the hosted path stays rules-only so the side hustle demo stays cheap and reliable.

Gemma

I ran gemma2:2b locally through Ollama for three jobs in the product path:

Judging borderline first-step sentences (viable vs too vague)
Rephrasing graded hints
Explaining a counterexample in plain language

Every model reply is leak-checked before the learner sees it, so Gemma cannot hand over a full solution. If Gemma is offline or wrong, deterministic rules take over. The public Render demo does not call Gemma (it uses LLM_PROVIDER=none for speed and zero cost). Same codebase switches to Gemma with one env change on a machine that has Ollama.

Tinker

I fine-tuned a small open model on my own labelled first-step data and measured a clear gain over the base model (table below). Production still defaults to rules; the tuned adapter is an optional upgrade, not a hard dependency.

.tech domain

The live product is served on the free first-year domain claimed through Hacktoberfest:

https://hacktoberfeste6c4p8.tech

DNS CNAME points at the Render service; HTTPS is active.

Fine-tuning the first-step classifier with Tinker

FirstStep evaluates coding learners by sorting their initial statements into five precise categories: restate, small_example, brute_force, observation, and too_vague. General foundation models frequently misinterpret these strict pedagogical boundaries. To bridge this gap, we used Tinker to fine-tune Qwen/Qwen3-8B via LoRA.

Iterative Experiments & Results

Rather than presenting a single isolated model, we maintained an honest, multi-stage experimental log across three distinct runs:

Run 1 (Initial Pilot): 60 training rows / 20 test rows $\rightarrow$ 55% accuracy (proved feasibility and shaped dataset expansion).
Run 2 (Main Scaled Experiment): 250 training rows / 50 test rows $\rightarrow$ 70% accuracy (established our primary baseline).
Run 3 (Live-Verified Execution): Re-executed live via the Tinker SDK $\rightarrow$ 72% exact accuracy & 82% viable intent accuracy (confirmed full pipeline reproducibility and generated live prediction logs).

Evaluation & Baseline

When benchmarked against our production deterministic rules baseline (which scores 40% on the identical 50-item test split), the Tinker-tuned model delivers substantial improvements in intent classification accuracy without leaking full code solutions.

Real Predictions: Base vs. Tinker-Tuned Model

Problem Context	Learner's Input	True Label	Base Model Prediction	Tinker-Tuned Prediction
Two Sum	"I will try checking every pair using two nested loops."	`brute_force`	`too_vague` ❌	`brute_force` ✅
Reverse String	"For input 'abc', the output should be 'cba'."	`small_example`	`restate` ❌	`small_example` ✅
Valid Anagram	"I need to solve this problem using code."	`too_vague`	`observation` ❌	`too_vague` ✅

All scripts, datasets, metrics, and prediction logs are fully archived in the repository under tinker_ft/ and runs/.

Details and reproduce steps: tinker_ft/RESULTS.md.

Local run:

python -m venv .venv
# Windows: .\.venv\Scripts\Activate.ps1
source .venv/bin/activate
pip install -r requirements.txt
uvicorn app.main:app --reload

What is still limited

Only a handful of problems so far.
The "transfer score on twisted variants" idea is sketched, not productised.
One real user.
The local model path is slower on CPU; the public demo stays on the rule fallback for speed.

Why Does Open Innovation Matter?

No API bill on the critical path. The free tier and the Render demo run with LLM_PROVIDER=none. Rules plus the finder are enough, which keeps the cost of the product itself near zero.
Data stays on the machine when the model is local. First-step sentences and code never leave the laptop if you point the app at Ollama.
The model is a helper, not the product. Leak checks strip anything that looks like a full solution. If the model hallucinates or is offline, the experience degrades gracefully instead of breaking.
Swappable and tunable. The same code talks to Ollama or any OpenAI-compatible endpoint. I tested with gemma2:2b locally, and Tinker let me tune a small open model on my own labelled data. The production demo stays model-free so latency stays low.

A closed API would have made the first-step judge and explanations easier to prototype, but it would also have tied the free tier to someone else's pricing and uptime. For a tool aimed at students and self-taught beginners, that dependency is a real risk.

My First User

I put the live site in front of a friend who has been grinding the same kind of problems.

They said the first-step gate stops the reflex of "I already know this, I'll just write the solution," the exact habit that later leaves them blank on a new variant. They would pay if there were more problems and filters by pattern and difficulty. They also asked for a way to check an approach while still writing code. That is useful feedback, but I am treating it as a later feature, not the identity of the tool.

One person is not a market. It is, however, the first piece of evidence that the idea is useful to someone who already cares about this skill.