DEV Community: Soumya ghosh

InkFlow: The Adaptive Writing Twin & Anti-AI-Slop Engine

Soumya ghosh — Sat, 17 Oct 2026 15:44:15 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

We are living through a time when AI-generated writing is becoming increasingly common, but much of it sounds remarkably similar.

When people use generic AI assistants to polish emails, academic papers, or technical reports, their original writing style can get replaced by repetitive buzzwords, predictable sentence structures, and a uniform corporate tone.

For students, researchers, professionals, and content creators, this creates a problem: their writing can lose the personal voice and natural rhythm that make it their own.

That's why I built InkFlow: The Adaptive Writing Twin & Anti-AI-Slop Engine.

InkFlow is an adaptive writing assistant designed to help people improve their writing while preserving their individual style.

It has four core capabilities:

Personal Style Genome: Uses a two-minute linguistic calibration exercise to build an eight-dimensional profile of a user's writing style, including formality, sentence-length variation, vocabulary diversity, clause structure, and punctuation habits.
AI-Slop Detection: Identifies common synthetic writing clichés and highlights phrases that make a draft sound generic.
Four-Stage Rewrite Pipeline: Generates or transforms text, removes unwanted stylistic patterns, checks semantic consistency, and performs a light grammar pass designed to preserve the author's punctuation preferences.
Adaptive Style Learning: Tracks writing changes across sessions and incorporates recurring stylistic preferences rather than overreacting to a single edit.

InkFlow is designed for university students, academics, non-native English professionals, content creators, and technical writers who want writing assistance without losing their own voice.

The Side Hustle: Monetization

InkFlow follows a freemium model.

Free Tier: Includes 1,500 words per day, one personal Style Genome profile, AI-slop highlighting, and access to the Humanize Studio.

Pro Tier — ₹39/month: Offers unlimited daily words, multiple writing profiles for different contexts, and priority access to higher-capacity open-weight models through an integrated Razorpay checkout supporting UPI, cards, and net banking.

The goal is to make personalized writing assistance affordable for students and independent creators.

Demo

Live Web Application: https://ink-flow-twin.vercel.app

Backend API: https://inkflow-backend-73hf.onrender.com

Demo Video:

hacktoberfest3rdchallenge.mp4 - Google Drive

drive.google.com

Here are some features you can explore:

Humanize Studio: Paste a draft or raw bullet points, identify common AI-writing clichés, and refine the text with character-level punctuation differences.
Two-Minute Voice Calibration: Complete four linguistic exercises to generate a personalized Style Genome.
Sentence Cadence Analytics: Inspect sentence-length variation across paragraphs.
Direct AI Writer: Draft emails and lecture notes, then refine them through the Humanize Studio.

Code

GitHub Repository: https://github.com/SoumyaGhosh2006/InkFlow

The project uses a modular full-stack architecture.

Backend: FastAPI, Python 3.11+, Motor for asynchronous MongoDB access, Groq inference, Pydantic v2, Razorpay integration, Sentry tracing, and Pytest.

Frontend: React 19, TypeScript, Vite, Tailwind CSS, Framer Motion, Zustand, Radix UI, Lucide icons, and Firebase Authentication.

The repository includes the implementation of the writing pipeline, style calibration, authentication, and subscription workflow.

How I Built It

1. Dual-Tier Open-Weight Inference

Rather than depending on a single model, InkFlow uses a primary model and a fallback model through Groq.

Primary model: openai/gpt-oss-120b handles complex stylistic transformations, nuanced rewriting, and Style Genome conditioning.
Fallback model: openai/gpt-oss-20b provides a lighter alternative when the primary model encounters timeouts, rate limits, or latency issues.

The backend uses FastAPI to coordinate requests from the React frontend and route them through the sequential rewriting pipeline.

2. The Eight-Dimensional Style Genome

A generic instruction such as "rewrite this in my tone" often produces an exaggerated imitation rather than a useful representation of someone's writing style.

InkFlow instead uses a short linguistic calibration exercise to capture different aspects of how a person writes.

The eight dimensions are:

Formality: The balance between academic language and conversational expression.
Syntactic Density: How frequently subordinate clauses appear in sentences.
Lexical Entropy: Vocabulary diversity and variation.
Burstiness Variance: Variation in sentence length, from short statements to longer, more complex sentences.
Punctuation Signature: The author's use of em-dashes, semicolons, commas, and other punctuation.
Clause Chaining: How the author connects related ideas.
Contraction Ratio: The frequency of forms such as "don't" and "it's."
Tone Affinity: Whether the writing tends to be assertive, exploratory, analytical, or pragmatic.

Together, these dimensions provide a more structured starting point for adapting the writing process to individual preferences.

3. The Four-Stage Sequential Rewrite Pipeline

InkFlow processes writing through four stages.

Stage 1 — Base Draft Generation: Expands raw bullet points or creates a first draft while preserving the original assertions.

Stage 2 — Style Transformation and Slop Removal: Applies the user's Style Genome and removes common synthetic clichés.

Stage 3 — Semantic Fidelity Check: Checks whether the transformed text retains the source's meaning and triggers a retry when significant semantic drift is detected.

Stage 4 — Quirk-Preserving Grammar Pass: Corrects errors while attempting to preserve characteristic punctuation choices and deliberate stylistic patterns.

The final result includes character-level differences so users can inspect the changes instead of blindly accepting a rewritten draft.

4. Engineering Challenges and Solutions

Challenge 1: Preserving Personal Punctuation

Conventional grammar correction can remove em-dashes, semicolons, and other punctuation habits that are part of someone's writing style.

InkFlow separates grammar correction into its own stage and uses deterministic checks to help protect those preferences.

Challenge 2: Preventing Semantic Drift

Aggressive cliché removal can accidentally remove important context, numbers, or factual claims.

InkFlow includes a semantic verification stage designed to detect meaningful divergence and retry when the rewritten text no longer adequately reflects the source.

Challenge 3: Improving the Mobile Writing Experience

A side-by-side editing layout can become difficult to use on smaller screens.

The mobile interface uses a segmented switcher between the source draft and the humanized result, a compact warning strip for detected phrases, and an automatic transition to the result after rewriting.

Challenge 4: Strengthening Authentication

During development, mock authentication tokens created the risk of visitors inheriting a test session instead of signing in with their own accounts.

The production implementation rejects mock tokens, removes frontend fallback tokens, uses Firebase browser-session persistence, and includes an inactivity-based logout policy. The application also allows users to explore drafting features while prompting them to sign in for protected operations.

5. Subscription and Reliability

InkFlow combines Firebase Authentication with a Razorpay-based subscription workflow. Its backend also includes Pydantic request validation, Sentry tracing, and automated tests.

The primary and fallback model arrangement is designed to improve resilience when the primary inference service experiences problems.

These components support the project's broader goal of making personalized writing assistance practical and affordable.

Why Does Open Innovation Matter?

Open innovation matters because writing is personal. People should have more control over how AI transforms their words instead of being forced into a single model's default writing style.

Greater control over writing style: Open-weight models give developers more flexibility to experiment with prompts, model selection, and style-conditioning techniques. This makes it possible to build a writing assistant around individual preferences rather than a generic corporate voice.

More flexible inference: InkFlow uses a primary and fallback model, helping the application respond to model availability and latency problems without relying entirely on one model configuration.

Affordable access: The project combines open-weight inference with a low-cost subscription model intended for students and independent creators.

A focus on authorship: The goal is not to replace a person's ideas with machine-generated prose. It is to help people express their existing ideas more clearly while retaining the characteristics that make their writing recognizable.

InkFlow is an exploration of how open-weight AI can support a more personalized writing experience.

My Agent Session

Architecting InkFlow: The Adaptive Writing Twin, Anti-AI-Slop Engine & Psycholinguistic Genome

You

Let's build InkFlow: an adaptive personal writing twin and anti-AI-slop assistant for university students and professionals. Commercial AI writing tools sanitize and homogenize text with robotic clichés ('delve into the tapestry', 'crucial testament'). We need: 1) A resilient 2-tier open-weight inference engine (Groq openai/gpt-oss-120b primary, 20b fallback), 2) A 4-stage sequential rewrite pipeline that purges slop but strictly preserves personal punctuation quirks (em-dashes, semicolons, conversational comma splices), 3) A 2-minute psycholinguistic Voice Calibration Drill (Pennebaker & King 1999 + Boston Cookie Theft), and 4) Daily/weekly word quota tracking.

Agent

I will architect InkFlow with a FastAPI backend and React 19 / TypeScript frontend adhering to a 4-Stage Sequential Rewrite Pipeline and an 8-Dimensional Psycholinguistic Style Genome extractor:
1. backend/services/providers.py: Configured resilient 2-tier Groq inference (openai/gpt-oss-120b primary with connect=2.5s / read=9.0s timeouts, auto-failover to openai/gpt-oss-20b within 800ms).
2. backend/services/pipeline.py: Implemented Stage 1 (Generate), Stage 2 (Tone & Slop Purge), Stage 3 (Semantic Meaning Check with reason-aware retry on >15% drift), and Stage 4 (Non-Destructive Light Grammar Pass preserving human punctuation quirks).
3. backend/services/grill.py & DrillCard.tsx: Engineered the 2-Minute Voice Calibration Drill mapping uninhibited writing to 8 psycholinguistic parameters (Formality, Syntactic Density, Lexical Entropy, Burstiness Variance, Punctuation Signature, Clause Chaining, Tone, Contractions).
4. backend/services/quota.py: Enforced lazy weekly/daily quota resets with 1,500 words/day on the Free tier.

▸ write_to_file backend/app/services/pipeline.py

Input

backend/app/services/pipeline.py

Output

Created 4-stage sequential transformation pipeline with reason-aware semantic retry and punctuation protection gates.

▸ write_to_file backend/app/services/providers.py

Input

backend/app/services/providers.py

Output

Implemented dual-tier Groq LPU provider failover (120B primary -> 20B fallback) with granular connection timeouts.

▸ run_command pytest tests -q

Input

$ pytest tests -q

Output

39 passed, 0 failures. Verified provider failover, 4-stage retry gate, punctuation preservation, and quota resets.

You

Integrate Razorpay Standard Web Checkout for the Pro subscription tier. We need: 1) Backend endpoint to create orders (POST /api/v1/payments/create-order) with HMAC-SHA256 signature verification (POST /api/v1/payments/verify), 2) Frontend checkout modal with UPI/Card/Netbanking support, and 3) Set the Pro plan to ₹39/month (democratized student pricing) instead of ₹499.

Agent

I have implemented Razorpay standard checkout for the ₹39/mo (3,900 paise) Pro subscription tier:
1. backend/app/api/v1/payments.py: Created POST /create-order and POST /verify endpoints with cryptographic HMAC-SHA256 signature validation.
2. frontend/src/components/payments/UpgradeModal.tsx: Integrated Razorpay Checkout SDK (checkout.js) supporting UPI, card, and netbanking, with confetti celebrations on upgrade and automatic tier elevation to 'pro'.
3. Set default order amount to 3900 paise (₹39) across backend services, frontend API, and test suites.

▸ replace_file_content backend/app/api/v1/payments.py

Input

backend/app/api/v1/payments.py

Output

Configured Razorpay order creation and signature verification with default amount 3900 paise (₹39).

▸ replace_file_content frontend/src/components/payments/UpgradeModal.tsx

Input

frontend/src/components/payments/UpgradeModal.tsx

Output

Updated Pro subscription UI and checkout payload to ₹39/mo.

You

When opening on a phone, the mobile view is severely broken. The textarea is cramped to 2 lines, output is pushed off-screen, and a wall of 9 slop chips plus docks cover the screen. Also, whenever I open on my phone or incognito, I am automatically logged in as 'Test User' with a dev token. Please fix the mobile layout so it's clean and easy to write, and fix the auto-login security bug.

Agent

Diagnosed and resolved both issues:
1. Mobile Layout Redesign (TransformationStudio.tsx): Replaced the squeezed dual-pane vertical stack on phones (< lg) with a mobile-segmented switcher: [ Source Draft ] vs. [ Humanized Result ]. When on Source Draft, the textarea gets the full phone viewport height. Replaced the 200px slop chip cloud with a compact 1-line amber alert banner. Tapping 'Humanize Draft' automatically slides the view to the Humanized Result tab with streaming diffs and action docks.
2. Auth Security Hardening (security.py & api.ts): Discovered the frontend was falling back to 'test-token' for unauthenticated users, which backend resolved to 'Test User'. Removed the fallback token and locked mock tokens strictly to DEBUG mode. In production, unauthenticated requests are properly treated as anonymous.

▸ replace_file_content frontend/src/components/studio/TransformationStudio.tsx

Input

frontend/src/components/studio/TransformationStudio.tsx

Output

Implemented responsive segmented view switcher (Source Draft vs Humanized Result) for mobile screens.

▸ replace_file_content backend/app/core/security.py

Input

backend/app/core/security.py

Output

Locked mock test tokens strictly to non-production environments with HTTP 401 enforcement.

You

Let users browse and use the tool without signing in (drafting, exploring tabs), but if they try to do ANY active operation like humanizing text or training the model, notify them with a sign-in modal, and block them until they sign in. After signing in with Google, keep them logged in permanently unless they click logout or remain inactive for 1 week. Also audit Razorpay environment variables.

Agent

Implemented the full unauthenticated exploration + action gating system and permanent session lifecycle:
1. AuthModal (frontend/src/components/auth/AuthModal.tsx): Created an interactive Google sign-in modal that triggers whenever an unauthenticated visitor attempts active features: Humanize Draft (useStudioStore), Voice Calibration Drill (DrillCard), AI Chat composition (ChatWriterView), or Writing Adaptations (ActionDock).
2. Permanent Session & 1-Week Inactivity Policy (firebase.ts & App.tsx): Configured Firebase browserLocalPersistence so logged-in users remain permanently signed in across browser restarts. Added throttled user activity tracking (inkflow_last_activity) with automatic logout after 7 consecutive days of zero interaction.
3. Razorpay Env Security Audit: Verified RAZORPAY_KEY_SECRET is strictly confined to backend code and purged an unnecessary secret copy from frontend/.env, ensuring checkout uses only public key IDs.

▸ write_to_file frontend/src/components/auth/AuthModal.tsx

Input

frontend/src/components/auth/AuthModal.tsx

Output

Created accessible AuthModal with Google authentication, value highlights, and activity gating.

▸ replace_file_content frontend/src/lib/firebase.ts

Input

frontend/src/lib/firebase.ts

Output

Configured browserLocalPersistence, 1-week inactivity timeout limit, and throttled activity tracking.

▸ run_command npm run build && pytest tests -q

Input

$ npm run build && pytest tests -q

Output

Frontend built in 2.37s with 0 errors. All 39 backend tests passed.

Direct Session Link: View full agentic session transcript on DEV

My First User

what my first customer said

reviewfromcustomer.mp4 - Google Drive

drive.google.com

Prize Categories

Based on the project description, these are the partner categories to consider. Include only those for which you can demonstrate the required integration.

Best Use of Render: The FastAPI backend is deployed on Render.
Best Use of Sentry Agent Tracing:

Team Submissions

@aishani_3009 was in my team, help me to develop this product.

My aim with InkFlow is to build writing assistance that adapts to people rather than making people adapt to AI.

Better writing should still sound like you.

Hacktoberfest #DEVChallenge #HF26Challenge #SideHustle #AI #OpenSource #WritingTools

Goblin Mode: Nature Scavenger Bingo — An Offline-First AI Nature Hunt

Soumya ghosh — Sun, 11 Oct 2026 07:03:00 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

What if AI could encourage us to spend less time looking at screens and more time exploring the world around us?

That question inspired me to build Goblin Mode: Nature Scavenger Bingo, an AI-powered outdoor nature scavenger hunt designed to turn an ordinary walk through a park, garden, or campus into an interactive discovery experience.

Instead of endlessly scrolling through an app, users receive nature-based missions, explore their surroundings, photograph real specimens, and use AI to verify their discoveries.

The project addresses two important challenges: keeping users focused on the real world instead of their screens, and identifying natural objects accurately from imperfect photographs.

Goblin Mode combines an illustrated field-journal interface with computer vision, taxonomic knowledge retrieval, location-aware quest generation, and offline functionality.

Its central character is Grimble, an eccentric goblin naturalist who guides users through their discoveries with voice narration and a field-notebook-inspired interface—complete with stitched leather borders, torn deckled parchment pages, brass corner brackets, washi masking tape, and handcrafted watercolor-and-ink SVG botanical sketches.

How It Works

1. Discover Nature Through Interactive Missions

The application presents a 3×3 nature scavenger bingo board filled with missions inspired by leaves, insects, flowers, and other natural objects.

When users select a mission, the interface enters Focus Mode and displays one active challenge. This encourages users to put their phones away while exploring and bring them back only when they need to photograph their discovery.

When a user clicks "Generate New Board" to refresh their missions, the 3×3 grid cleanly unmounts and displays a dual concentric emerald-and-amber loading spiral with an enforced >= 850ms minimum transition window—preventing jarring sub-second visual flashes while giving clear feedback that a fresh board is being synthesized.

2. AI-Powered Nature Verification

Users can photograph a specimen and submit it for verification. Before the image reaches the vision model, the backend enhances the image, evaluates its visual details, and creates a dual-scale inspection image containing both the full photograph and a magnified region.

The verification pipeline combines image analysis, taxonomic knowledge, and multimodal AI to distinguish genuine discoveries from incorrect lookalikes or featureless images:

Stage 1 — Computer Vision Pre-Processing (image_enhancer.py): Raw camera photos are analyzed with a Laplacian edge filter (ImageFilter.FIND_EDGES) to measure structural edge density (ActiveEdgeRatio) and color variance (ColorStd), flagging blank walls or tiny dust specks before AI inference. The image is then converted into YCbCr color space to balance luminance (Y channel) contrast at a 45% blend without distorting natural green chlorophyll hues, sharpened via UnsharpMask(radius=1.6, percent=145%, threshold=3) to recover blurry leaf veins and insect legs, and composited into a single 784×448px Side-by-Side Dual-Scale Naturalist Inspection Plate (Left: 448×448px Full Frame | Right: 332×448px 2× Magnified Center Detail Crop).
Stage 2 — 4-Gate Taxonomic Referee (verify.py): Evaluates Subject Integrity (Gate 1), Mandatory Taxonomic Anatomy (Gate 2), Hard-Negative Impostors (Gate 3), and Outdoor Camera Blur Tolerance (Gate 4).

For example, the system is designed to distinguish a Peepal leaf (Ficus religiosa)—which requires a heart-shaped cordate base, straight pinnate-reticulate yellow venation, and an elongated tail-like caudate drip-tip ($\ge 20\%$ of blade length)—from visually similar indoor Money Plant (Epipremnum aureum) vines or Betel (Piper betle) leaves, and strictly reject a tiny dark speck on a floor as evidence of a Weaver Ant (Oecophylla smaragdina) unless 3-part body segmentation, a narrow petiole waist, and 6 jointed legs are visible.

Server-side validation checks the model's response before awarding progress—forcing passed = false and 0 XP if is_impostor_or_fake == true, confidence < 0.85, or the CV speck detector triggered—helping reduce incorrect identifications to achieve 90–95%+ taxonomic precision.

3. Dynamic Quest Generation

Goblin Mode can generate nature missions using location context rather than relying exclusively on a fixed list of challenges.

Its Universal Two-Tier Taxonomic RAG system (botanical_rag.py) combines a curated biodiversity index (Tier 1, using strict whole-word boundary regexes \b...\b) with dynamically generated species-verification criteria (Tier 2). Whenever a player attempts a randomly generated location-based quest that isn't in the static catalog (such as Touch-Me-Not Mimosa pudica, Crimson Callistemon Bottlebrush, Seven-Spotted Coccinella Ladybug, or Plumeria rubra), our backend queries the open-weight 120B openai/gpt-oss-120b model (reasoning_effort="low") on the fly to synthesize its scientific name, mandatory diagnostic traits, and lookalike impostors to reject. Cached results in _DYNAMIC_RAG_CACHE are reused in 0ms, while deterministic domain-class fallback rules handle cases where the AI service is unavailable.

4. Offline-First Exploration

Outdoor connectivity is not always reliable, so the application includes client-side HTML5 Canvas image normalization (64px min to 1280px max bounding box, #FFFFFF alpha compositing, 0.88 JPEG compression under 300 KB) and IndexedDB persistence (idb-keyval) for local progress and queued specimens.

A 3-tier procedural quest-generation system cascades from Local Ollama (gemma2:2b) $\to$ Cloud Groq (openai/gpt-oss-120b) $\to$ a client-side Combinatoric Matrix (combinatoricMatrix.ts) that multiplies 15 sensory adjectives, 15 nature nouns, and 10 location contexts to produce up to 2,250 combinations for zero-signal offline exploration.

Account-specific local storage (goblin_bingo_board_v4_<uid> vs. goblin_bingo_board_v4_guest) also keeps individual users' boards and XP progress strictly isolated on the same device.

5. Voice, Atmosphere, and Accessibility

Grimble provides expressive voice narration through ElevenLabs (eleven_multilingual_v2) when configured, backed by a deterministic SHA-256 disk audio cache (backend/audio_cache/) that drops repeat voice latency from ~1,400ms to <15ms and supported by a dedicated Centralized Dual-Channel Audio Controller (audioManager.ts) that separates SFX and Voice channels so UI clicks never overlap or interrupt Grimble's field commentary.

The interface also adapts its visual atmosphere to the time of day in Indian Standard Time (UTC +05:30) on a fast 10-second polling heartbeat, with distinct Morning/Day (07:00–17:00 IST) sunlit pollen motes, Evening/Sunset (17:00–19:00 IST) amber-violet twilight washes, and Night (19:00–07:00 IST) moonlit washes with a crescent moon, twinkling stars, and drifting bioluminescent fireflies that are strictly hidden outside nocturnal hours.

Together, these features aim to make nature exploration feel like an interactive field adventure rather than another screen-based game.

Demo

Live Web App (Please try on phone for best experience):

goblin-nature-bingo.vercel.app

Live Backend Health Telemetry: https://goblin-nature-bingo.onrender.com/api/health

Demo video: https://drive.google.com/file/d/1DqGSOqb6mjMUv41udrZGpEPFkdf78fXF/view?usp=sharing

Code

GitHub Repository: https://github.com/SoumyaGhosh2006/Goblin_Nature_Bingo

SoumyaGhosh2006 / Goblin_Nature_Bingo

🌿 Goblin Nature Bingo

Offline-First, Location-Aware Outdoor Field Journal & AI Botanical Scavenger Hunt

📖 Overview

Goblin Nature Bingo is an interactive, offline-first outdoor scavenger hunt web application guided by Grimble, a voice-synthesized goblin naturalist who gets players off their screens and physically outside exploring parks, gardens, campuses, and neighborhood trails.

Instead of endlessly scrolling through a screen, users receive nature-based missions on a 3×3 weathered leather-and-parchment field binder, step outside to explore their surroundings, photograph real botanical and zoological specimens, and use a multi-stage Computer Vision + Universal Taxonomic RAG + Open-Weight Multimodal AI referee to verify their discoveries.

Core Capabilities

Tactile Naturalist Field Journal & Screen-Free Focus Mode: A 3×3 deckled-paper Bingo grid featuring stitched leather borders, brass corner brackets, washi masking tape, and bespoke ink-and-watercolor SVG botanical illustrations. Selecting any quest tile enters Focus Mode, collapsing the UI into a single sensory outdoor mission…

View on GitHub

Explore the implementation, report issues, and contribute to making outdoor AI experiences more reliable and accessible.

How I Built It

Frontend: React 19, TypeScript, Vite, Tailwind CSS, Framer Motion, HTML5 Canvas, and IndexedDB using idb-keyval.

Backend: Python 3.11, FastAPI, Pillow, httpx, and Pydantic.

AI and Computer Vision: Qwen 3.8 27B and GPT-OSS 120B through Groq, with local Ollama integrations for Gemma 2 and Moondream.

Authentication and Persistence: Firebase Authentication, Cloud Firestore, and account-scoped IndexedDB storage.

Voice and Monitoring: ElevenLabs text-to-speech and Sentry SDK integration.

Deployment: Vercel for the frontend and Render for the FastAPI backend.

The architecture separates image enhancement, taxonomic retrieval, visual verification, quest generation, persistence, and audio management into distinct components. This makes individual systems easier to maintain and gives the application multiple fallback paths when a service is unavailable.

Our Engineering Journey: Real Problems We Faced & How We Tackled Them

Building an AI app that works reliably outdoors required solving several non-obvious systems and computer-vision problems that emerged during real-world testing:

1. The "Everything Is Blurry" Bug & RGB Auto-Contrast Distortion

What Went Wrong: When we first deployed the backend to Render and tested real phone photos—including a real damp mossy wall in a bathroom and outdoor leaves—the verifier kept rejecting valid photos as "blurry or obscured." Tracing the pipeline revealed two culprits: first, our cloud container was attempting to reach a local Ollama vision instance (127.0.0.1:11434) first and falling back to a blind placeholder description when Ollama wasn't running on Render; second, when we added standard RGB ImageOps.autocontrast to sharpen blurry camera shots, it stretched the Red, Green, and Blue histograms independently. On a green leaf with very little red-channel variance, independent RGB stretching distorted natural chlorophyll greens into dark, unnatural blotches!
How We Solved It: We routed cloud vision directly to multimodal qwen/qwen3.8-27b and re-engineered image_enhancer.py to convert images into YCbCr color space, applying a 45%-blended autocontrast exclusively on the Luminance (Y) channel before merging back with the untouched Cb and Cr chrominance channels. Paired with UnsharpMask(radius=1.6, percent=145%, threshold=3), this recovered crisp leaf venation, serrated edges, and mossy bryophyte textures from slightly blurry phone shots while preserving 100% of true botanical RGB hues.

2. When the Model Became "Too Chill": Stopping Floor Dots & Vine Impostors

What Went Wrong: After making the prompt tolerant of slight camera blur, the model became overly lenient. When we tested it by submitting a photo of tiny black and white dots on a floor for a "Weaver Ant Trail" quest, it approved them as ants! When we submitted a random heart-shaped indoor Money Plant vine leaf for a "Heart-Shaped Peepal Leaf" quest, it approved that too—only rejecting photos when the frame was a completely blank wall.
How We Solved It:
1. Laplacian Edge & Speck Telemetry: We added pre-inference computer vision telemetry using ImageFilter.FIND_EDGES and ImageStat. Any image with ActiveEdgeRatio < 0.010 and ColorStd < 18.0 (or EdgeMean < 2.0) is automatically flagged with is_featureless_or_speck = True, mathematically catching floor dots and blank surfaces.
2. 4-Gate Taxonomic RAG & Whole-Word Regexes: We built botanical_rag.py with explicit Mandatory Diagnostic Traits and Hard-Negative Impostors to Reject, plus a server-side post-validation guard (confidence >= 0.85 and is_impostor_or_fake == False). During testing, we also caught a classic regex bug: substring matching caused the keyword "ant" in our arthropod classifier to falsely match "Touch-Me-Not Plant" (pl-ant) and "Fragrant Plumeria" (fragr-ant)! Switching to strict word-boundary regular expressions (\b{keyword}s?\b) fixed the collision immediately. In benchmark runs, floor dots and wrong vine leaves were cleanly rejected at confidence = 0.00–0.05.

3. Overcoming the 7,000 ITPM Token Wall & Supporting Infinite Location Quests

What Went Wrong: Because players can generate quests dynamically anywhere in the world, we couldn't rely on a static hardcoded species list. However, when we initially sent two high-resolution base64 images (1280px Full Frame + 768px 2× Detail Crop) and ran dynamic RAG synthesis on the same vision model (qwen/qwen3.8-27b), a single verification consumed 5,194 input tokens—hitting Groq's 7,000 ITPM free-tier rate limit on the very second photo! We also discovered that legacy .env configurations referencing llama-3.2-11b-vision-preview failed with HTTP 400 model_decommissioned.
How We Solved It:
- 5.5× Vision Token Reduction: Instead of sending two separate high-res images, image_enhancer.py composites the 448×448 Enhanced Full View and the 332×448 2× Magnified Center Zoom side-by-side into a single 784×448px Naturalist Inspection Plate (~450 vision tokens). Combined with /no_think and max_tokens: 220, total input tokens dropped from 5,194 to ~900 tokens per verification (an 82% reduction), enabling 6–7 consecutive photo verifications per minute with zero rate-limit errors.
- Decoupled 120B Dynamic RAG Synthesizer: We routed Tier-2 Dynamic Taxonomic RAG synthesis and board generation to Groq's 120B openai/gpt-oss-120b (reasoning_effort="low"), which uses a separate rate-limit bucket and synthesizes full scientific rubrics for any unseen location quest in ~0.3s (cached in _DYNAMIC_RAG_CACHE for 0ms repeat lookups).
- Self-Healing Model Config: Added automatic runtime sanitization in config.py that transparently upgrades decommissioned .env model strings to qwen/qwen3.8-27b.

4. Eliminating Audio Layering, Clock Drift & Cross-Account State Leaks

Centralized Audio Preemption: Rapidly clicking specimen tiles, lighting toggles, and Grimble's voice button originally caused 2–3 audio instances to play simultaneously. We replaced scattered .play() calls with a singleton dual-channel controller (audioManager.ts) that runs pause() + currentTime = 0 with a 60ms debounce on the SFX channel while protecting the Voice channel from minor UI interruptions.
Account-Scoped Storage Hydration: To prevent Player A's offline board and XP from leaking when Player B logs in on the same device, we bound IndexedDB keys to Firebase Auth's onAuthStateChanged UID (goblin_bingo_board_v4_<uid>) and purged in-memory state on logout.

Why Open Innovation Matters

Open-weight models and accessible developer tools make it possible to experiment with AI beyond conventional chatbot interfaces.

Goblin Mode explores how these technologies can support outdoor learning, biodiversity awareness, and hands-on discovery. Its combination of cloud models (qwen/qwen3.8-27b and openai/gpt-oss-120b), local inference options (gemma2:2b and moondream via Ollama), and deterministic offline fallbacks (2,250-combination procedural matrix) is designed to reduce dependence on a single proprietary vendor, survive model deprecations seamlessly, and run at $0 cost for students and educators.

Local inference through Ollama can also support privacy-conscious exploration when the relevant models are installed and configured on the user's device, keeping backyard and campus photographs completely on local hardware.

The broader goal is to make AI a companion to real-world experiences, not a replacement for them.

My Agent Session

Building Goblin Mode: Universal Taxonomic RAG & Dual-Scale Computer Vision Referee

You

Fix three issues in Goblin Mode: Nature Scavenger Bingo:
1. Atmosphere phase updates too slowly after device time changes (poll every 10-15s in Auto mode).
2. Redefine IST time boundaries to Day (07:00-17:00), Evening/Sunset (17:00-19:00), and Night (19:00-07:00) with fireflies strictly nocturnal.
3. Fix overlapping audio playback by implementing a centralized dual-channel audio manager (SFX and Voice) that resets currentTime = 0 and protects Grimble's voice commentary from minor UI clicks.

Agent

I will update RealTimeAtmosphere.tsx to poll every 10 seconds using exact UTC-to-IST (+05:30) minute boundary calculations (Day 07:00–17:00, Sunset 17:00–19:00, Night 19:00–07:00) and build a singleton Dual-Channel Audio Manager (audioManager.ts) separating SFX and Voice channels.

▸ write_to_file frontend/src/utils/audioManager.ts

Input

frontend/src/utils/audioManager.ts

Output

Created centralized dual-channel audio preemption manager with SFX (60ms debounce + pause/reset) and protected Voice channel.

▸ replace_file_content frontend/src/components/RealTimeAtmosphere.tsx

Input

frontend/src/components/RealTimeAtmosphere.tsx

Output

Updated IST phase calculation to 10-second polling interval with strict nocturnal firefly rendering (19:00–07:00 IST).

You

When someone clicks on 'Generate new board', hide the board and show an animated loading circle/spiral until the new board is ready, then reveal the board.

Agent

I will add an isGeneratingBoard state in App.tsx with a minimum 850ms transition threshold and render a dual concentric emerald/amber loading spiral inside BingoBoard.tsx while new quests are synthesized.

▸ replace_file_content frontend/src/components/BingoBoard.tsx

Input

frontend/src/components/BingoBoard.tsx

Output

Added AnimatePresence board unmount transition and dual concentric rotating loading circles with live status copy.

You

Now we need the image verification pipeline to achieve 90-95% taxonomic accuracy across both curated quests and any randomly generated location-based quests. It must not reject slightly blurry real photos (like damp mossy walls), but it must strictly reject fake impostors like black/white floor dots submitted as Weaver Ants or random Money Plant vine leaves submitted as a heart-shaped Peepal leaf. Use computer vision enhancers and a Taxonomic RAG system.

Agent

I will architect a 3-stage Computer Vision + Universal Two-Tier Taxonomic RAG + 4-Gate Multimodal Referee pipeline:
1. backend/services/image_enhancer.py: Computes Laplacian edge density (ImageFilter.FIND_EDGES) to flag featureless floor specks, applies 45%-blended YCbCr Y-channel auto-contrast (preserving true botanical RGB hues), sharpens venation via UnsharpMask(radius=1.6, percent=145, threshold=3), and builds a 784x448px Side-by-Side Dual-Scale Naturalist Inspection Plate (Left: 448x448 Full Frame; Right: 332x448 2x Magnified Center Zoom), cutting vision input tokens by 5.5x (5,194 -> ~900 tokens).
2. backend/services/botanical_rag.py: Implements Tier-1 whole-word regex matching (\b...\b) for 17+ core species and Tier-2 Live Dynamic Taxonomic RAG Synthesis via openai/gpt-oss-120b (with in-memory LRU caching) for 100% of unseen location-generated quests.
3. backend/routes/verify.py: Evaluates the inspection plate with qwen/qwen3.8-27b across 4 sequential gates and enforces a strict server-side post-validation override (confidence >= 0.85, is_impostor_or_fake == False, is_featureless_or_speck == False).

▸ write_to_file backend/services/image_enhancer.py

Input

backend/services/image_enhancer.py

Output

Created YCbCr luminance contrast + UnsharpMask + 784x448 side-by-side dual-scale inspection plate generator.

▸ write_to_file backend/services/botanical_rag.py

Input

backend/services/botanical_rag.py

Output

Created Universal Two-Tier Botanical & Zoological RAG Engine with curated species index and live openai/gpt-oss-120b dynamic taxonomic synthesizer.

▸ write_to_file backend/routes/verify.py

Input

backend/routes/verify.py

Output

Updated /api/verify with 4-Gate multimodal evaluation, /no_think token optimization, 429 retry backoff, and server-side post-validation guards.

▸ run_command python test_universal_rag.py

Input

$ python test_universal_rag.py

Output

Verified dynamic RAG synthesis across 5 unseen random quests (Mimosa pudica, Callistemon citrinus, Coccinella septempunctata, Pinus spp., Plumeria rubra) and confirmed rejection of floor specks (conf=0.05) and impostor vine leaves (conf=0.00).

Direct Link: Building Goblin Mode: Universal Taxonomic RAG & Dual-Scale Computer Vision Referee

Prize Categories

Best Use of Render: Deploys and orchestrates our FastAPI AI verification, Pillow Computer Vision enhancer, and Dynamic Taxonomic RAG backend (goblin-nature-bingo-api) via a declarative render.yaml Blueprint and Procfile.
Best Use of Gemma: Integrates Google's open-weight Gemma 2 (gemma2:2b) model via Ollama (backend/services/ollama_client.py) for local zero-cost procedural quest generation and botanical verification.
Best Use of ElevenLabs: Powers Grimble the Goblin's voice narration (backend/services/elevenlabs_svc.py, backend/routes/voice.py) with deterministic SHA-256 disk caching and a dual-channel audio preemption manager.
Best Use of Sentry Agent Tracing: Uses sentry-sdk>=2.0.0 (backend/main.py, backend/routes/verify.py) with FastAPI transaction tracing (traces_sample_rate=1.0) to monitor multimodal vision latency, rate-limit retries, and RAG pipeline performance.

What I Learned

Building Goblin Mode gave me an opportunity to explore the challenges of combining computer vision, retrieval-augmented generation, offline-first web development, and multimodal AI in one application.

I learned that a useful AI experience requires more than a model call. Image quality, color-space math (YCbCr vs. RGB), false-positive impostor gates, token budget optimization (5,194 $\to$ ~900 tokens), SHA-256 caching, local IndexedDB persistence, per-UID account isolation, and multi-tier failure recovery all matter when software is used outside controlled conditions.

The project also reinforced an important design principle: technology should encourage meaningful experiences rather than demand constant attention.

Step outside. Find something unexpected. Complete your next nature quest.

Hacktoberfest #DEVChallenge #HF26Challenge #TouchGrass #AI #OpenSource #Gemma #TabPFN #ElevenLabs #Sentry #Render

SchoolDesk: AI-Powered Administration and Teaching Support for Schools

Soumya ghosh — Sun, 04 Oct 2026 19:35:17 +0000

title: SchoolDesk: AI-Powered Administration and Teaching Support for Schools
published:

tags: devchallenge, weekendchallenge, hf26challenge

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

I built SchoolDesk, a dual-purpose administrative and academic web platform designed specifically for my father, who is a school principal.

Managing a government school in India involves immense bureaucratic overhead, particularly the daily accounting for the "Midday Meal" scheme (tracking rice consumption, expenses, and student attendance). My father and his staff were losing hours daily to manual paper ledgers. Furthermore, teachers were spending significant off-the-clock time developing chapter notes and complex whiteboard diagrams.

This platform solves both problems:

The Administrative Ledger: A voice-activated inventory tracker that takes natural language audio, transcribes it, and automatically structures it into a secure, downloadable, government-compliant Excel sheet and a generated "analog-style" handwritten digital ledger.
The Teaching Aid (Admin Notes): A client-side PDF scanner that ingests textbook chapters and uses a rigorously constrained LLM to output a master formula sheet, Higher-Order Thinking Skills (HOTS) questions, and dynamically generated SVG whiteboard diagrams.

Demo

school-desk-lime.vercel.app

video link:

hacktober1stchallenge.mp4 - Google Drive

drive.google.com

Code

SoumyaGhosh2006 / SchoolDesk

SchoolDesk

Dual-Purpose School Administrative Ledger & AI Pedagogical Assistant

📖 Overview

In government and public schools, headmasters, principals, and teachers face two recurring, time-consuming administrative challenges every single day:

Manual Mid-Day Meal (MDM) Ledger Bookkeeping: Manually recording daily student attendance, calculating ingredient costs (eggs, vegetables, oil, pulses, spices, fuel), reconciling bulk rice grain stocks, and maintaining flawless handwritten inspection registers for government compliance.
Time-Intensive Lesson Preparation: Extracting formulas, drafting Higher-Order Thinking Skills (HOTS) questions, synthesizing student mistake pitfalls, and drafting whiteboard diagrams across heavy textbook chapters.

SchoolDesk solves both problems with a unified, voice-driven digital workspace. It allows school staff to speak their daily meal expenses into existence in seconds, generate audit-ready spreadsheets and realistic handwritten registers, digitize physical vouchers with a built-in camera document scanner, and instantly convert syllabus PDFs into structured, textbook-grade teaching aids.

🚀 Core Features & Use Cases (User Guide)

SchoolDesk is designed to be completely…

View on GitHub

How I Built It

Tech Stack

Frontend: React, Vite, Tailwind CSS (Typography), remark-gfm / rehype-katex (for LaTeX rendering), html-to-image (for flawless PDF exports bypassing CSS color crashes).
Backend: FastAPI (Python), PyPDF2.
AI & Open Source: Meta's Llama 3 70B (via Groq for lightning-fast inference), OpenAI Whisper (for audio transcription).
AI Collaborators: Antigravity (coding agent), Gemini, and ChatGPT for architectural thought-partnering and prompt engineering.

System Architecture & Workflow

The application is built on two highly resilient, state-driven pipelines.

Pipeline 1: The Voice-to-Ledger Engine

[Voice Input]
     ↓
[Audio Sanitization / Base MIME Extraction]
     ↓
[Whisper API Transcription]
     ↓
[Raw Text]
     ↓
[LLM Data Extraction]
     ↓
[Strict Pydantic JSON Schema]
     ↓
[Frontend State Update]
     ↓
[Human Verification / Editing]
     ↓
[CSV or Handwritten Ledger Export]

Pipeline 2: The Pedagogical Teaching Aid Engine (Admin Notes)

[PDF/Image Upload]
     ↓
[Client-side jsPDF Scanner / PyPDF2 Extraction]
     ↓
[Raw Textbook Text]
     ↓
[LLM Inference (Llama 3 70B)]
     ├─> State 1: Zero Hallucination Grounding (Extracts exact formulas)
     ├─> State 2: 20% Prerequisite Memory Injection (Temperature = 0.2)
     ├─> State 3: Strict Escaping (Raw String Python Prompts)
     └─> State 4: SVG Generation (Forced Complexity Parameters)
     ↓
[Frontend Rendering]
     ├─> Sanitized GitHub Flavored Markdown (GFM)
     ├─> KaTeX CSS for Textbook-Quality Equations
     └─> Tailwind Prose for Professional Typography
     ↓
[Google Docs Bridge / High-Res PDF Export]

Why Does Open Innovation Matter?

Open innovation is the only way a project like this can exist in the real world. Closed, proprietary APIs are often too expensive, rigid, or heavily rate-limited for public sector education systems in developing nations. By utilizing open-weight models like Llama 3 and open-source audio transcription, we can achieve enterprise-grade automation on a shoestring budget.

The Real-World Impact & Metrics

Administrative Efficiency: What previously took 1.5 to 2 hours of manual calculation, cross-checking, and physical ledger entry is now accomplished via a 30-second voice note while walking down the school hallway. The data is instantly structured, mathematically verified, and ready for government audit.
Pedagogical Quality: Designing engaging, challenging lesson plans with accurate visual aids is time-consuming. The Teaching Aid module reduces a 3-hour lesson prep down to 15 seconds. By injecting "prerequisite formulas" and HOTS questions, it actively upgrades the quality of education delivered to the students.
Market Positioning: 95% of current EdTech focuses on the student (AI tutors, homework solvers). This project targets the administrators and teachers—a massive, underserved demographic that desperately needs tools to reduce burnout and administrative bloat.

Prize Categories

SchoolDesk currently uses open-source/open-weight AI and local inference, and using render.com to deploy my backend and to host the database,

tags: devchallenge, weekendchallenge, hf26challenge

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

I built SchoolDesk, a dual-purpose administrative and academic web platform designed specifically for my father, who is a school principal.

This platform solves both problems:

The Administrative Ledger: A voice-activated inventory tracker that takes natural language audio, transcribes it, and automatically structures it into a secure, downloadable, government-compliant Excel sheet and a generated "analog-style" handwritten digital ledger.
The Teaching Aid (Admin Notes): A client-side PDF scanner that ingests textbook chapters and uses a rigorously constrained LLM to output a master formula sheet, Higher-Order Thinking Skills (HOTS) questions, and dynamically generated SVG whiteboard diagrams.

Demo

school-desk-lime.vercel.app

video link:

hacktober1stchallenge.mp4 - Google Drive

drive.google.com

Code

SoumyaGhosh2006 / SchoolDesk

SchoolDesk

Dual-Purpose School Administrative Ledger & AI Pedagogical Assistant

📖 Overview

In government and public schools, headmasters, principals, and teachers face two recurring, time-consuming administrative challenges every single day:

Manual Mid-Day Meal (MDM) Ledger Bookkeeping: Manually recording daily student attendance, calculating ingredient costs (eggs, vegetables, oil, pulses, spices, fuel), reconciling bulk rice grain stocks, and maintaining flawless handwritten inspection registers for government compliance.
Time-Intensive Lesson Preparation: Extracting formulas, drafting Higher-Order Thinking Skills (HOTS) questions, synthesizing student mistake pitfalls, and drafting whiteboard diagrams across heavy textbook chapters.

🚀 Core Features & Use Cases (User Guide)

SchoolDesk is designed to be completely…

View on GitHub

How I Built It

Tech Stack

Frontend: React, Vite, Tailwind CSS (Typography), remark-gfm / rehype-katex (for LaTeX rendering), html-to-image (for flawless PDF exports bypassing CSS color crashes).
Backend: FastAPI (Python), PyPDF2.
AI & Open Source: Meta's Llama 3 70B (via Groq for lightning-fast inference), OpenAI Whisper (for audio transcription).
AI Collaborators: Antigravity (coding agent), Gemini, and ChatGPT for architectural thought-partnering and prompt engineering.

System Architecture & Workflow

The application is built on two highly resilient, state-driven pipelines.

Pipeline 1: The Voice-to-Ledger Engine

[Voice Input]
     ↓
[Audio Sanitization / Base MIME Extraction]
     ↓
[Whisper API Transcription]
     ↓
[Raw Text]
     ↓
[LLM Data Extraction]
     ↓
[Strict Pydantic JSON Schema]
     ↓
[Frontend State Update]
     ↓
[Human Verification / Editing]
     ↓
[CSV or Handwritten Ledger Export]

Pipeline 2: The Pedagogical Teaching Aid Engine (Admin Notes)

[PDF/Image Upload]
     ↓
[Client-side jsPDF Scanner / PyPDF2 Extraction]
     ↓
[Raw Textbook Text]
     ↓
[LLM Inference (Llama 3 70B)]
     ├─> State 1: Zero Hallucination Grounding (Extracts exact formulas)
     ├─> State 2: 20% Prerequisite Memory Injection (Temperature = 0.2)
     ├─> State 3: Strict Escaping (Raw String Python Prompts)
     └─> State 4: SVG Generation (Forced Complexity Parameters)
     ↓
[Frontend Rendering]
     ├─> Sanitized GitHub Flavored Markdown (GFM)
     ├─> KaTeX CSS for Textbook-Quality Equations
     └─> Tailwind Prose for Professional Typography
     ↓
[Google Docs Bridge / High-Res PDF Export]

Why Does Open Innovation Matter?

The Real-World Impact & Metrics

Administrative Efficiency: What previously took 1.5 to 2 hours of manual calculation, cross-checking, and physical ledger entry is now accomplished via a 30-second voice note while walking down the school hallway. The data is instantly structured, mathematically verified, and ready for government audit.
Pedagogical Quality: Designing engaging, challenging lesson plans with accurate visual aids is time-consuming. The Teaching Aid module reduces a 3-hour lesson prep down to 15 seconds. By injecting "prerequisite formulas" and HOTS questions, it actively upgrades the quality of education delivered to the students.
Market Positioning: 95% of current EdTech focuses on the student (AI tutors, homework solvers). This project targets the administrators and teachers—a massive, underserved demographic that desperately needs tools to reduce burnout and administrative bloat.

Prize Categories

SchoolDesk currently uses open-source/open-weight AI and local inference, and using render.com to deploy my backend and to host the database,