DEV Community: Sunbal Ghayas

who would you talk to?

Sunbal Ghayas — Sun, 01 Nov 2026 14:31:27 +0000

Hacktoberfest Open-Source AI Challenge: Ghost in the Machine Submission 👻🌼

Sunbal Ghayas

Nov 1

I built a haunted castle for the people I miss. Give it a photo and a story, and they pull up in the Hall.

#devchallenge #hf26challenge #ai #python

8 min read

I built a haunted castle for the people I miss. Give it a photo and a story, and they pull up in the Hall.

Sunbal Ghayas — Sun, 01 Nov 2026 02:26:04 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine

What I Built

Ghost Gallery is a haunted castle you scroll through. You start outside in the fog, with one window lit.

The castle in the fog with small ghosts drifting past and the title "Where all tales come to an end... or perhaps, just the beginning."

Scroll, and the camera slides through the glass into a midnight banquet where Einstein, Cleopatra, Shakespeare, Marie Curie and Tesla are already seated.

Five translucent ghosts around a candlelit table: Einstein, Cleopatra, Shakespeare, Marie Curie and Tesla

Scroll again and a portal opens onto a graveyard with headstones for people history forgot. You can leave each one a white rose.

A foggy graveyard with a card that says "Didn't find who you were looking for?" and two buttons: Summon Spirit and Remember Loved One

That's the Halloween half. The half I care about is that you can add someone of your own.

Give a name, a memory and a photo. The photo is cut out, washed pale and made see-through, and your person drifts into the room with the others. Then you can talk to them, by typing or by voice.

They only know what you told them. Ask about something outside your story and they should say, gently, that the memory has faded, and ask you to tell them more. I didn't want a grandmother ghost who invents things. (More on how that went below.)

That's how I read the theme. Día de los Muertos is about telling the stories of the dead so they stay. The castle is the costume. The "tell me about her" box is the ofrenda.

TL;DR. The thinking runs on Gemma 3 through Ollama on your own machine. Speech-to-text runs in your browser with OpenAI's open-source Whisper. The see-through cutout runs in your browser too. No accounts, no API keys, no bills.

Demo

Code

sunbal822 / GhostGallary

Ghost Gallery 👻🌼

A haunted castle for the people we miss. Scroll from the foggy exterior, through a lit window, into a midnight banquet of famous spirits, then through a portal to a graveyard where you can leave a white rose.

The part that matters: add someone of your own. Give a name, a memory and a photo. The photo is cut out, washed pale and made see-through, and your person joins the room. Then talk to them, by typing or by voice. They only know what you told them, and say so when the memory has faded.

Built for the Hacktoberfest Open-Source AI Challenge, Ghost in the Machine.

Demo / write-up:

Everything runs on your machine

Job	Open piece	Where it runs
Reads a loved one's photo (what they wear, hold, glow colour)	Gemma 3 via Ollama	localhost
Plays the ghost in chat	Gemma 3 via Ollama (any

…

View on GitHub

How I Built It

The open pieces

Gemma 3 via Ollama does three jobs. It looks at the uploaded photo and picks what the ghost wears on their head and holds in their hand, a glow colour, and one warm line about them. It plays the ghost in chat. And it writes the profile when you summon a famous person. Everything goes to localhost, and the model is one environment variable. (Swap in gpt-oss:20b if you like.)
Whisper, in the browser (via Transformers.js, in a web worker) turns your voice memo into text. The recording never leaves your device.
@imgly/background-removal (an ISNet model on ONNX Runtime Web) cuts the person out of the photo, right in the browser. Canvas code then makes them pale and translucent, keeps a faint trace of their real clothing colour, and fades the lower half into mist.
Wikipedia supplies portraits for the famous spirits.

The rule: the model proposes, the code decides. For the photo, Gemma has to choose from fixed lists: ten head accessories, eight held items. Anything off the list becomes “none,” and a colour that isn’t valid hex falls back to a default. If Ollama is down, the ghost is still created with gentle default text instead of an error, and chat says “voice is faint tonight, start Ollama.”

The chat prompt is built only from what you gave it: the name, your story and the epitaph. It keeps the last 14 turns, caps replies at a few sentences, and forbids inventing painful facts, diagnoses, secrets or promises.

Your original photo is never saved. The server keeps the see-through cutout and the text.

Not everything is a model. The banquet whispers and headstone epitaphs are hand-written. The wind and the low drone are synthesised with the Web Audio API, with no audio files. The whole journey is one animation loop driven by scroll position.

Talking to ghosts (and telling them things)

Typing a eulogy is hard. Saying it out loud is easier. So there’s a mic next to the send button in every chat, and “tell it by voice” on the memory box when you create a ghost. Your voice becomes text locally, and you can edit it before saving.

The "Summon Someone You Loved" form filled in for Grandma Rosie, with voice buttons next to the name, story and epitaph fields

The ghosts can hear you, but they don't talk back out loud yet. That's next.

What didn’t go to plan

The red sweater. My first version tinted a small ghost sprite with the person’s clothing colour. For a loved one in a red sweater, the whole body turned red. That’s when I switched to cutting out the real photo.
Liam Payne came out as a silhouette. My own ghosts (hi, Grandma Susie) looked right, but summoned famous people kept the plain placeholder. The server tried to download their Wikipedia portrait exactly once, and when that failed it silently gave up. Now it tries several sources, then lets the browser try, then waits for the real figure before showing the ghost. If there’s truly no photo, it asks you for one instead of quietly summoning a blank.
My ghost quoted something nobody said. I tested with a made-up grandmother. When I said hello, she answered with a line about memory being the bridge between this world and eternity, "as I always said."

A chat with Grandma Rosie: she replies that memory is the bridge between this world and eternity, "as I always said"

She never said it. It was my own placeholder quote, the filler the app stores when you don't give an epitaph or saying, and I'd been feeding it to the model as a fact about her. That's exactly the invention I swore this ghost wouldn't do, and I only caught it because I read my own screenshot. Placeholder text is now never passed to the model as a fact about a person.

I ripped out my own cloud fallback. An earlier version could fall back to Gemini. It worked, and it was cheating. If the claim is “open-source AI at the core,” the fallback has to be open too, so everything now goes through Ollama or runs in the browser.
My first test ghosts came back with the default text ("A cherished soul remembered across time"), which means the model never answered in time.

Why Does Open Innovation Matter?

A photo of someone who has died, plus the story you type under it, is about as personal as data gets. I don’t want it on a server I don’t control.

Private by design. Photo and story go to Ollama on localhost. Your voice is transcribed in your browser. No accounts. Ghosts are private by default, tied to an anonymous id in your browser, and sharing to the gallery is a checkbox you have to tick. (That’s a convenience, not a security boundary.)
A memorial shouldn’t have an expiry date. An ofrenda is kept for generations. A model I’ve downloaded still runs if an API is repriced or retired.
Works with the internet off, mostly. Once the models are downloaded, reading a photo, talking to a ghost and speaking to one need no connection. What still does:
The first run downloads the cutout model (about 80 MB) and the Whisper model (also around 80 MB), then caches them.
Page fonts come from Google Fonts.
Summoning a famous person looks them up on Wikipedia.
I can change how it behaves. JSON mode, reply length, temperature and the model itself are all mine to set. I could swap Whisper sizes or the chat model in one line.

**The honest trade-off. **A local model is slower and less polished than a frontier one. A first summon can take a while while Gemma writes the profile. Summoned ghosts live in server memory, so they disappear when the server restarts; ghosts you make yourself are saved.

The Spirit of It

I made Grandma Rosie [who she is to you, or that she's someone you invented to test it]. I gave the story the kind of details a person actually remembers: she was always in a red knitted sweater, always baking something, and her pecan pies were the best thing in the house. Then I gave it one photo.

Grandma Rosie's profile: her photo turned into a pale, see-through figure fading into mist, beside her story

What came back is a woman in a muted red cardigan, pale and glowing, fading into smoke from the waist down. The red survived the wash. That part isn't even the model: a few lines of canvas code keep a faint trace of the real clothing colour. But it was the first thing that looked like her.

Then she took a seat.

Grandma Rosie floating at the banquet between Einstein and Cleopatra and Shakespeare

There she is at the banquet, between Einstein and Cleopatra, as if she'd always been invited.

Then I talked to her. First, something that is in her story:

Chat with Grandma Rosie: "do you still make pies over there" and she answers that she still bakes and always loved a good pecan pie

Me: hey i miss you do you still make pies over there Rosie: Oh, my dear, it's so lovely to see you. Yes, I do still bake, though it's not quite the same. I always did love a good pecan pie, didn't I? Tell me, what's been happening with you?

The pecan pie came from my story, not from the model's imagination. Then I tried the thing I was nervous about. I asked about something I never told her:

Chat with Grandma Rosie: asked about her favourite song, she says the years do fade things and asks what it is

Me: nothing much lately i listen to your fvorite song on repeat do you remember it Rosie: That's so sweet of you, dear. My favorite song...oh, my, the years do fade things. It was a lovely tune, though, full of warmth. Do tell me, what is it?

That's the moment the whole project is for. She didn't invent a song title. She said the years fade things and asked me to tell her.

Prize Categories

Best Use of Gemma: Gemma 3 through Ollama reads the photo, plays the ghost, and writes the summoned profiles, all locally, inside a validator that decides what’s allowed to stand.

[Boost]

Sunbal Ghayas — Sun, 25 Oct 2026 18:07:25 +0000

Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady Submission 🦥

Sunbal Ghayas

Oct 24

I talk to my laptop before bed, and by morning my to-do list is done: Jarvis

#devchallenge #hf26challenge #gemma #opensource

8 min read

I talk to my laptop before bed, and by morning my to-do list is done: Jarvis

Sunbal Ghayas — Sat, 24 Oct 2026 16:34:04 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

What I Built

Before bed I record a voice note: "write a Python program that checks if a number is prime, draft an email to Professor Khan asking for an extension on quiz 2, summarize the class group chat, and make the file for my assignment." Then I go to sleep. By morning Jarvis has done the work and left a report. When I open the Jarvis window, it speaks to me: what it finished, and what still needs me.

TL;DR. Jarvis turns a spoken to-do list into typed tasks and does them overnight. The listening and thinking (faster-whisper and Gemma 3 4B through Ollama) run entirely on my laptop. Nothing is uploaded except, optionally, the short text of the morning summary, which goes to ElevenLabs to be spoken. There are no API fees, so a night costs nothing beyond electricity.

What it handles:

Code: writes the program, runs it, and checks it doesn't crash.
Email: drafts only. Jarvis never sends anything.
Messages: a short WhatsApp or SMS text, not an email.
Reminders and calls: with real time awareness, and a pop-up when they are due.
Chat summaries, files, and everything else.

The lazy part: I never open a menu or pick a model. I say what I need in my own words, and Jarvis chooses the task types, the order and the model.

Demo

In the screenshot above, one voice note became four tasks (code, email, chat summary, file) and a morning report in 2 minutes 18 seconds. The ping to 8.8.8.8 at the top of the window times out, so the network was down for that run. The spoken summary needs internet, so I tested it separately.

A second run with a different voice note, start to finish:

Second full run: 4 tasks in 2m 44s, the code passed its own tests on the first attempt

The morning report: what was done, what needs my approval, what needs my help

A reminder Jarvis set from a voice note, firing on time

Code

sunbal822 / Jarvis

Jarvis: an overnight voice-to-tasks assistant that runs on your own laptop

Record a voice note before bed. Jarvis turns it into typed tasks, does them while you sleep, and gives you a morning report (and a spoken summary).

Transcription (faster-whisper) and the thinking (Gemma 3 4B through Ollama) run locally. Nothing is uploaded. The spoken summary uses ElevenLabs and is optional.

What it does

Task type	What Jarvis does
code	Writes the program, runs it, and checks it doesn't crash
email	Drafts only. It never sends anything
message	Writes a short WhatsApp/SMS-style text
reminder	Parses the time with plain code and pops up when it is due
chat_summary	Summarizes a chat you exported into `~/jarvis_output/inputs/`
file	Creates a document or file
other	Questions, explanations, plans

Requirements

Python 3.10+
Ollama, then: ollama pull gemma3:4b
pip install faster-whisper ollama pydantic
For the Jarvis window (jarvis_ui.py): [CHECK the imports at…

View on GitHub

Try it: python run_overnight.py my_voice.ogg. For a real night, run_nightly.bat is started by Windows Task Scheduler, and the README has the setup steps.

The nightly task in Windows Task Scheduler

How I Built It

The stack: faster-whisper (small.en) for transcription, Gemma 3 4B through Ollama for task extraction and drafting, Pydantic for strict JSON, ElevenLabs for the spoken morning summary, and plain Python for everything that must never be wrong.

The rule behind the whole project: the model judges, code guarantees. A 4B model is small enough for my laptop, but it makes confident mistakes. So the model decides what the tasks are, and ordinary code decides what is allowed to stand:

Reminder times are parsed by plain code, never by the model, so "tomorrow at 3pm" always means exactly that. I wrote 18 tests for the parser, including traps like "he sat down on Friday" (not Saturday) and "around 5 minutes" (not a clock time). All 18 pass.
A validator deletes any reminder time with a day or number the speaker never said, and marks the task "needs clarification" instead of guessing.

A "summary" only counts as a chat summary if the transcript mentions a chat. A summary of a topic or of emails is a normal task.

Code written by the model is scanned before it runs. If it uses risky calls (starting programs, the network), Jarvis refuses to run it and tells me.
If step 1 crashes while the OS shuts down a native audio library, the pipeline checks that tasks.json was written and carries on.

Why such a small model. I started with Gemma 12B. My laptop restarted three times mid-run. I switched to the 4B model and a bigger Whisper (small.en, because the small Vosk model heard "let this be" as "August"), then spent the effort on guardrails instead of parameters.

I measured it. I recorded real voice notes, transcribed them, and hand-labelled them with the right answers. The final set has 28 spoken to-do lists (21 real voice notes, 7 synthetic).

Run 3 looks like a step backwards, and it was. The new rules fixed the wrong answers, but 4 of 28 cases crashed with timeouts, so valid JSON fell to 86%. Fixing that took two more rounds.

The first eval, 18 cases

Before: the 28-case baseline results

After: the final results, 100% exact task list and 100% reminder time

A caveat I want to be upfront about: I tuned the rules on these same 28 cases, so the 100% means "I fixed everything I found," not "it will never miss." The clarification flag also dipped slightly (91% to 89%). The harder set exposed what the first 18 hid: reminder times the model made up.

What was failing, and what I changed. Every row below is a real case from the eval:

What the eval taught me:

The model invented times. "Set a reminder to call my mom" came back as "in 20 minutes". "Remind me on Saturday" came back as "Saturday at 00:00:00". The validator now blanks both and asks.
At temperature 0, one transcript can loop forever. A transcript with a typo ("Sit a reminder...") made the model count "147, 148, 149..." until it timed out, and temperature 0 means it loops the same way every retry. Now each retry changes the sampling settings, the answer length is capped, and the last attempt adds a repeat penalty and a "keep every field short" hint.
The bug I shipped. My own validator deleted the correct time in "7:00 PM", because it read the "00" as a number the speaker never said. The eval caught it. I would not have found it by looking at a few examples.

When it said no

On one run the code task came back BLOCKED. The model wrote a login app that started another program (subprocess), and Jarvis wrote the file, refused to run it, and listed it under "needs your approval." The next run passed on the first attempt, after I told the prompt to build apps as plain functions with in-memory data. Same voice note, same model: the first run was blocked, the second tested clean. That is why the guard exists.

Jarvis blocking a script that wanted to start another program

Why Does Open Innovation Matter?

Because the whole product is "I tell it everything." My voice, my emails, my assignments, my chats. I will not send that to a closed API every night.

Privacy by design. The voice, the transcript, the tasks and the drafts stay on my laptop. The only thing that leaves is the short spoken summary, and it is optional.
Zero marginal cost. A nightly assistant on a paid API has a bill that grows every night. Whisper and Gemma cost nothing to run.
Control over the failure modes. With open weights I could cap output length, change sampling on every retry, and build the validator around the exact mistakes this model makes. I couldn't have done the repeat-loop fix through a closed API.
Nobody can take it away. If a model is deprecated or repriced, my assistant still runs.

The honest trade-off: a frontier model would write nicer drafts and need fewer guards. A 4B model needs the guards, and takes about 30 seconds per extraction on my CPU. That is why it fits "Slow and Steady": it works while I sleep, so speed doesn't matter.

The Lazy Payoff: What it would have cost me by hand

Four jobs that I estimate would take about 75 minutes by hand. I spent 31 seconds talking, plus about 5 minutes the next morning reading results and filling in two names.

75 minutes by hand versus 31 seconds of talking and a 1 min 56 s run

Here is the night, straight from the log. I recorded one 31-second voice note, and Windows Task Scheduler started Jarvis at 11:59 PM. I didn't touch anything.

23:59:50 Jarvis starts on its own
00:00:34 voice note understood: 4 tasks (44 s)
00:01:06 Python program written, run, and tested ✔
00:01:13 email drafted, nothing sent ✔
00:01:13 reminder saved ✔
00:01:45 summary written, morning report ready ✔
Total: 1 minute 56 seconds.

The log of last night's run, start to finish

What it would have cost me by hand:

The Python program, with testing and trial: about 30 minutes
The email draft: about 10 minutes
The reminder (on my phone): about 5 minutes
A short summary in good shape: about 30 minutes

That is about 75 minutes. My part was 31 seconds of talking, plus about 5 minutes in the morning to read the results and fill in two names.

The morning report: four tasks done, two names left for me to fill in

The program Jarvis wrote, passing its own tests

And at 8:00 the next morning, the reminder fired on time, and the Jarvis window spoke it.

The 8:00 AM reminder firing, with the voice waveform playing

It isn't magic. I still read the code before I use it, fill in two names, and fact-check the summary. But the blank-page work, the part I put off, was done before I woke up.

Prize Categories

Best Use of Gemma: Gemma 3 4B does the task extraction and drafting, run locally with Ollama, with a code validator around it.
Best Use of ElevenLabs: Jarvis speaks the morning summary, what was done and what needs me. Transcription runs locally with Whisper.

That’s Not the Green I Meant”: A Colour Preview and Quote Tool for Crochet Sellers

Sunbal Ghayas — Sat, 17 Oct 2026 15:06:29 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle
Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle Submission 💸

What I Built

I started my crochet business only recently. Within a few weeks of meeting other crochet artists, I noticed we were all losing sleep over the same three problems.

“Same thing, but in different colours.” A customer sends a reference photo and wants it in other colours. Often they aren’t sure which shade they mean. “Green” could be light green, sage or even olive. We agree in words, and the finished piece arrives weeks later. Sometimes it isn’t what they pictured. Nobody did anything wrong; we just never looked at the same picture.
Pricing something that doesn’t exist yet. To price a piece I need to know how many grams of yarn it will use and how many hours it will take. I only know that after it’s made, but the customer wants a number today. So we guess, and most of us guess low.
Keeping track. Which orders have I agreed to? Which one is due in two days? If I accept one more, does another go late? Spread across chat threads, that is a lot to hold in your head.

So I built KARSAZ Stitch Desk, a small web app that handles all three. The seller goes through five steps:

Upload the customer’s reference photo and paste their message.
Say whether they want a colour change.
Pick the parts of the photo to recolour (petals, leaves, body, trim) and choose the colours, so the customer sees their order before any yarn is cut.
Enter the size or age, quantity, deadline and any extra changes.
Get a price, a ready date, a before-and-after picture, a reply message to send, and a quote card the customer can keep.

Every quote can be logged as an order. The Orders tab shows what is agreed, what is overdue and what is at risk of being late, and each seller sets their own yarn cost, hourly rate and markup in the Prices tab.

Demo

Watch the video:

Try it yourself:

karsaz.tech

Upload a photo of a crochet item on the New order tab. To see the Orders and Prices tabs, create a test account on the Account tab.

An honest note: the live site runs on a small server, and Gemma photo reading and Smart selection need more memory than it has. I run those on my own computer in the video. On the live site you choose the item by hand and colour parts with automatic colouring or hand-drawn shapes.

One order, start to finish

Here’s a real order going through the app. A customer sends a photo of a star baby cocoon and asks for it for a 1-year-old by 18 October. These screenshots were taken on my own computer, where Gemma and Smart selection run.

Step 1: Photo in

A custom order starts the way it does in real life: a reference photo and a message. The seller uploads the photo and pastes in whatever the customer wrote, such as “Want this for a 1 year old in mint green”. The message is optional, because many customers just send a picture.

On my own computer, Gemma reads the photo and the message to work out which item it is and what the customer asked for. It only identifies. It never produces a price or a date.

Step 2: Do they want a colour change?

This step is a single question: does the customer want the item in different colours, or exactly as in the photo? If not, the app skips the colour work and goes straight to the details. If yes, it opens the colour screen. It is quick on purpose, because many orders are “exactly this one, please”.

Step 3: Choose the colours

This is the step that solves “that’s not the green I meant”. The customer sees their order in the new colours before any yarn is cut. The seller can colour everything at once or one part at a time.

Option A:

One tap with a palette. The app suggests palettes made for this kind of item. For a baby cocoon: soft pastel, sunny morning, lavender dream and more. One tap colours every part.

Option B:

Part by part. Clicking a part of the photo, here the hood, selects it. The highlighted area is what will change. Then the seller taps a colour.

If “green” isn’t specific enough, “Any other colour” opens a picker with an exact hex code.

The new colour keeps the photo’s own light and shadow, so it still looks like crocheted yarn rather than a flat paint fill.

The customer can see this before any yarn is cut. I picked deliberately loud colours to show that each part is handled separately.

When the selection isn’t perfect. The “Fine-tune” panel lets the seller add area, remove area or draw a shape by hand. The quick-fix buttons (“Colour spilling outside”, “Old colour left at the edge”) adjust the last part. Fixing a rough edge takes a click, not a restart.

Step 4: Size, deadline and details

Now the practical details. The seller confirms the item, then enters the size or age, the quantity, the customer’s deadline and any extra changes, such as “add ears”. Extra changes add a small amount to the hours, and the app uses all of this to calculate the quote. This is the only typing the seller has to do.

Step 5: Price, ready date and reply

One button, “Create quote and reply”, and everything comes together on one screen: before and after side by side, the price, the grams and hours, and an estimate of yarn per colour.

The part I’m proudest of is the deadline check. The customer wanted this by 18 October. The app tells the seller that the earliest realistic date is 26 October, and explains why: about 7 days of crocheting at 3 hours a day, with only 1 day left. It then drafts a reply with the size in cm, the price and an honest date. The seller can copy it and edit it.

From here the seller presses “Log this order” or “Make quote card”. The quote card turns the quote into a single image with the shop logo, the customer’s name, before and after side by side, the price and the ready date. It’s easy to send in a chat, and it replaces a long message of numbers.

After the five steps: keeping track

Orders: Logged quotes go into the Orders tab. The seller moves each one from Quoted to Pending to In progress to Done, and only Pending and In progress count toward the workload check.

Prices: Every quote comes from numbers the seller sets: yarn cost, hourly rate, markup, rounding and hours per day. Change them once and every future quote follows. My numbers are defaults. Each seller uses their own.

Account: Guests can make quotes and previews. An account keeps orders and prices in a private space.

Code

sunbal822 / KARSAZ_Stitch_Desk

KARSAZ Stitch Desk

An order assistant for a crochet business. Upload a customer's reference photo and message, pick the colours on the photo, and get a price, a ready date, a before/after preview, a reply draft and a quote card you can send.

What it does

Photo and message – upload the customer's photo and paste what they wrote.
Colour change? – say whether the customer wants different colours.
Pick the colours – click each part of the photo and choose its colour (palettes, swatches or a colour picker).
Size, deadline, details – age or size, quantity, due date, extra changes.
Quote – price, ready date, before/after picture, a reply you can copy, and a downloadable quote card.

Other tabs:

Orders – search and filter your orders, change status (Quoted, Pending, In progress, Done, Cancelled), see counts and the month's booked value. Overdue and at-risk orders are flagged.
Prices –…

View on GitHub

A quick map of the project:

app.py is the web app, with the five steps and the Orders, Prices and Account tabs.
quote.py holds the pricing and ready-date formulas.
pipeline.py turns a request into a quote and a reply message.
extract.py reads the photo and message with Gemma.
recolor_v5.py does the part-by-part recolouring.
orders.py and accounts.py handle the order log, deadline checks and sign-in.

How I Built It

Open models do the seeing and the reading. The hardest part of a custom order is turning a reference photo and a vague message into something clear. Three open models do that work:

Gemma 3 (4B), running locally through Ollama, looks at the photo and reads the customer’s message to work out which item it is and what they asked for.
ISNet cuts the item out of its background, so the app can find its parts.
Segment Anything lets the seller click a part of the photo and get its exact outline.

I gave the models the jobs they are good at, seeing and reading. Prices, weights and dates come from plain Python, so a small model on a laptop is enough, and its mistakes can never reach a customer’s quote.

Job	Open piece
Reading the photo and the customer’s message	Gemma 3 (4B) through Ollama, running locally
Cutting the item out of its background	ISNet (`isnet-general-use`) through rembg
Click-to-select on a part of the photo	Segment Anything through rembg (optional, local)
Recolouring	OpenCV, NumPy and Pillow
The app	Gradio
The order log	An ordinary Excel file written with openpyxl
Hosting	Render, with a persistent disk

Three design decisions are worth explaining.

Colours come only from the customer’s words. The model’s guess at colours is thrown away. A small parser reads the message (“red flower and green leaves”) and ties each colour to the part it names. If the customer wrote no colours, there are none. Nothing is guessed, so the preview can’t drift from what the customer asked for.
A quote is a formula, not a guess. Each catalogue item has yarn grams, hours and a number of colours. The quote scales them by size, adds a small increase for every extra change the customer asks for, and then applies the seller’s own rates. Here is the default tulip:

Part	Working	Result
Yarn	80 g × Rs 225 per 100 g	Rs 180
Labour	5 hours × Rs 50	Rs 250
Extra colour	1 extra × Rs 20	Rs 20
Cost		Rs 450
Price	Rs 450 × 1.4 markup, rounded to the nearest Rs 50	Rs 650
Ready date	5 hours ÷ 3 hours a day, rounded up, plus 2 spare days	about 4 days

For wearables, the age in months scales the yarn and hours along a size chart, and the reply shows the matching height and chest in centimetres.

The deadline check counts work you have already agreed. The ready date isn’t today plus hours. It starts after your agreed orders. When a new order would push another one past its date, the app says so in plain words, and the Orders tab flags overdue and at-risk jobs. It works through jobs earliest deadline first, which is how a human would plan the week.

Recolouring that still looks like yarn. A flat colour fill looks like paint. The “natural lighting finish” keeps the photo’s own light and shadow on the new colour, so the preview looks like real yarn. Parts can be found three ways: automatically by grouping the item’s pixels by colour, by clicking with Smart selection, or by drawing the corners of a shape by hand. A hand-drawn edge follows the drawn line exactly.

What went wrong.

The first version only worked on my laptop, with one set of orders. Making it multi-user meant giving each account its own folder for orders and prices, with passwords hashed using PBKDF2.
My first deploy ran out of memory on the smallest paid server. The cause wasn’t the model I expected: even with Smart selection off, cutting the item out of its background loads a second model. I turned Smart selection off by default on the website and gave the server more memory.

What’s next: a password reset, remembering the sign-in after a page refresh, and a repeat-order shortcut for returning customers.

Why Does Open Innovation Matter?

Privacy: A customer’s reference photo and message are personal. Run locally, they never leave the seller’s computer. (The hosted demo does upload photos to its server; the local version is the private one.)
Cost: Margins on handmade work are thin. There are no per-photo fees for background removal or for reading a message. A small model on a laptop costs electricity.
Swappable: The model is one line, MODEL = "gemma3:4b" in extract.py. A seller with a stronger machine can try a bigger one.
Fixable: When the app picks a wrong part of a photo, I can read the code and fix it.

My First User

I wanted someone who actually sells crochet to try it. A friend of mine runs her own crochet business and asked to stay anonymous. She became the first person outside my laptop to use Karsaz Stitch Desk.

She had a real customer who wanted a white tulip in a different colour. Normally that means guessing, crocheting, or hours of scrolling on Pinterest, and hoping the customer likes it. Instead she uploaded the photo and tried pink, light blue and purple. Eventually she landed on lilac with green leaves.

Then she wanted to see how far it could actually go. So she uploaded a child frok to see how it goes. This is the result.

Then to take it up a notch she uploaded a bunny and recoloured the ears, arms and legs separately.

I asked what she'd pay. Nothing yet, because this was a trial. But she said she'd consider a monthly subscription if it costs less than Adobe-type tools, mainly to keep her crochet work in one place.

She also found a weakness. With very blurry or very dark photos, the Segment Anything selection picks objects poorly. It finds roughly the right thing, but not cleanly. The next step is a warning when a photo is too blurry or dark.

It's one seller and one trial, so I'm not calling it proof. But for a tool I built for my own business, it was a good first real test.

Prize Categories

Best Use of Gemma: Gemma 3 (4B) runs through Ollama on the seller’s own computer, reading the customer’s reference photo and message locally.
Best Use of Render: Render hosts the live app at karsaz.tech and deploys it automatically from GitHub. A persistent disk keeps each seller’s accounts, orders and prices safe, and Render runs the ISNet background-cutting step. Gemma and Smart selection run on my own computer.
Best Use of a .tech Domain: Karsaz_Stitch_Desk is live at https://karsaz.tech, so anyone can open it and try it without installing anything. The .tech domain gives the project a short, memorable address that fits what it is: a tool built for sellers, with the app served from Render under that name.
Best Use of ElevenLabs: The narration for the demo video was generated with ElevenLabs. I wrote the script myself and used an ElevenLabs voice to deliver it, which let me produce a clear, consistent walkthrough of the full flow, from the customer’s photo and message to the seller’s order.

Next time a customer says “that’s not the green I meant,” I want it to be before the first stitch, not after the last.

PIP: a desktop penguin that loves you but hates it when you stay on the desktop for too long

Sunbal Ghayas — Sun, 11 Oct 2026 10:46:00 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

Pip is a small, chronically online penguin who lives on your Windows desktop. His life mission is to get you off the screen.

He tracks how long you've really been at the computer:

30 minutes: he raises a hand and nudges you to go outside.
60 minutes: he turns red and starts steaming.
5 minutes after that: he locks your screen until you pick a quest and prove you did it.

Quests

Outdoors: bring Pip a leaf, a flower, a pebble or some actual grass, take a brisk lap around the block, or sit in the sun (or look at the sky if it's night).
Indoors: dance, cook, stretch, tidy a corner, or call someone you like.
Pip is weather-aware. If it's raining, too hot or freezing, he swaps the outdoor quest for an indoor one and tells you why.

Proof, so you can't just click "done"

Item quests: you hold the object up to the camera and a local vision model checks it. Screens and printed pictures are rejected, and old photos can't be reused.

Activity quests: you have to actually be away from the screen. Then Pip asks one specific question about what you did, and a judge model checks your answer against the question, the real weather and the time of day. Saying "it was sunny" at 9pm gets you a stern look.

The rest of Pip

He grows bigger with every finished quest and unlocks accessories (scarf, party hat, cape, crown).
You can customize Pip in many colours, so you feel more connected to him.
Eye-rest (20-20-20), blink and posture reminders from the webcam.

A to-do list he checks in on, a journal he writes in his own voice after each quest, and a stats dashboard with streaks.
Voice chat, with offline listening.

Demo

Field Test

I took Pip outside and tried to fool him. It's harder than it sounds.

The leaf quest
Pip told me to go for a walk and bring him a leaf.

Before I left, I tried showing him a leaf-pattern picture. He said no, that's not a real leaf.
I went out, did the walk and came back with a piece of bark. Rejected.
Then I showed him an actual leaf, and only then did he unlock the screen.

The old-photo test
I also tried showing him an old picture instead of going out. He rejected it, because he remembers the pictures he's already accepted and won't take the same one twice.

Trying to trick the vision model

Pebble quest: I showed him a seashell that looked a lot like a pebble. I expected it to slip through, but he rejected it.
Grass quest: I held up a bunch of leaves piled together. He still rejected it. He worked out that they were leaves, not grass.

What surprised me: I thought a 4B model running locally would be easy to fool, especially with look-alikes. It wasn't. It told the difference between a leaf and bark, a pebble and a shell, and grass and a pile of leaves.

The result: the only way to get my screen back was to get up, go outside and bring back the right thing.

Code

sunbal822 / PIP-the-desktop-penguin-that-makes-you-touch-grass

Pip: the desktop penguin that makes you touch grass

Pip is a small, chronically online penguin who lives on your Windows desktop. His one mission is to get you off the screen.

He tracks how long you've really been at your computer. At 30 minutes he raises a hand and nudges you. At 60 minutes he turns red and starts steaming. Five minutes later he locks your screen until you pick a quest and prove you did it.

Everything runs locally: the language model, the vision model, speech recognition and the camera checks.

Features

Quests: outdoor (bring Pip a leaf, flower, pebble or grass, walk around the block, sit in the sun) and indoor (dance, cook, stretch, tidy, call someone).
Weather-aware: if it's raining, snowing, too hot or freezing, Pip swaps outdoor quests for indoor ones.
Proof of quest:
- Item quests: hold the object up to the camera and a…

View on GitHub

Setup instructions are in the README. You need Windows, Python, and Ollama with gemma3:4b.

How I Built It

Everything runs on my own Windows PC in a single Python file, with no paid API.

Piece	What it does
Gemma 3 (4B) via Ollama, open-weight	Pip's personality and replies, the answer judge, and his diary entries
Gemma 3 vision (same model)	Looks at the photo you hold up and labels it (leaf, flower, pebble, grass, cup...)
Vosk	Offline speech-to-text, so I can just talk to Pip
OpenCV	Checks whether a face is in view (the "are you really away" test), blink rate and posture, all in memory
pyttsx3	Offline voice (with an optional neural voice if `edge-tts` is installed)
Open-Meteo	Free weather, no API key
Tkinter	Draws the see-through penguin and the whole UI

The main lesson from using a small local model: code decides, the model talks. A 4B model is a great character but a bad referee. So the timers, the screen lock, sound effects and the weather sanity check are plain code. Gemma only writes Pip's lines and judges answers inside strict JSON prompts. Verification is layered too: the photo label, then a duplicate-photo check, then screen and printed-picture rejection. If the judge ever crashes, it doesn't punish the user.

Why Does Open Innovation Matter?

Pip watches me through a webcam, listens through a mic, knows my to-do list and writes a diary about my day. I would never want to stream that to a server I don't control.

Privacy: the language model, vision model, speech recognition and face/blink/posture analysis all run on my machine. The background camera monitor never saves frames. The only photos kept are the ones I deliberately capture as quest proof, and they stay in a local folder.
Free to run: a pet that talks to me all day and looks at my photos would add up to a real bill on a closed API. Locally it costs electricity.
I can tune it: because I own the model, I can re-roll when it repeats its own examples, force JSON judging, and swap in a bigger Gemma with one line if the photo check gets sloppy.
Works offline after setup: chat, photo checking and voice input need no internet once everything is installed (Vosk downloads its small speech model once, the first time). Only the weather and the optional neural voice need internet.

Prize Categories

Best Use of Gemma: Pip's brain runs on Gemma 3 (4B) locally through Ollama. Gemma writes Pip's personality and replies, judges whether your answer to his question really holds up, writes his diary entries, and (using its vision ability) checks the photo you hold up to the camera. If the photo check is sloppy, I can swap in a bigger Gemma by changing one line.