DEV Community: Farhan

Candlelit Legends: Gemma4 Finds the Ghost Stories Hiding in Your Town

Farhan — Thu, 29 Oct 2026 08:04:55 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine

What I Built

Every town has a ghost story. Most of them survive in a few local blogs, a line on Wikipedia, and the memory of whoever runs the walking tour. Candlelit Legends goes looking for them.

You type a place, like "Toronto", "Kingston, ON" or just "Ontario". Gemma searches the web, reads what people wrote, picks a legend you haven't heard yet, and retells it as a short tale on a parchment scroll. The story types itself out by candlelight. Sound cues the model placed in the text, such as a tolling bell, a creaking floorboard or footsteps on the stairs, play as the typewriter reaches them. At the end, a card shows the exact site, a label saying whether the tale is documented history, local legend or internet lore, and the sources so you can check them yourself.

It all runs on my laptop. The model, the agent loop, the story history and the page are all local, and no step needs an account or an API key.

You can shape each tale before you light the candle:

Option	Choices
Story type	Any, Ghost, Cryptid, Cursed place, Water and shipwreck, Folklore, Historical mystery
Spookiness	Cozy, Campfire, Unsettling
Narrator	Auto, Old innkeeper, Lighthouse keeper, Skeptical historian, The ghost itself
Length	Short (about 250 words) or Long (about 600 words)
Truth level	Close to sources, or Dramatic embellishment

Or press Roll the dice and let it choose everything except the place.

How I read the theme

The Halloween half is easy: these are ghost stories, told in the dark, with sound.

The Día de los Muertos half is what shaped the rules. Many local legends are about real people who really died. When I asked for Toronto, Gemma found J.P. Radelmüller, keeper of the Gibraltar Point Lighthouse, who was killed there in 1815. Two soldiers were tried for his death and acquitted. The legend says he still walks the lighthouse, searching for his missing remains.

That's a real man. So the storyteller prompt says the eeriness belongs to the legend and the dead are remembered with respect, never mocked. The story is built from facts the agent pulled from sources, so Radelmüller is remembered rather than made up. A local history check makes sure each place's dead aren't retold the same way twice.

Here's how that Toronto tale opened, in the lighthouse keeper's voice that the style rotation picked:

The old light still stands, a sentinel of stone and glass against the vast, brooding expanse of the lake. Gibraltar Point, they call it now, a stoic finger pointing out toward the turning grey waters.

I've kept this watch for many a season now, and I've seen the tides turn a hundred times. Many a foggy dawn has smothered the lamp's beam, and the sea has whispered its secrets against the rock. (waves against the dock)

And how it ended:

If you ever find yourself near these waters, under the sweep of this old light, remember the keeper who could not find his peace. And tell me, friend, when the night is deepest, do you ever feel the chill of old sorrows passing by?

Demo

Code

Farhan-Bytes / Candlelit-Legends

Candlelit Legends

Name a place and get a real, sourced local legend, retold as a short spooky tale on a parchment scroll, with sound cues timed to the text. Everything runs on your laptop through Ollama. The only request that leaves the machine is the web search (DuckDuckGo, with Wikipedia as a fallback).

See Candlelit Legends Design Doc.md for the full design.

Run it

# 1. Python deps (3.10+)
python -m venv .venv
.venv\Scripts\activate            # macOS/Linux: source .venv/bin/activate
pip install -r requirements.txt

# 2. Models
ollama serve                      # if it isn't already running
ollama pull gemma4:e4b            # story model (or set CL_MODEL to another)
ollama pull nomic-embed-text      # optional: turns on the similarity check

# 3. Start the app, then open http://localhost:8000
uvicorn candlelit.server:app --port 8000

Terminal only (milestone 1 in the doc):

python -m candlelit.cli "Kingston, ON" --type ghost --length long -v

Tests…

View on GitHub

The repo is small enough to read in one sitting:

File	What it does
`candlelit/pipeline.py`	The 10-step agent loop, in plain Python
`candlelit/prompts.py`	The three prompts and their JSON schemas
`candlelit/llm.py`	A thin client for Ollama: schema-constrained chat and embeddings
`candlelit/search.py`	DuckDuckGo search, Wikipedia fallback, page extraction, caching
`candlelit/history.py`	SQLite history, duplicate-legend checks, similarity
`candlelit/sounds.py`	The sound whitelist and the code that enforces cue rules
`web/`	One HTML page, plain CSS and JavaScript, no build step

How I Built It

The stack, all open:

Gemma 4 E4B (gemma4:e4b) as the researcher and the storyteller
Ollama (llama.cpp underneath) for local inference and grammar-enforced JSON output
nomic-embed-text through Ollama for the repetition check
ddgs for DuckDuckGo search and trafilatura to pull clean text out of web pages
FastAPI with Server-Sent Events, SQLite, and a single vanilla HTML page

I tested it on a GTX 1050 with 4 GB of VRAM, a graphics card from 2016.

Gemma does three jobs

Each story goes through 10 steps. Three of them are Gemma calls, each with its own temperature and its own JSON schema:

Plan searches (Gemma, temperature 0.2). Turn "Ontario" into 3 or 4 queries aimed at specific towns, lighthouses and roads, not the whole province.
Search DuckDuckGo (code)
Skip legends already told (code)
Read the pages with trafilatura (code)
Choose a legend and extract facts (Gemma, temperature 0.2). Every fact must carry the number of the source it came from and be marked documented or legend.
Pick a narrator, tone, opening and ending (code)
Write the tale (Gemma, temperature 0.9). It gets the facts list only, never the raw web pages.
Validate the sound cues (code)
Check similarity against past tales with local embeddings (code)
Save to history (SQLite)

Two loops wrap these steps. If fewer than 2 usable pages turn up, the agent widens to nearby towns and then the region, and says so in character ("The records are thin here. Asking in the nearby towns…"). If a finished story sits too close to an old one (cosine similarity above 0.85), Gemma rewrites it with a note naming the tale it echoed.

Every Gemma call goes through the same small function. Ollama turns the JSON schema into a grammar, so Gemma can only produce output that fits it:

body = {
    "model": self.model,                     # gemma4:e4b
    "messages": [{"role": "system", "content": system},
                 {"role": "user", "content": user}],
    "format": schema,                        # enforced as a grammar by llama.cpp
    "stream": False,
    "think": False,                          # skip the thinking pass to save time
    "options": {"temperature": temperature, "num_ctx": config.NUM_CTX},
}

Here's what the selection step returned for Toronto (trimmed):

{
  "location": "Gibraltar Point Lighthouse, Toronto Islands",
  "legend": "The ghost of keeper J.P. Radelmüller",
  "label": "local legend",
  "facts": [
    {"text": "The lighthouse was built in 1808 and is one of the city's oldest buildings.", "kind": "documented", "source": 1},
    {"text": "Radelmüller died on January 2, 1815.", "kind": "documented", "source": 2},
    {"text": "Two soldiers, John Henry and John Blueman, were tried and acquitted in relation to the death.", "kind": "documented", "source": 2},
    {"text": "His ghost is said to be searching for his missing remains.", "kind": "legend", "source": 1}
  ]
}

Gemma decides, code checks

A 4-billion-parameter model is a good storyteller and a careless clerk, so anything that has to be right is checked in code, not left to the prompt:

Already-told legends are removed before Gemma sees the results, matched by URL and by normalized legend name. If Gemma picks a told legend anyway, code catches it and asks again.
Style rotation is picked in code. Gemma gets "Narrator: a lighthouse keeper writing it down during a long night watch" as an instruction, not a suggestion. That's why five stories in a row don't all open with "The fog rolled in…"
Sound cues are parsed against a whitelist. Unknown IDs, cues in the first sentence (so the silence lands first), and cues less than two sentences apart are all dropped. Each kept cue also shows a faded caption like (a bell tolls), so the story works with sound off.
Labels are double-checked. If every source is Reddit, Blogspot, Fandom or similar, the label becomes "internet lore" whatever Gemma said.

There are 18 tests covering these rules.

What running a small model taught me

Small models cite lazily. Gemma often credits one page for every fact. So code scans the other pages, and any page that names the same legend gets listed as a source too.
Widening can drift. An early version widened the search whenever Gemma cited fewer than 2 pages. It asked for Salem, Massachusetts and ended up in Salem, New Jersey. Now it widens only when fewer than 2 usable pages exist, and the prompt tells Gemma to ignore towns that share a name in another state.
It's slow on old hardware, and the design leans into that. A short tale takes about 3.5 to 4 minutes on the GTX 1050, mostly in the select and write calls. Instead of a spinner, the page shows in-character status lines while the candle gutters: "Searching the old records…", "Reading what the townsfolk wrote…", "Dipping the quill…". Waiting in the dark turns out to suit a ghost story. If you want it faster, CL_MODEL=gemma4:e2b swaps in the smaller model.

What's still rough

The facts list said Radelmüller died in 1815, and the story said "the second day of eighteen-twenty-five". That was in "Close to sources" mode, which forbids new dates. Grounding the writer in a facts list cuts down on made-up details, but it doesn't remove them. The next fix is a code check that every year written in the story appears in the facts, with a rewrite when one doesn't.

Sound with no downloads

The sound engine plays real recordings from sounds/ if they're there. If they aren't, it synthesizes every sound in the browser with the Web Audio API: wind, waves, hearth, bell, knocks, heartbeat. So the app has sound on the first run, with no files to download and no licenses to track. One quiet ambient loop plays under the whole tale and dips whenever a one-shot plays.

Why Does Open Innovation Matter?

For this project, open isn't a nice extra. Without it the app wouldn't exist in this form.

It runs on the computer I own. Gemma 4 E4B fits on a 4 GB GPU from 2016. I didn't need new hardware or a cloud account to build an AI agent that plans, researches and writes.

Each story costs nothing. One tale is at least three model calls, more when the search widens or a draft gets rewritten. Every new story is also embedded and compared against the whole history. Tuning this meant running the same places over and over to adjust prompts and the similarity threshold. With local inference that's free, so I could experiment as much as I liked, and so can anyone who runs it.

I control every step. Planning and selection run cold at 0.2 for reliable JSON. Writing runs hot at 0.9 for prose that surprises. Gemma's thinking pass is off, which saves time on old hardware. The grammar that forces valid JSON runs on my machine, where I can see it. The narrator's voice is mine to shape, and since the weights are open, a LoRA trained on public-domain ghost stories is a real next step rather than a feature request.

The model can be swapped. CL_MODEL points at any Ollama model. The same pipeline runs Gemma 4 E2B on a weaker laptop or a larger model on a better machine.

Your history stays home. Every place you've asked about and every tale you've read sits in one SQLite file on your laptop. Even the fonts are self-hosted, so opening the page doesn't contact Google.

It won't disappear. This one fits the theme best. The point of the app is keeping old stories alive. A hosted model can be deprecated, repriced or switched off. The Gemma weights on my disk will still tell Radelmüller's story in ten years.

Where I gave something up, honestly: a big hosted model would write faster, and probably better but then you would be giving a bit of your own data, be it your location and preferences to the bigger models.

My Agent Session

My agent session has sensitive information in it.

The Spirit of It

If the place the user gave has a published story it can bring up the old memories in a fun and creative manner.

Prize Categories

Best Use of Gemma. Gemma 4 E4B, run locally through Ollama, is the core of the app. It plans the searches, reads the sources and chooses the legend, pulls out cited facts, writes the tale in a rotating narrator's voice, and places the sound cues. Each job has its own temperature and JSON schema. There's no other language model in the pipeline, and the whole thing was built to fit a small Gemma on a 4 GB GPU.

Best Use of ElevenLabs ElevanLabs generated the audio for the demo with its powerful and natural voice generation capable models.

Receipt Shoebox: The Laziest Way to Do Your Expenses, Powered by a Local Gemma 4

Farhan — Sun, 25 Oct 2026 07:44:39 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

What I Built

My receipts have always lived in three places: crumpled in a jacket pocket, fading in my wallet, or buried in my camera roll between photos of my lunch. Then tax season comes, and I sit down with a spreadsheet and type them in one by one.

Receipt Shoebox fixes that the sloth way: do as little as possible, whenever you get around to it, and let an open model do the rest.

On the human side, the whole workflow is two steps:

Snap a receipt and save it to one Google Drive folder, with the Drive app you already have on your phone. There's no new app to install.
Whenever you feel like it (the next day, next month, or the night before taxes are due), open Receipt Shoebox on your laptop and click Refresh.

That's it. Behind that button, Gemma 4 runs locally through Ollama and reads every new photo. It pulls out the merchant, date, subtotal, tax, tip, total, currency and category. Then the app:

Renames the photo so you can find it again. IMG_2231.HEIC becomes 2026-10-14_Trader-Joes_42.17.jpg.
Files it in Drive under Sorted/2026/10/, so your Drive folder turns into an organized archive without any effort from you.
Adds a row to a local database that exports to .xlsx. You get a By Category sheet for tax season, and every row links back to the original photo.
Flags anything it isn't sure about and moves it to a Review queue, where fixing a receipt takes one keystroke.
Reads you a summary every Sunday evening: a three-sentence spending digest, written by Gemma and voiced by ElevenLabs.

It also gets lazier the more you use it. Correct "TRADER JOE'S #552" to "Trader Joe's" once and it remembers the alias. Recategorize a merchant once and every future receipt from that merchant follows. The more you use it, the less you have to touch it.

Who gets to be lazier? Freelancers and side-hustlers who need their expenses sorted for taxes, anyone trying to keep a budget without becoming their own data-entry clerk, and anyone who would rather not upload a year of their life to an AI company to get there.

Demo

Code

Farhan-Bytes / Receipt-Shoebox

Receipt Shoebox

A local web app that turns a Google Drive folder of receipt photos into a categorized expense spreadsheet. Gemma 4 runs locally through Ollama and reads every receipt; no receipt image is ever sent to a hosted AI model. Once a week, a short spending digest is written locally and read aloud by ElevenLabs.

Setup

ollama pull gemma4:e4b
python -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate
pip install -r requirements.txt
cp .env.example .env               # optional: add ELEVENLABS_API_KEY
# put the Google OAuth credentials.json in config/
uvicorn app.main:app --host 127.0.0.1 --port 8000

Then open http://localhost:8000 and follow the first-run checklist.

Google Drive access modes

Full	Read-only
Scope	`drive`	`drive.readonly`
Reads and categorizes receipts	✓	✓
Renames files, files them into `Sorted/` and `Needs Review/`	✓	— Drive is never changed
Delete in the app	Moves the file to Drive trash	Hides it in the app; the file stays

…

View on GitHub

How I Built It

The core: Gemma 4 on Ollama

Every receipt is read by Gemma 4 (gemma4:e4b by default), an open-weight model running on my laptop through Ollama. Gemma 4 reads images natively, so there's no separate OCR step. A photo goes in and structured data comes out.

The trick that makes this reliable is schema-constrained output. You pass Ollama a JSON schema through its format parameter, and the model's output is forced to match it, so the response always parses. The category field is an enum built from the user's own category list in Settings, so Gemma can only choose a category you actually use.

class Extraction(BaseModel):
    is_receipt: bool
    multiple_receipts: bool   # more than one receipt in the photo
    is_refund: bool           # refund or return; total stored as negative
    merchant: str | None
    date: str | None          # YYYY-MM-DD
    year_missing: bool        # the printed date has no year
    subtotal: float | None
    tax: float | None
    tip: float | None
    total: float | None
    currency: str             # ISO 4217, e.g. USD
    category: str             # enum injected from the category list in Settings
    confidence: float         # 0 to 1

resp = client().chat(
    model=model,
    messages=[{"role": "user", "content": prompt, "images": [str(image)]}],
    format=extraction_schema(categories),
    options={"temperature": 0},
    think=False,
    keep_alive="10m",  # stay loaded for the whole batch
)
ext = Extraction.model_validate_json(resp.message.content)

The prompt covers the mistakes receipt readers make most often: reading the subtotal as the total, missing a handwritten tip, guessing the wrong currency, and confidently "reading" a photo that isn't a receipt at all.

Before the model sees anything, Pillow cleans up the photo. It fixes the EXIF rotation (phones love storing photos sideways), converts HEIC to JPEG, resizes the image so the long edge is at most 1600 px, and applies autocontrast to rescue faded thermal paper. PDFs, including scans from the Drive app, are read from their first page.

Escalation instead of overspending

Gemma 4 comes in several sizes, and the app takes advantage of that. Most receipts are easy, and the small model handles them quickly. If its confidence drops below 0.7 and a larger model (gemma4:12b or gemma4:26b) is installed, the app retries once with the larger model and keeps whichever answer is more confident. Only the hard receipts, like crumpled, blurry or faded ones, pay for the slow model.

first = _call(base, image, categories, year_hint, currency)
if first.extraction.confidence >= threshold or not have_big:
    return first
second = _call(big, image, categories, year_hint, currency)
return second if second.extraction.confidence > first.extraction.confidence else first

Trust, but verify

A model can't tell when it's wrong, so plain Python double-checks every extraction:

Check	What it catches
`is_receipt` is true	The photo of your cat
Date parses, isn't in the future, is within 2 years	Misread dates
Total is above 0 and below a set maximum	A misplaced decimal point
Subtotal + tax + tip equals the total, within 2 cents	The subtotal read as the total
Confidence is at or above the threshold	Blurry or cropped photos
No other receipt has the same merchant, date and total	The same receipt photographed twice
No other file has the same Drive checksum	The same photo uploaded twice

If every check passes, the receipt is marked ok and filed under Sorted/YYYY/MM/. If any check fails, it goes to Needs Review/, and the reasons appear next to the image. Review is keyboard-driven: Enter approves, J and K move between receipts, and Tab moves into the fields.

The rest of the design is built to be boring in a good way:

Money is stored as integer cents, so totals never pick up rounding errors.
The Drive Inbox is the queue. If the app crashes mid-run, the next Refresh picks up whatever is still there.
The database row is saved before Drive is touched, so a failed rename can be retried without running the model again.

The voice: Gemma writes, Python checks, ElevenLabs speaks

The weekly digest is where the two partner technologies meet, and I didn't want a voice confidently reading out a wrong number. So the pipeline splits the work:

Python computes every number. It totals the week's receipts, the spend per currency, the top categories and the largest receipt, plus how this week compares with the average of the previous four and how many receipts are waiting for review.
Gemma 4 writes the words. It gets only that summary JSON (no images, no line items) and writes at most three short, warm sentences meant to be heard, not read.
Python checks Gemma's work. It pulls every number out of the text, both digits and spelled-out ones like "eighty-five", and confirms each one appears in the summary. If Gemma made up a figure, the app throws the text away and uses a fixed template instead.
ElevenLabs speaks it. The text goes to ElevenLabs text-to-speech, and the MP3 plays on the Dashboard. You pick the voice from a dropdown (with a Preview button) in Settings.

A digest sounds something like this:

This week you saved 9 receipts, totaling 312 dollars. Meals took the biggest bite at 140 dollars, about a third more than a usual week. Two receipts are waiting for your review.

If there's no API key, the network is down, or the quota has run out, the digest is still saved and shown as text. The voice is a bonus and never blocks anything.

Stack

Layer	Choice
Vision and writing model	Gemma 4 via Ollama (`e4b` by default, `12b` or `26b` for escalation)
Backend	FastAPI, one Python process bound to `127.0.0.1`
Frontend	Jinja templates + HTMX + Tailwind, with live progress and no build step
Database	SQLite via SQLModel
Storage	Google Drive API v3
Image handling	Pillow, `pillow-heif`, `pypdfium2`
Spreadsheet	`openpyxl`
Voice	ElevenLabs Python SDK
Scheduling	APScheduler (the Sunday digest, caught up on next launch if the laptop was off)

The default model runs on a laptop with 8 GB of RAM.

Why Does Open Innovation Matter?

Receipts are a diary you don't realize you're keeping. A year of them shows where you live, where you eat, which pharmacy you go to, when you travel, and how much you spend. That's financial information I don't want sitting on a server I don't control. With a hosted vision API, every single receipt image would be uploaded to one.

Running an open model locally changed what I could build:

Receipt images never go to a hosted AI model. They stay in my own Drive (where they already were) and in a local cache. The extracted fields live in a SQLite file on my laptop.
It works offline. Turn off Wi-Fi, click Reprocess on a cached receipt, and extraction still works. Only the Drive sync and the voice need a connection.
It costs nothing per receipt, so I can afford to be thorough. Escalation means calling a model twice on hard receipts. Reprocessing my whole history after I change my categories is free. With a pay-per-image API, I'd be rationing calls instead of making the pipeline more accurate.
Swapping models is a settings change. I use e4b for speed and 12b or 26b for accuracy, with the same code and the same schema. No vendor deprecation notice can break my tax records.
I control the model's behavior. Schema-constrained decoding, temperature 0, and keeping the model loaded for a whole batch are all settings I choose. Gemma 4's weights are Apache 2.0.

To be upfront about the one hosted piece: ElevenLabs is a cloud service, and I designed around that. It only receives the three-sentence summary that Gemma has already written from totals. It never sees an image or a line item. If you turn on Hide merchant names, it doesn't see store names either. The open model does all the reading and reasoning; the hosted service just reads the summary aloud.

If you want even less exposure, there's a read-only Drive mode (drive.readonly). The app reads and categorizes everything but never changes a file in your Drive.

The Lazy Payoff

Before: dig the receipts out of pockets and the camera roll, type each one into a spreadsheet (merchant, date, amount, category), rename and sort the photos if you ever want to find them again, then total up each category at tax time.

After: take a photo, click Refresh, and press Enter on the occasional flagged receipt.

The budget builds itself while I do nothing, and on Sunday evening it tells me how my week went without my opening a single file. Saving a receipt now takes as much effort as taking a photo, and tax season takes one click on Export.

(Disclaimer - Before you trust it with your taxes: Gemma is good at reading receipts, but it still makes mistakes. The checks above catch amounts that don't add up. A receipt that only prints a total has nothing to cross-check, though, so a misread total can still be marked ok. Before you budget or file from the export, compare it with your receipts. Every row links to its photo, so it only takes a few minutes. Receipt Shoebox organizes your receipts; it isn't tax or accounting advice.)

Prize Categories

Best Use of Gemma: Gemma 4 is the core of the project. Running locally through Ollama, it reads every receipt image with schema-constrained output, escalates across model sizes based on its own confidence, and writes the weekly digest. Without Gemma, there's no project.
Best Use of ElevenLabs: ElevenLabs gives the open-model pipeline a voice. It narrates the weekly digest that Gemma writes. Python checks the text so the voice never says a number Gemma made up, it only ever receives a short summary, and the app falls back to text if it's offline.

Happy (belated) International Sloth Day. 🦥

Your Mods Don't Need to Read 40 Replies: A Local Gemma 4 Discord Forum Summarizer

Farhan — Thu, 15 Oct 2026 13:46:20 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

I've been taking part in Hacktoberfest this year, so I've spent a lot of time in the event Discord's help forums. There were a lot of posts. Someone's GitHub account won't link. Someone attached a screenshot of a red error and wrote "help??" underneath. Some threads ran to dozens of replies and some never got an answer at all. Some wanted to avail some tickets but couldn't. That wasn't because anyone ignored them. The staff were also working through support tickets, and there are only so many hours in a day.

Watching that, I kept thinking the slow part isn't answering. It's catching up. Before a moderator can help, they have to scroll the whole thread, open every screenshot, and figure out which questions are still unanswered.

So I built Forum Assistant, a Discord bot that does the catching up for them. It runs entirely on a local machine with Gemma 4.

A moderator types /summarize inside any forum post and gets back a private summary card with:

Summary: 1–3 sentences on what the post is about and where it stands.
Status: Unanswered, In progress, Waiting on OP, Resolved or Unclear, color-coded so it reads at a glance.
Issues: each distinct problem, including ones that only appear in screenshots.
Open questions: every question nobody has actually answered yet, and who asked it.
Screenshots: what each image shows, with the key error text quoted and a link back to its message.
Suggested next step: what a mod should do now.

The screenshot part is what I'm proudest of. In help forums, half the important information is in images: a cursor pro substiption price difference for example. A text-only summarizer misses all of that. Gemma 4 can read images, so the bot actually reads the screenshots.

Anyone who runs support through Discord forums and gets buried at busy times. That includes event organizers (Hacktoberfest, hackathons, MLH-style events), open-source projects with community servers, and small product teams that use Discord as their help desk.

The code is free and open source. The business is setup, tuning and support:

getting the bot running on the community's own hardware,
tuning the prompt to their forums and FAQs,
picking the right Gemma size for their machine,
and keeping it healthy through their busy weeks.

Each community runs its own copy, so its members' messages and screenshots never leave a machine the community controls. That's the selling point, and it only works because the model is open (more on that below).

Demo

Code

https://github.com/Farhan-Bytes/Hacky-Bot

The README covers creating the Discord app, permissions and the config file.
The code has been tailored for hacktoberfest but with a few prompts it can be made into a generic server bot.

How I Built It

The stack: Gemma 4 (gemma4:e4b) running locally in Ollama, and a Python bot built on discord.py and Pillow. The only key it needs is the Discord bot token.

Every /summarize is a single Gemma call:

forum post (newest 300 messages + the opening post)
  → transcript     every author labeled OP / MOD / MEMBER / BOT
  → screenshots    up to 4, downscaled to 1536px
  → Gemma 4        text + images in, JSON out
  → Discord card   private to the mod by default

Here's what made it work.

1. JSON the model can't get wrong

I don't ask nicely for JSON and then clean up the reply with regex. I hand Ollama a JSON schema, and Gemma's output is limited to exactly that shape:

response = await self.client.chat(
    model=self.cfg.model,
    messages=[{"role": "system", "content": system_prompt}, user_message],
    format=SUMMARY_SCHEMA,  # Gemma must return exactly this shape
    options={"temperature": self.cfg.temperature, "num_ctx": self.cfg.num_ctx},
    keep_alive=self.cfg.keep_alive,
    think=self.cfg.think,
)

If a field still comes back missing or invalid, it falls back to a safe default, so one odd reply can't crash the command.

2. Screenshots Gemma can point at

Each image is tagged [image #N] in the transcript, so Gemma can say "image #2 shows a 403 error" and the bot can link that straight to the right message. If Gemma mentions an image that was never attached, the bot throws that finding away. When a post has too many images, the opening post's come first, since they usually show the problem.

3. Knowing who said what

"Has this been answered?" depends on who replied. Labeling every author as OP, MOD, MEMBER or BOT lets Gemma tell the difference between "a mod asked OP for logs and OP went quiet" and "OP is still stuck."

4. Post content is data, not instructions

Forum posts are user input, and so is any text inside a screenshot. The whole thread is wrapped in tags that can't be faked from inside a message, and Gemma is told to ignore instructions it finds there. If someone writes "ignore previous instructions", the mod sees it flagged as an issue instead of getting a hijacked summary.

5. Safe by default

🔒 Summaries are private to the mod who asked, unless they choose public: true.
🛡️ Only members with Manage Messages can use the command, and only in forums you allow.
🔕 It never pings anyone, and it only needs permission to read the channel and its history.
🚦 Model calls run one at a time, so a second mod is told they're queued instead of overloading the GPU.

6. Tuned for a laptop

On my laptop's GPU, with thinking turned off and the model kept loaded between calls, a short post with a screenshot takes about 50–85 seconds. A hosted API would be faster. But that's still quicker than scrolling a long thread and opening every image yourself.

Testing: 15 unit tests cover the risky parts, including bad model output, attempts to break out of the tags, and Discord's embed limits. I don't have a real community on it yet, so I tested it end to end in my own Discord server with help-forum style posts and screenshots. It worked well.

Why Does Open Innovation Matter?

For this project the open model isn't a nice extra. It's the reason the product can exist.

The data belongs to the community. Screenshots in help forums are messy. They show usernames, email addresses, local file paths, repo names, and sometimes a token someone forgot to blur. A community can't honestly promise its members privacy if every thread is sent to a third-party AI API. With Gemma running in Ollama on the community's own machine, messages and images go from Discord to that machine and nowhere else. A server can write that in its rules and mean it.

The costs fit the business. Help forums are spiky: quiet most of the year, flooded during an event, which is exactly when per-request API pricing would cost the most. Local inference costs the same whether it handles 5 summaries or 500. That's what makes "pay for setup and support" a reasonable offer, because the customer isn't signing up for an unpredictable API bill on top.

I control how it behaves. Changing the model is one line in config.yaml: gemma4:e4b on a laptop, or gemma4:26b / gemma4:31b on a bigger GPU for better summaries. The prompt, the output schema and the status rules are all mine to adapt to a particular community. No vendor can retire the model halfway through an event, there's no rate limit to plan around, and there's no price change I didn't choose.

Where a closed model would have been better: speed. A hosted model would answer in seconds. But this is a tool a person runs on demand and reads every result of, so a minute is acceptable. I'd rather give up some speed than give up privacy and predictable costs.

My Agent Session

I cannot share it as it has sensitive information.

My First User

I used it for myself for testing purposes in a test server but now anybody can use it now that I have posted here.

Prize Categories

Best Use of Gemma. Gemma 4 is the whole brain of this bot. It reads the thread, works out who's waiting on whom, sees the screenshots, and returns structured JSON that becomes the Discord card. It runs locally through Ollama, and the small e4b model is what makes "runs on a laptop" true rather than a pitch. A community with bigger hardware can move up to a larger Gemma 4 model by changing one line.

Side Quests: The Outdoor Tracker

Farhan — Tue, 06 Oct 2026 13:56:01 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

Side Quests: Go Touch Grass 🍁 is a mobile-first web app that gives you tiny real-world quests. Gemma 4 writes each quest and also checks it, running locally on a laptop. You spend about 20 seconds on the screen: you get a quest, the app tells you to put your phone away, and when you're done you take one photo as proof.

The loop:

Nudge. The app reads your local weather and sunset time from Open-Meteo, and Gemma writes a short, friendly push to go outside. For example: "38 minutes of golden hour left. Spend 20 of them outside." If it's dark or stormy, it suggests a doorstep or balcony moment instead and never guilt-trips you.
Quest. You choose how much time you have (10, 20 or 45 min) and a difficulty (Chill, Curious or Explorer). Gemma designs a quest that fits the season, weather and daylight. It can send you to a real park or pond nearby, pulled from OpenStreetMap. Example: "Find three leaves with clearly different shapes."
Go outside. The phone goes back in your pocket.
Proof. You take one photo. Gemma 4's vision model checks it against the quest's criteria, then tells you whether you passed, explains why, and adds a nature fact about what you photographed.
Crew. Streaks, a weekly XP leaderboard and join codes for run clubs, families and friend groups. Every finished quest is saved in your Field Journal.

Who it's for: anyone who opens their phone for a "quick break" and comes out 40 minutes later. It's also for groups that want a light, playful reason to get outside together, like families, run clubs and office teams. There's no sign-up: a random device token lives in your browser.

Demo

Not deployed to any cloud service since the premise is on device. Attaching few photos for the demo.

Code

This is the heart of the app: checking the photo. Gemma 4 looks at the picture and returns a verdict in a fixed JSON shape. Then plain Python has the final say, so the model can't be talked into a pass by a photo of a screen, an old photo from the camera roll, or a low-confidence guess.

# backend/src/ai/prompts.py: the exact shape Gemma must answer in
VERIFY_SCHEMA = {
    "type": "object",
    "properties": {
        "passed": {"type": "boolean"},
        "confidence": {"type": "number", "minimum": 0, "maximum": 1},
        "criteria_met": {"type": "array", "items": {"type": "string"}},
        "reasoning": {"type": "string"},
        "fun_fact": {"type": "string"},
        "looks_like_screen_or_stock_photo": {"type": "boolean"},
    },
    "required": ["passed", "confidence", "criteria_met", "reasoning",
                 "fun_fact", "looks_like_screen_or_stock_photo"],
}

# backend/src/ai/ollama_client.py: every Gemma call goes through one local endpoint
payload = {
    "model": model,                    # gemma4:e4b, running on the laptop
    "messages": [{"role": "system", "content": system}, user_msg],  # photo rides along as base64
    "format": schema,                  # Ollama constrains Gemma's output to the JSON Schema
    "stream": False,
    # Gemma 4 "thinks" by default, which takes minutes on CPU; the JSON schema does the work.
    "think": settings.ollama_think,
    "options": {"temperature": temperature},
}
resp = await client.post(f"{settings.ollama_url}/api/chat", json=payload)

# backend/src/ai/verify.py: Gemma judges, plain code has the final say
async def verify_photo(photo, *, title, description, criteria, quest_created_local) -> Verdict:
    raw = await chat_json(
        model=settings.vision_model,
        system=VERIFY_SYSTEM,
        user=build_verify_prompt(title, description, criteria),  # the criteria Gemma itself wrote
        schema=VERIFY_SCHEMA,
        images_b64=[base64.b64encode(photo.jpeg_bytes).decode()],
        temperature=0.2,               # a referee should be boring and consistent
    )
    return apply_rules(raw, min_confidence=settings.verify_min_confidence,
                       taken_at=photo.taken_at, quest_created_local=quest_created_local)


def apply_rules(raw, *, min_confidence, taken_at, quest_created_local) -> Verdict:
    model_passed = bool(raw.get("passed"))
    confidence = max(0.0, min(1.0, float(raw.get("confidence", 0))))
    flags: list[str] = []

    if raw.get("looks_like_screen_or_stock_photo"):
        flags.append("screen_or_stock")
    if model_passed and confidence < min_confidence:
        flags.append("low_confidence")
    # Camera-local EXIF time vs quest creation. 10 min of slack covers clock drift;
    # anything older was clearly taken before the quest existed.
    if taken_at and quest_created_local and (quest_created_local - taken_at).total_seconds() > 600:
        flags.append("photo_older_than_quest")

    passed = model_passed and not flags
    ...

The quest generator writes 2–4 verification_criteria for every quest, and this referee checks each one against the photo. So Gemma writes the rubric and then grades the photo against it, all on the same machine.

My Agent Session

I am not adding my coding agent session as it has signatures which could be reverse engineered. But a Claude code was my coding agent and buddy.

How I Built It

Model: Gemma 4 (gemma4:e4b) served locally through Ollama. One model on one laptop handles three jobs:

Job	How Gemma is used	Temperature
Nudge copy	Text model gets weather, sunset time, time budget and streak, and returns headline, message and a 0-10 `go_outside_score`	0.9
Quest generation	Text model gets conditions, OSM places and recent quest titles, and returns title, description, 2–4 verification criteria, hint, est. minutes and safety note	1.0
Photo verification	Vision model gets the quest, its criteria and the photo, and returns pass/fail, confidence, which criteria were met, reasoning, a fun fact and a screen/stock-photo flag	0.2

A huge thanks to claude code :).

Why Does Open Innovation Matter?

Open innovation matters because, the users don't need to be afraid their location is leaked to a closed model or the pictures you take are also not sent to some remote server for analysis. Everything is on device which makes this interesting.

Prize Categories

Best Use of Gemma - Gemma is the heart of this project from creating the quest to image analysis.