Lore Lab: I Tested Weather Folklore Against 10 Years of Rain (and Let Gemma Propose, but Never Decide)

Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass Submission 🌿

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

"When the wind turns south, rain is in its mouth."

A traditional saying I kept hearing and never checked.

I never checked. So I built something that does. And when the answer is "just chance", it tells you so.

In 30 seconds:
🎬 Demo video · 🌐 Live app · 💻 Code (MIT)

Why This Exists

Everyone has a saying about the sky. Red sky at night. Cows lying down. Swallows flying low. We repeat them, nod, and move on. Almost nobody tests them, and the tools that could either need a statistics degree or hand you a confident answer with nothing behind it.

I wanted something in between: a tool that takes a saying seriously enough to test it, and honestly enough to say when it fails.

What I Built

Lore Lab turns a folk weather saying into a small, fair experiment, then sends you outside to run it.

Type a saying, in any language. Gemma, a small open-weight model, turns it into something you can check: the sign to look for, how to observe it, what it predicts, and for how long. You review it and set every number yourself. Then one of two things happens:

🔬 Lab Test	👀 Field Test
The sign is something an instrument records: pressure, wind direction, humidity, high cloud.	The sign is a salt shaker, a swift, the look of the moon. No instrument records it.
The saying is scored against ten years of past weather.	You are the instrument. Step outside, tap Seen or Not seen, put the phone away.
Result in seconds.	Lore Lab checks the sky for you later.

Every result sits next to the bar to beat: how often it rains anyway. A saying only means something if it beats chance. "Same as chance" is a real result, and Lore Lab shows it with the same pride as "Promising".

Who is it for? Anyone whose family said things about the sky and who would rather go and look than keep scrolling.

Touch Grass, Literally

This week's theme asks for something that gets people off the screen. Lore Lab is built around one idea: the screen should be the shortest part of the experience.

The Field Test ends on one instruction:

"Logged. We'll check the sky for you. Put your phone away."

You tap once. You don't come back to report. The app remembers when the weather window closes and fills in the outcome later from weather data. The best session with Lore Lab is the one where you open it, look at the sky for real, tap, and close it.

What it was like to actually do it

It was about 04:34 in Delhi, well before sunrise, when I stepped outside to log my first look. The salt in the shaker had not caked, so I tapped Not seen. The app told me to put my phone away, and I did. After that, the sky had to answer on its own.

Demo

🎬 90-second video:

🌐 Try it in 60 seconds: lore-lab.onrender.com
Pick "When the wind turns south, rain is in its mouth" → accept the rough match → choose South → place Delhi → run the Lab Test.
(Free hosting sleeps, so the first load can take about a minute. The hosted demo serves pre-generated hypotheses and has no persistent logbook, so Field Test looks only save in a local install.)

1. The verdict

Delhi, 303 evenings with a south wind. It rained in the next 12 hours after 42% of them (95% interval 37 to 48%), against 19% on any evening. That's about 2.2× chance. The card says Promising, and no more than that. (After a season adjustment, the lift is about 1.4×. More on that below.)

2. The moment the app refused to cheat

Gemma read "When the glass falls low, prepare for a blow" as predicting wind. The Lab only scores rain, so it would not run it. I did not edit the outcome to "rain" to force a result. A tool that bends the question until it gets an answer isn't testing anything.

3. An honest in-progress result

I logged one look for a salt-shaker saying at about 04:34 IST on 11 October, with a 12-hour outcome window. The salt had not caked, so I tapped Not seen. When the window closed, the app resolved it: rain followed, 0.2 mm, exactly at my threshold.

That doesn't make the saying right. It only predicts rain when the sign is there, and I didn't see the sign, so this look doesn't count toward a verdict. It rains on about 19% of Delhi evenings in the ten-year history anyway. The card reads On the trail, 0 of 10 looks, 1 not seen, and it won't show a hit rate until there are ten.

I could have counted that rain as a "hit" and made the app look good. It would have been wrong.

Code

Sam02i / Lore-Lab

Test your family's weather sayings against 10 years of real data with local Gemma. Gemma proposes, code decides. Then go touch grass.

🌿 Lore Lab

Your family's weather sayings, tested against ten years of real data.

Then go outside, look at the sky, and put your phone away.

🎬 Demo video · 🌐 Live demo · 📝 DEV write-up

---

The idea

"When the wind turns south, rain is in its mouth."

Everyone has a saying like this. Almost nobody checks it. Lore Lab checks it, and tells you honestly when the answer is "just chance."

A small open-weight model (Gemma) turns a saying into a hypothesis you can test. You review it and set every number. Plain code scores it, and every result is shown next to the bar to beat: how often it rains anyway.

Gemma proposes. Code decides.

Two ways to test a saying

🔬 Lab Test	👀 Field Test
The sign is something an instrument records: pressure, wind direction, humidity, high cloud.	The sign is something

…

View on GitHub

MIT licensed. The scoring code (backtest.py, lab.py, season_check.py) is plain Python with no model in the loop.

How I Built It

One rule: Gemma proposes, code decides.

Saying ─► Gemma proposes a hypothesis (JSON, schema-validated, closed vocabularies)
       ─► YOU review it and set the threshold   (the model never picks a number)
       ─► plain code decides: Lab Test or Field Test
       ─► Lab:   10-year backtest · Wilson interval · lift · verdict
          Field: look outside ─► Seen / Not seen ─► outcome filled in lazily from weather data

🧠 Gemma, open-weight, local. gemma3:4b runs through Ollama on my laptop. It returns a JSON hypothesis: translation, the sign, how to observe it, the predicted outcome, a proxy weather variable, and any ambiguities. The schema has no threshold field at all, so the model can't choose the number that decides the result. You do, and the server refuses to run a Lab Test without it.

⚖️ Code decides. Whether a saying can be backtested is plain code. The sign must be atmospheric, the proxy variable must be on a whitelist, and the fit must not be "none". A "rough" fit needs my explicit OK. The verdict is arithmetic: Promising only if the whole 95% Wilson interval sits above the bar to beat, Backfires only if it sits below, and Same as chance otherwise.

📜 Rules written before results. The easiest way to fool yourself with data is to choose the rules after you see the numbers. So I committed these constants before looking at any result:

Constant	Value
Minimum sign-present days for a Lab verdict	30
Minimum sign-present looks for a Field verdict	10
What counts as rain	0.2 mm or more
Outcome window	12 hours, from the next full hour
Backtest "look" time	18:00
History	10 years

⏳ Lazy outcome resolution. I first planned a workflow engine, then deleted it. Free hosting sleeps, so a timer can silently never fire. Instead each look stores when its window closes, and every time the app opens, an idempotent function fills in any due outcome from Open-Meteo hourly precipitation. A failed fetch retries up to five times, then the look is marked failed and counted separately, never quietly dropped or guessed.

📏 Measured, not assumed

A model I haven't measured is a model I'm trusting on faith. On my 21-case test set, gemma3:4b with prompt v2.1:

Measure	Result
Valid JSON	21 of 21
Sign kind correct (atmospheric or not)	17 of 18
Backtestable correct	19 of 20

Where it failed. The model labelled "Red sky at night, sailor's delight" as an atmospheric sign, so it came back as backtestable. The code trusts the model's label, so nothing automatic caught it. That's exactly why every hypothesis goes through human review, and why I publish the accuracy instead of hiding it.

First results (Delhi, 12 h window, 0.2 mm or more of rain)

Saying	Sign-present days	Hit rate (95% CI)	Bar to beat	Lift	Verdict
When the wind turns south, rain is in its mouth	303	42% (37 to 48%), 128 hits	19%	×2.19	Promising
Mackerel sky, mackerel sky, never long wet, never long dry	794	47% (43 to 50%), 371 hits	19%	×2.42	Promising
Salt shaker (Field)	0 of 10 looks with the sign (1 not seen; rain followed, not counted)	not shown below 10 looks	19%	n/a	On the trail

Both Lab sayings use a rough proxy that I accepted. Wind within 45° of south stands in for "turns south", and high cloud cover of 80% or more stands in for a mackerel sky. I ran each once and didn't try other thresholds. An earlier exploratory run on falling pressure predates the rules above, so I report it but don't count it.

The monsoon check

I worried that these results were just the monsoon. South winds and high cloud are both more common in July and August, when it rains more anyway. So I built a season-adjusted check, season_check.py, that compares each sign day with the base rate of its own month instead of the overall 19%.

	South wind	Mackerel sky
Hit rate (95% CI)	42% (37 to 48%)	47% (43 to 50%)
Bar to beat, overall	19%	19%
Bar to beat, same month	30%	33%
Bar to beat, same year and month (stricter)	32%	35%
Lift over the overall bar	×2.2	×2.4
Lift over the same-month bar	×1.4	×1.4
Lift over the same-year-and-month bar	×1.3	×1.3
Verdict	Promising	Promising

What it found:

• Part of it was the monsoon. Roughly a third to a half of the original effect disappears once each day is compared with its own month. The lift drops from about 2.2× to about 1.4×.
• The rest held up. The whole 95% interval still sits above the month-matched bar, so both verdicts survive.
• It isn't a July-and-August quirk. Rain followed south-wind evenings more often than other evenings in all 12 months. Mackerel sky did the same in 11 of 12 (December was a tie).

"Promising" is still a hint, not proof. Consecutive evenings aren't independent, so the intervals are probably more optimistic than they look. It's also one city and modelled data. The check controls for time of year, not for everything else that makes rain more likely.

Hosted on Render

Render doesn't run Gemma, so the hosted app serves hypotheses that Gemma generated locally and saved in pregenerated.json. It runs on Render's free plan, so there's no persistent logbook. The Delhi weather history ships as a cache file in the repo, because the free host's shared IP hit Open-Meteo's daily request limit. It's the same history my local run downloaded, and the hosted result matches my local one. My real looks live on my local app.

🎙️ The video

The narration was generated with ElevenLabs text-to-speech from a script I wrote, so the voice says exactly what the app does and nothing more.

What I Learned

• Constraining the model made it more useful. The schema has no threshold field and closed vocabularies. The model could not "help" in the one place where help would have corrupted the result.
• Refusing is a feature. The falling-glass saying and the unseen salt look both ended with the app declining to count something. Those moments are what make the verdicts worth reading.
• Checking my own result made it smaller and more believable. The season adjustment cut the lift from 2.2× to 1.4×. A weaker claim that survives a fair test is worth more than a bigger one that doesn't.
• A small model plus a human step beats a big model alone. gemma3:4b got 17 of 18 sign kinds right, and the one miss is exactly what human review exists for.
• Free hosting forces better design. Because the host sleeps, I replaced a timer with outcomes that resolve whenever the app next opens.

What's Next

Running the same season check on a second city (Pune's weather history is already in the repo), a comparison against a second Gemma size, a Dolbear's-law (cricket chirp) test, photos with a look, offline mode, and a daily look card. I would rather ship a small honest tool than a big vague one.

Limits (please read)

• Reanalysis (modelled data), not weather-station readings.
• One place, ten years: a hint, not proof.
• Consecutive evenings aren't independent, so the confidence intervals are probably narrower than they should be.
• The season adjustment controls for time of year, not for every other cause of rain.
• The proxies only approximate the sayings.
• Lore Lab does not prove a saying true and does not forecast better than meteorologists.
• Field samples from a few days of building are tiny, and the app says so.
• Not built: a second-city check, a second Gemma size comparison, Dolbear's law test, photos, offline mode, a daily look card.
• The artwork in the app was collected from Pinterest. I don't know the original artists and I don't claim rights to it.

Why Does Open Innovation Matter?

• 🌍 Sayings live in many languages. An open model can be run, tested and swapped for the languages that matter. I measured it on a test set instead of trusting a black box.
• 🔒 A family's saying never has to leave the device. Local inference keeps the proverb on the laptop.
• 🧪 Model choice becomes a variable. I tested the model, reported where it was wrong (the red-sky miss), and designed around it: a schema it can't write a threshold into, and a human review step.
• 💸 The whole pipeline runs for free, on a laptop plus a free hosted demo. A closed API would have put a bill and a data policy between a grandmother's proverb and the person who wanted to check it.

Prize Categories

• 🟢 Best Use of Gemma: Gemma 3 4B, run locally through Ollama, is the extraction step. Its JSON is schema-validated, scored on a 21-case test set, and never allowed to pick the threshold.
• 🟣 Best Use of Render: live at lore-lab.onrender.com on Render's free plan, serving Gemma's pre-generated hypotheses and running the Delhi Lab Test.
• 🔵 Best Use of ElevenLabs: the narration of my demo video was generated with ElevenLabs text-to-speech from a script I wrote.

The Takeaway

Most AI demos try to look right. I wanted Lore Lab to be willing to say "this is just chance." The model suggests, you decide the numbers, plain code scores the result, and the app always shows how often it rains anyway.

The wind did turn south. After taking the monsoon out, rain was still about 1.4× more likely than chance in Delhi. That's a hint worth chasing, not a proof, and now I know where to look next.

What's a saying your family swears by? Drop it in the comments and I'll run it.

Close the tab. Look at the sky. Tap once. Put your phone away. 🌿

Built by **Samiksha Tripathy* · GitHub: Sam02i*

Comments

[Vaibhav Ghai]

This is such a creative and thoughtful project! I really like how you combined weather folklore with 10 years of real rainfall data, while using Gemma as a tool to suggest rather than blindly decide. A great example of using AI responsibly with actual data.

[Samiksha Tripathy]

Thank you so much !!