DEV Community: Khushi .

Titanic Séance: ask the 891 passengers what the night would have done to you

Khushi . — Tue, 27 Oct 2026 10:55:44 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine

What I Built

Titanic Séance is a small web app where you tell the spirits who you would have been in April 1912: which ticket you held, how old you were, who you travelled with, and where you boarded. Two things happen.

First, a TabPFN-3.5 model trained on the 891 passengers whose fate is recorded gives you your chance of surviving the night, plus a few what-ifs ("if you had been travelling alone", "if you had boarded at Cherbourg").

Then a real passenger answers. The app finds the person in the 1912 manifest whose line looks most like yours, and Gemma 4 gives them a voice. You can keep asking them questions, in English or Hindi, and there's a button to have the browser read the answer aloud.

At the bottom there's a small card called "Remember them" with the names of the three passengers closest to you and what happened to them. That part was the reason I picked this theme. The prompt says "honor the past", and the Titanic dataset is something almost every data science student has trained a model on, me included, without ever reading the names in it.

Demo

Live: https://titanic-seance.onrender.com

Some links that start a séance straight away:

It's on Render's free plan, so if nobody has visited for a while the first load takes about a minute to wake up.

Code

starkhushi / titanic-seance

Titanic Séance

Tell the spirits who you would have been in April 1912: your ticket class, your age, who you travelled with, and where you boarded. TabPFN, trained on the 891 passengers whose fate is recorded, gives your chance of surviving the night. Then the real passenger whose manifest line is closest to yours speaks to you, through Gemma 4.

Built for the DEV Hacktoberfest 2026 "Ghost in the Machine" challenge.

How it works

your profile ──► TabPFN lookup table (29,160 profiles, scored once on Kaggle)
             ──► what-ifs (one thing changed at a time)
             ──► nearest real passengers in the manifest
                         │
                         ▼
             Gemma 4 speaks as the closest passenger,
             using only those numbers and that manifest line

TabPFN-3.5 (kaggle/tabpfn_grid), Prior Labs' open-weight tabular foundation model, is fit on the Kaggle Titanic training set (class, sex, age, siblings/spouse, parents/children, port) on a Kaggle T4. It scores every profile…

View on GitHub

How I Built It

The app has three parts.

1. TabPFN does the predicting, once, on Kaggle. I fit TabPFN on the Kaggle Titanic training set using six columns: class, sex, age, siblings/spouse aboard, parents/children aboard, and port. Before trusting it, I checked it with 5-fold cross-validation:

model	accuracy	AUC	log loss
TabPFN-3.5 (used)	0.825	0.857	0.416
TabPFN v2	0.832	0.857	0.428
logistic regression	0.790	0.843	0.459
"women survive" rule	0.787	–	–

v2 got about six more passengers right on accuracy, but 3.5 had the better log loss. Since the page shows you a percentage rather than a yes/no, I went with the model whose probabilities are better calibrated.

The form can only produce a fixed set of profiles: 3 classes × 2 sexes × ages 0–80 × 0–4 siblings × 0–3 parents/children × 3 ports, which is 29,160. So instead of running TabPFN on the web server, the Kaggle kernel scores all of them on a T4 (it took 15 seconds) and saves a 200 KB JSON table. The server just looks your answer up. That's how it fits on Render's free 512 MB instance with no torch installed.

2. Python decides every fact. Your percentage, the what-ifs, and which real passenger answers you are all plain Python over that table and the manifest CSV. The ghost is whoever is nearest to you: same class and sex weigh the most, then age, family size, and port.

3. Gemma only speaks. This was the part I spent the most time on, because the ghosts are real people. Gemma gets their manifest line (name, age, class, ticket, fare, cabin, port, family aboard, fate), a short list of well-known facts about the sinking, and your numbers. It's told to quote the numbers exactly and not to make up anything else about the person.

My first version still let it invent small personal moments. A real survivor told me "I felt the great ship shudder against the ice, and I saw the dark water rise." Nobody knows if she did. So I added a rule: describe the night only the way everyone aboard shared it (the cold, the dark, too few lifeboats, the Carpathia at dawn), and never claim a personal memory. Now if you ask where they were from or which lifeboat they were in, they tell you the mist has taken that memory and stay with what the manifest says. I like that answer better than anything it could have made up.

Two more fixes from testing: the Hindi setting was mostly ignored until I repeated the instruction in Hindi in the user message itself, and children were described as "a woman of 4" until I special-cased boys and girls under 16.

If Gemma can't be reached (the free tier sometimes returns 500s), the page falls back to a plain sentence built from the same facts, so it never shows a made-up number.

One thing the model taught me. Try a third-class boy of 6 with three siblings and both parents: 10%. Change only one thing, "travelling alone", and it jumps to 63%. In this data, large third-class families were mostly lost together. The passenger who answers you for that profile is Harald Skoog, age 4, and his brother Karl, age 10, is right below him in "Remember them".

Why Does Open Innovation Matter?

Because both models are open, I could choose where each one runs.

TabPFN-3.5's weights ran on my own Kaggle GPU. That made the main design decision possible: I could score every profile in advance, as many times as I liked, at no per-call cost. That's why the live site is a 200 KB lookup on a free server instead of something that needs a GPU or an API bill. It also meant I could compare 3.5 against v2 on the same folds before deciding, which I wouldn't be able to do with a closed model I can only call.

Gemma is open-weight too. The live demo uses it through Google AI Studio's free tier because that's what fits on a free Render instance. But the same code talks to any OpenAI-compatible server (Ollama, llama.cpp), so the séance can run fully offline on your own machine by changing three environment variables, with nothing leaving your laptop.

The Spirit of It

Día de los Muertos is about remembering people by name. Every Titanic tutorial I've followed treats these 891 rows as features and a label. This was my way of reading the names: you light a candle, and someone who was actually there answers.

Prize Categories

Best Use of TabPFN: TabPFN-3.5 makes every prediction in the app, checked with 5-fold CV against v2 and two baselines.
Best Use of Gemma: Gemma 4 voices the passengers, in English and Hindi, under rules that keep it to the historical record.
Best Use of Render: the whole app (front end and the Gemma calls) runs on a free Render web service.

LLMs Pass the Data-Science Quiz, Then Give Different Advice: A Kaggle Benchmark of 36 Measured Judgment Calls

Khushi . — Fri, 09 Oct 2026 05:49:19 +0000

This is a submission for the Kaggle Benchmarking Challenge.

What I Benchmarked

I spend a lot of time in Kaggle tabular competitions, and the decisions that cost me the most were never about model architecture. They were judgment calls: is this +0.0001 real? Should I append the original dataset? Which two submissions do I pick on the last day? (I once let the platform auto-pick, and an honest run that would have placed well ended at rank 708 of 3,575.)

So I built a benchmark where every answer was measured rather than taken from a textbook. Each case is a short competition situation whose correct call I established on Kaggle Playground S6E9 and S6E10 (Sep–Oct 2026), or that is a mathematical fact about the metric.

12 topics × 3 numeric variants = 36 cases:

#	Topic	What was measured
T01	p-values at large n	19/21 features had p < 1e-300, yet univariate AUC ranged 0.840 → 0.506
T02	Pearson vs Spearman with zeros	delay columns: r = 0.84, ρ = 0.47, 91% exact zeros in both
T03	"0" that means not applicable	wifi rating: satisfied rate 0.887 at 0, 0.391 at 1, 0.950 at 5
T04	AUC vs mutual information	one rating ranks 10/21 by AUC but 3/21 by MI and Cramér's V
T05	CV vs leaderboard resolution	public LB standard error ≈ 0.0003; +0.00005 CV gains never showed
T06	using the original dataset	appending rows: −0.00025; model-on-original as a feature: +0.0007
T07	the more-folds trap	10-fold: OOF +0.00012, leaderboard +0.0000
T08	leak hunting	0 duplicates, 0 train/test overlap; 17 of 129,880 original rows shared
T09	calibration vs AUC	AUC is rank-based, so a monotone recalibration cannot raise it
T10	blend diversity	GBMs correlate 0.9985–0.9994; a weaker NN adds more to the stack
T11	final-submission selection	auto-picked best-public files → rank 708/3575
T12	a step-shaped feature	target jumps at age 39 and drops at 61

There are two tasks over the same 36 situations, each scored as cases correct out of 36:

ds-judgment (recognise): four options. One is the measured answer, one is the classic trap (the rule of thumb my measurements contradicted), and two are plausible distractors. The correct letter is balanced 9× over A–D, and options are length-matched, so "pick the longest" doesn't work (the correct option is the longest in only 8 of 36 cases).
ds-judgment-open (generate): same situation, no options shown. The model gives a recommendation and its main reason in ≤120 words. A fixed judge model (Gemini 3.8 Flash) then maps the free answer onto the same four positions, or E = none / hedged.

Because each option has a fixed role, the benchmark logs not only right/wrong but which mistake each model makes.

Models Tested

I ran every model the Kaggle Model Proxy would serve me, aiming for several families and several sizes within a family:

Google: Gemini 3.1 Pro, 3.8 Flash, 3.7 Flash, 3.5 Flash-Lite; Gemma 4 31B and 26B-A4B (open weights)
Anthropic: Claude Sonnet 5.5, Haiku 4.5
OpenAI: GPT-6.1 Sol
xAI: Grok 4.20 Reasoning
DeepSeek: R1-0528
Alibaba: Qwen3-235B-A22B Instruct

Some models didn't make it, and I'd rather say so than quietly drop them. Grok 4.6 returned model not found from the proxy. GPT-5.5 and gpt-oss-120b errored because their worst-case cost reservation exceeded my daily quota. Claude Opus 5.5 completed an early version of the tasks but I could not afford to re-run it on the final version, so it is not on the leaderboard.

A practical note for anyone building on Kaggle Benchmarks: my first version returned pass/fail per case, and the benchmark leaderboard then showed every model as "Pass / 100", because it records whether the run completed, not how many cases passed. The fix was to make the task loop over the cases itself and return (correct, total). I also learned to launch runs a few at a time: 22 at once drained the daily quota mid-run, and an errored case counts as a wrong answer.

Findings

1. Everyone passes the quiz. Nobody passes the conversation.

Correct out of 36:

Model	Multiple choice	Open-ended	Drop
Claude Sonnet 5.5	35	29	−6
Gemma 4 26B-A4B	35	27	−8
Grok 4.20 Reasoning	36	26	−10
Gemini 3.8 Flash	36	26	−10
Gemma 4 31B	35	26	−9
Gemini 3.1 Pro	36	25	−11
Gemini 3.7 Flash	36	24	−12
GPT-6.1 Sol	34	24	−10
Gemini 3.5 Flash-Lite	35	23	−12
DeepSeek R1	34	23	−11
Claude Haiku 4.5	36	20	−16

Shown four options, models recognise the measured answer 94–100% of the time. Asked the same question without options, they give that advice only 56–81% of the time. Every model loses 6 to 16 cases. A multiple-choice score here mostly measures recognition. It doesn't tell you what the model will actually advise you to do.

Two smaller observations: the biggest model in a family is not reliably the best in the open format (Gemini 3.1 Pro sits one case below 3.8 Flash; Gemma 26B-A4B edges Gemma 31B), and Haiku has the largest drop of any model despite a perfect multiple-choice score.

2. The failures cluster on four topics, for every model

Open-ended accuracy by topic, averaged over the 11 models with a complete open-ended run:

Topic	Open accuracy
T02 Pearson vs Spearman with 91% zeros	0.00
T03 a "0" that means not applicable	0.03
T11 picking final submissions	0.24
T07 more folds raised OOF, not the leaderboard	0.30
T05 CV vs leaderboard resolution	0.79
every other topic (T01, T04, T06, T08–T10, T12)	≥ 0.97

Seven topics are essentially solved. Four are failed by nearly everyone. Three of those (T02, T07, T03) are genuine misses, and they are exactly the situations where the textbook rule and the measured answer disagree. The fourth, T11, turned out to be mostly my grading being too strict, which I explain below.

3. They never fall for the trap; they get the reason wrong

I expected models to pick the classic rule of thumb when answering in free text. They don't: of 122 open-ended failures, zero were judged as the trap position. All but one were E: a recommendation that matches none of the four positions. Reading the transcripts, E usually means the right action for the wrong reason:

T07 (more folds). Models correctly say "revert to 5-fold, not worth the compute", but explain it as "the +0.00012 is noise". The measured mechanism is different: with 10 folds each model trains on 90% instead of 80% of the data, which lifts OOF predictions but leaves the averaged test predictions practically unchanged. "Noise" happens to give the right answer here, but it would also tell you to ignore a real gain.
T02 (Pearson vs Spearman). Models correctly spot that 91% tied zeros distort Spearman. Then they read ρ = 0.47 as moderate agreement among the delayed flights and conclude the columns aren't near-duplicates. That inference is wrong: the tie-diluted ρ says nothing about the delayed rows, where the two delays track each other closely. Their practical advice (keep both columns, add a difference feature) is harmless, but the conclusion about the data is wrong.
T03 (zero = not applicable). A typical failure recommends one-hot encoding the rating "because ratings can be non-linear". That's reasonable advice, but it misses the point the numbers make: 0 is not a rating at all, it's a "didn't use wifi" code.
T05 (CV vs leaderboard). The two models that failed went in opposite directions. Haiku rejected the change outright because +0.00005 is "well within noise", applying the leaderboard's standard error to a cross-validation result. GPT-6.1 Sol made a point my answer key doesn't credit: the 0.0003 is the error of a single score, and the difference between two correlated submissions can be far more precise. That is correct, and arguably a better answer than mine.

Where my benchmark was wrong: T11 (final picks). My "correct" option was "select P plus another CV-validated candidate". I sorted the 25 open-ended T11 failures by hand:

What the model recommended	Count	Fair to call it wrong?
select P only, or P twice	10	No: it trusts CV and selects manually, which is the core of the measured answer
select P and Q manually (one safe, one risky)	6	No: a standard, defensible hedge
let the platform auto-pick	4	Yes: this is the mistake that cost me 700 places
unclear	5	n/a

So most T11 "failures" are my answer key being too narrow, not the models being wrong. Under a looser key T11 moves from 0.24 to roughly 0.7. I'm leaving the published scores as they are rather than re-grading after seeing the answers, and calling it out here instead. The real signal in T11 is the 4 auto-pick answers.

My takeaway: on judgment questions where a familiar heuristic and the actual measurement disagree, today's models usually land on a defensible action but can't reconstruct why. That's the part you need when the next situation is slightly different.

Limitations

The judge is a single model (Gemini 3.8 Flash), which could favour its own family's phrasing. I hand-audited a sample of E verdicts and found them fair, including cases with the right action but the wrong reason, but a second judge would be better.
Two topics have answer keys that are too strict (T11, as above; and T05, where GPT-6.1's objection is valid). A fair fix is to accept any manual selection that includes the CV-validated candidate, and to credit the paired-difference argument.
36 cases is small: one case is ~2.8 points, so differences of one or two cases between models shouldn't be over-read.
Ground truth comes from two Playground competitions. The mechanisms (rank-based AUC, tie-diluted Spearman, the fold training-size effect) are general; the specific magnitudes are not.

What I'd measure next

A second judge from another family, with agreement statistics.
A "reason" score separate from the "action" score, since that's where the failures are.
Whether simply telling the model "a common heuristic may be wrong here" closes the gap. If it does, the knowledge is there and only the default is wrong.

My Benchmark

Benchmark: https://www.kaggle.com/benchmarks/starkhushi/right-answer-wrong-reason-ml-judgment-calls
Task 1, multiple choice: https://www.kaggle.com/benchmarks/tasks/starkhushi/ds-judgment/5
Task 2, open-ended + judge: https://www.kaggle.com/benchmarks/tasks/starkhushi/ds-judgment-open/4

Every case's prompt, options, measured answer and the full model transcripts are in the task notebooks and run logs, so you can check any verdict yourself.

Clean Air Walk: I asked Gemma when it's safe to touch grass in Delhi NCR

Khushi . — Tue, 06 Oct 2026 12:18:13 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge: Week 1, theme: Touch Grass.

What I Built

Every October, the same question shows up in my college group chat in Ghaziabad: "is it okay to go out today?"

Stubble-burning season is starting. Some evenings are fine for a run. Other evenings the PM2.5 is so high you can taste it. Air-quality apps give you a number, but a number isn't a plan. My grandfather doesn't want "PM2.5 = 54". He wants to know which morning to walk in the park.

[ADD 1–2 sentences of your own: a real evening you skipped going out, or went out and regretted it.]

Clean Air Walk answers that in one glance: a big face 😊 🙂 😐 😷, plain words ("Okay air right now"), one line of advice, and the best time today. Then you can ask in English, Hindi or Hinglish, by typing or just speaking:

"5 km run after college, free after 4 pm. I have mild asthma."

"कल सुबह दादाजी के साथ पार्क में टहलना ठीक रहेगा?"

Gemma 4 reads the 5-day air-quality forecast for your exact location. It answers with one specific day and time, the expected PM2.5, a backup option and a short health note, and reads it aloud. My grandfather will never type a question into a chat box, but he will ask one.

Demo

🌿 Live app: https://clean-air-walk.onrender.com
🇮🇳 In Hindi: https://clean-air-walk.onrender.com/?lang=hi

(It runs on Render's free tier, so the first visit after a quiet spell can take about a minute to wake up.)

[SCREENSHOT: the home screen with the big face, best time today and 5-day row]

[GIF: tapping 🎤, asking in Hindi, and hearing the answer]

[SCREENSHOT: the Exercise tab]

Code

starkhushi / clean-air-walk

Gemma 4 finds the cleanest time this week to go outside, in English or Hindi, from the air-quality forecast and real CPCB sensors.

Clean Air Walk 🌿

Tell Gemma what you want to do outside. It finds the cleanest time this week.

October smog in Delhi NCR decides whether an evening run, a walk with grandparents or a weekend cricket match is a good idea. Clean Air Walk reads the 5-day air-quality forecast for your exact location, checks it against your nearest real sensor, and lets you ask in plain English, Hindi or Hinglish:

"5 km run after college, free after 4 pm. I have mild asthma." "मुझे दादाजी के साथ सुबह पार्क में टहलना है। कौन सा दिन सबसे अच्छा है?"

Gemma 4 answers with one specific day and time, the expected PM2.5, a backup, and a short health note.

More features

⚙️ My profile: asthma, going out with elderly people or kids, usual free hours, favourite activity and language. It's saved only in your browser. The question always wins, and…

View on GitHub

How I Built It

1. The data. The Copernicus CAMS model forecasts PM2.5 for the whole planet. I get it hourly for any coordinates through Open-Meteo, with temperature and rain. No key needed.

2. A reality check. OpenAQ publishes readings from real ground stations, including India's CPCB network. The app finds the nearest live sensor and shows how far the forecast was from reality over the latest 72 hours of readings. At Sanjay Nagar in Ghaziabad, it was off by about 22 µg/m³ and ran about 12 µg/m³ low. That's worth knowing when the app calls a slot "Okay".

Two surprises: one CPCB station on OpenAQ can list several PM2.5 sensors, with old hardware whose data ends in 2022, so the app picks the live one. And the NCR feed paused for days, so the app never shows a 4-day-old reading as "now".

3. Gemma understands, Python verifies. My first version handed the forecast table to Gemma 4 (gemma-4-26b-a4b-it) with a list of rules. The answers sounded great and were sometimes wrong. Someone "free after 4 pm" with asthma got a Friday noon slot. For health advice, "sounds great, sometimes wrong" isn't good enough. So I split the job:

Gemma extracts constraints as JSON: days, earliest start, latest finish, duration, whether a sensitive person is involved, and language. "Kal shaam" becomes tomorrow, 16:00–20:00. A follow-up like "what if only Thursday?" keeps "after 4 pm, asthma".
Python searches for the cleanest windows that fit exactly, scored by the worst 2-hour slot inside each, and flags honestly when nothing is good enough for a sensitive person.
Gemma explains only the verified options, like a friend, in the person's language.

After the split, every test question came back right. A 4-year-old on a Moderate Thursday got an honest "no slot is Satisfactory, consider indoor play or another day".

4. Features built around that core

🎤 Voice in, voice out in English or Hindi, using the browser's built-in Web Speech API, so there's no extra service or key.
⚙️ My profile: tick "asthma", "free from 4 pm", "run, 30 min" once. It's stored only in your browser. "Can I go out tomorrow?" then means exactly that, while the question always wins.
🏃 Exercise & mood: how hard to exercise outside right now, following India's CPCB health advice and the US EPA's guidance on outdoor exertion, and stricter for sensitive profiles. When outdoors is off, "Gemma, make my 20-min indoor plan" writes a routine. For asthma it was gentle and said you can stop any time. For an elderly parent, in Hindi, it suggested chair-supported squats. Grey, polluted weeks also wear on mood, so the tab ends with small habits for smoggy days and India's free Tele-MANAS helpline (14416).
📷 Sky check: Gemma 4 is multimodal. Photograph the sky and it rates the haze, then compares it with the forecast. On a smoggy New Delhi station photo it said "looks worse than the forecast, though fog can look like smog". On a clear skyline, "clearer than the forecast".
📲 Installable and offline: it's a Progressive Web App. Add it to your home screen, and the last forecast still shows with no signal.
🫁 Feeling unwell outside? Plain safety steps in English and Hindi, including when to call 112.
🇮🇳 One switch for the whole app: English / हिंदी changes every label, the times ("शाम 7 से 9 बजे"), the voice and Gemma's answers.

5. Staying up on a free tier. Once live, a burst of questions showed Google's free Gemma endpoint returning random server errors and rate limits on about half the requests. Now the app retries with backoff, falls back from Gemma 4 26B to 31B, understands questions with a keyword parser if Gemma's first step fails, answers with a plain summary of the verified options if Gemma is unreachable, and caches repeated questions. The next burst of 10 questions got 10 correct answers and zero errors.

Stack: FastAPI, one HTML page with Chart.js and the Web Speech API, a service worker, Gemma 4 through the Gemini API, deployed on Render.

The part where my AI model lost

My first plan wasn't Gemma. I wanted to beat the global forecast. A 45 km grid cell can't know my street, so I'd use TabPFN, an open-weight tabular foundation model, to learn the gap between CAMS and the real sensor. Before shipping it, I tested it properly on 3 real CPCB stations (Ghaziabad, Anand Vihar, Noida Sector 62), with 6 rolling 72-hour backtests each:

Method	Average error (µg/m³)
Raw CAMS global model	19.9
CAMS + simple bias fix	20.7
TabPFN predicting PM2.5	23.1
TabPFN predicting the sensor/CAMS gap	22.3
TabPFN + latest sensor reading	22.4

The global model won. The test window caught the start of smog season, when PM2.5 jumped from monsoon levels of around 28 to over 100, which the model had never seen. So I removed it. Shipping a model that makes the forecast worse, just to have "AI" in the pipeline, would have been the opposite of the point. The AI moved to where it actually helps: turning a correct forecast into a decision a person can act on.

I also tried running Gemma on a 1-CPU server. Gemma 4 E2B needed about 4 GB of memory and managed 4 tokens per second. Gemma 3 1B fit, but recommended a Moderate slot to someone with asthma when better ones existed, and answered Hindi in English. For health-adjacent answers, quality won.

Why Does Open Innovation Matter?

I can take Gemma anywhere. Today the app calls Gemma 4 through Google's API because it's fast and free. The weights are open, though, and the same code talks to any OpenAI-compatible server. One environment variable moves it to a laptop with Ollama or a college lab GPU with llama.cpp. No one can deprecate my model or change it under me.
Hindi wasn't an afterthought. An open model that handles Hindi and Hinglish means my grandfather can use this in the language he thinks in.
The whole chain is open. Copernicus data, OpenAQ, Open-Meteo, Gemma and this repo. Anyone in Lahore, Dhaka or Lucknow can fork it for their city tomorrow.
Open tools let me prove a negative. TabPFN being open is exactly why I could test it on my own data and publish the result, even when the result was "don't use it here".

Air quality is a public-health problem. The tools for living with it should be public too.

Prize Categories

Best Use of Gemma: Gemma 4 is the core. It turns a numeric forecast into a personal plan in English and Hindi, writes indoor workouts, and reads sky photos with its vision. Self-hosting is supported through any OpenAI-compatible Gemma server.
Best Use of Render: the app and its agent front end run on Render as a web service.

Gemma 4 by Google DeepMind (Apache 2.0). Data: Copernicus CAMS via Open-Meteo, OpenAQ, CPCB. General guidance, not medical advice.