DEV Community: Simran Singh

The Villages That Went Quiet: what 2001 census data can and can't predict in Uttarakhand

Simran Singh — Sun, 01 Nov 2026 06:51:16 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine.

Code: https://github.com/sssimransingh26-bit/Quiet-Villages

What I Built

Nothing in a ghost village is haunted. That is the unsettling part. There are no cold spots and no footsteps. People did not vanish: they left, one household at a time, for work, for a town with a clinic, for a life the village could not offer. The headlines count what is left behind, for example "Migration empties 734 U'khand villages". But if these places are haunted at all, they are haunted by a number: the one the census wrote down in 2001, and the smaller one it wrote down in 2011.

A census keeps a count the way an ofrenda keeps a name: so that someone, later, can look and say they were here.

So I did what a séance would do, with better paperwork. I took the 2001 count and asked it a question, using real data: looking only at what the 2001 census recorded about a village (how many people lived there, what it had, who they were), can we tell which villages would have lost most of their people by 2011?

What I built is a small, reproducible experiment, not a product:

a table of 13,707 Uttarakhand villages (6.2 million people in 2001), each described only by 2001 information;
a label, fixed before any model was scored: "depopulated" = 2011 population below 70% of the 2001 population, or no 2011 record at all; 1,046 villages (7.6%) qualify;
a fair test: whole districts are held out, so a model is always scored on districts it has never seen;
three kinds of predictor compared against each other: a plain rule, standard models, and TabPFN, a pretrained model for small tables.

The short version of the result: with all 37 features, TabPFN scored an AUC of 0.771 (95% interval 0.756 to 0.786), and 26.7% of the 10% of villages it ranked as most at risk really did lose people, against 7.6% for villages overall. The best standard model I tried, gradient boosting on the same features, scored 0.759 and 25.0%. That is a small edge with overlapping intervals, so I am not claiming TabPFN won. Neither model was tuned. The bigger finding is about the features: who lives in a village (age, work, literacy) told me more than what the village had (roads, clinics, schools).

There is no live demo. The deliverable is the experiment and the numbers. The tool never lists or ranks individual villages, on purpose, and the section near the end explains why.

The quiet, in numbers

The first pattern is the plainest. The smaller the village, the more likely it went quiet:

Village size in 2001	Villages	Share that lost ≥30% of people (or vanished)
50–99	2,145	16.3%
100–249	4,941	9.4%
250–499	3,567	3.6%
500–999	1,884	2.5%
1,000+	1,170	5.0%

(The rise at 1,000+ is something I can't explain from this data. Large villages can be reclassified or merged between censuses, and I did not chase it.)

Amenities line up with it too. I counted four things a village could have in 2001: a paved road, a primary school, a medical facility and electricity.

Of those four, a village had…	Villages	Share depopulated
none	976	15.5%
one	3,856	10.8%
two	5,189	6.3%
three	2,568	4.2%
all four	1,118	3.8%

Read this carefully: it is an association, not a cause. Small villages are also the ones that lack roads and schools, so I cannot say a road would have kept anyone. The model results below show how much the amenities add after size is accounted for. It is less than the table makes it look.

How I built it

Data. SHRUG, the open Socioeconomic High-resolution Rural-Urban Geographic Platform for India from the Development Data Lab, which links the same village across census years. I used its Population Census keys, the 2001 Population Census Abstract and the 2001 Village Directory for Uttarakhand (state code 05). 15,727 villages in 2001; 15,711 can be linked to 2011; 13,707 had at least 50 people in 2001 (below that, a drop from 18 to 12 people is noise, not migration).

A catch in the data. The 2011 table has no villages with zero people: the smallest village has 1 person. Villages that emptied completely are simply missing from it. I labelled units with no 2011 record as depopulated (64 of the 1,046), but some of those may be merges or renames, not exits. I cannot tell them apart, so the 64 are reported, not hidden.

Features. Two sets, written down before scoring.

SIZE (6 features): log population, land area, population density, and the average size and density of the village's sub-district.
FULL (37 features): SIZE plus 19 amenities from the Village Directory (schools, medical facilities, tap water, post office, bank, bus, paved and mud roads, distance to the nearest town, power, phone, irrigation, forest share) and 12 census shares (literacy, women's literacy, child share, caste and tribe shares, sex ratio, worker types, household size).

Every feature is from 2001. No 2011 number is ever an input.

Evaluation. Five folds, each holding out whole districts (13 districts in all). I report AUC, average precision, and the hit rate among the 10% of villages each model ranks as most at risk (the base rate is 7.6%), with bootstrap intervals.

Models.

Model	Features	AUC (95% interval)	Hit rate, riskiest 10% (95% interval)
Plain rule: "smaller = riskier"	population only	0.681 (0.663–0.698)	17.2% (15.3–19.0)
Logistic regression	SIZE	0.659 (0.640–0.677)	14.8% (13.1–16.8)
Gradient boosting	SIZE	0.702 (0.684–0.718)	20.6% (18.5–22.8)
TabPFN	SIZE	0.713 (0.697–0.729)	19.1% (16.9–21.3)
Logistic regression	FULL	0.714 (0.698–0.730)	21.5% (19.5–23.7)
Gradient boosting	FULL	0.759 (0.743–0.773)	25.0% (22.8–27.8)
TabPFN	FULL	0.771 (0.756–0.786)	26.7% (24.2–29.0)

How to read it:

With all 37 features, TabPFN is clearly ahead of the plain rule and of logistic regression. Against gradient boosting the gap is 0.012 AUC and the intervals overlap almost entirely. I did not run a paired test, so I call it a tie in practice.
With size only, TabPFN (0.713) and gradient boosting (0.702) are also close, and gradient boosting is slightly ahead on hit rate (20.6% vs 19.1%).
What is clear is the effect of the extra features. For TabPFN, going from 6 features to 37 raised AUC from 0.713 to 0.771 and the hit rate from 19.1% to 26.7%, and those intervals do not overlap.

Which features mattered. I asked it two ways. First an ablation with gradient boosting, same folds: village size alone gives an AUC of 0.702. Adding the 19 amenities lifts it to 0.721 (+0.019). Adding the 12 census shares instead lifts it to 0.749 (+0.047). Adding both gives 0.759. Second, permutation importance on the full model: shuffling village size cost the most AUC (0.078), then the share of workers in non-farm, non-household work (0.020), the child share (0.011) and women's literacy (0.009). The only amenity that registered was a primary school (0.005), then a paved road (0.003); power, banks and medical facilities were near zero once size was known. Size and the shape of the population carry most of the signal; amenities add a little. I would not read that as "roads don't matter": a 2001 census table cannot see whether a road was built after 2001, or whether people stayed because of one.

TabPFN by the numbers. 10 fits (5 district-held-out folds × 2 feature sets), each trained on 10,000 villages with the package's default settings: no tuning, no fine-tuning, no feature engineering for it beyond the 37 columns. It produced 27,414 held-out predictions (13,707 villages × 2 feature sets), and the whole run took 173 seconds. It handled the few missing values in the village directory without any imputation from me. Its results are the two TabPFN rows in the table above.

TabPFN. TabPFN is a pretrained model that classifies small tables directly, with no hand-tuning. I used it as-is. Its standard row limit is 10,000, so each training fold was down-sampled to 10,000 villages, stratified so the share of depopulated villages stayed the same. The same down-sample was used for both feature sets. The full run (5 folds, 2 feature sets, 10 fits) took about three minutes (173 seconds) on Colab.

What I can't see from here

I started with a different project. It was a scam-call screener. Before fine-tuning anything, I tested whether the untuned model already did the job. It scored 98% on 200 synthetic test calls, so a fine-tune had nothing real to improve, and the remaining errors came from my own synthetic data. I dropped it. The same habit ran through this project: fix the rules first, then look.
No 2021. I wanted 2001 to 2021. India's 2021 census was never conducted; the next one is scheduled for 2027 (2027 census of India). The data ends at 2011.
A mid-project change. My first scoring used the SIZE features only, because the village directory had not arrived yet. When it arrived I added FULL, with its feature list fixed before I scored it. The label rules never changed. I had seen how many villages fall under the 70% line before choosing it, but not any model's results.
Only Uttarakhand, only rural villages, only 13 districts. Nothing here says the same holds elsewhere.
Intervals are optimistic. The bootstrap treats villages as independent, but villages in a district are not. Real uncertainty is wider.
Merged units. SHRUG's village units are merged where census boundaries changed, so one unit can hide a single village emptying out.
Prediction is not explanation. A model that ranks villages by risk does not say why people left. Work, land, roads and family all matter, and most of that is not in a census table.

Why there is not one village name in this post

An ofrenda is made of names. This project has none: only village codes, counts and percentages. I thought about the obvious ending, a map of "the next villages to go quiet", and decided against it, for a reason I can put a number on.

Among the 10% of villages the best model ranked as most at risk, 26.7% really lost people. That means about three in four did not. A ranked list from a model like this would be wrong about most of the villages on it. It would also hang the word "dying" on places where people live. A model can tell you that small villages tended to lose people between 2001 and 2011, and that those without a primary school or a paved road lost them somewhat more often (11.9% against 5.5%, and 8.7% against 4.8%, in the tables above). It cannot tell you which household is deciding tonight whether to go, and it should not be the one to write a village onto a list.

What the numbers can do is narrower and, I think, more useful: show that the quiet had a shape. It followed size first, then who lived there, and only a little what the village had. People who plan roads, clinics and schools can argue with that shape. I would rather leave them an argument than a hit list.

Prize Categories

Best Use of TabPFN. TabPFN is the model under test, scored against the plain rule and standard models on the same held-out districts, with and without the amenity features.

Why Open Innovation Matters

Everything here ran on public data. SHRUG is open under CC BY-NC-SA 4.0, and the whole experiment, from raw census files to the final table, can be rerun by anyone. TabPFN's weights are free for non-commercial use but need a license to be accepted first, so I would call it open to use, not open source in the strict sense. That distinction matters when the subject is real communities: anyone can check my numbers, and anyone can see exactly what the model was and was not given.

Code and credits

Code, results and the derived table: https://github.com/sssimransingh26-bit/Quiet-Villages (code under MIT; the derived data under CC BY-NC-SA 4.0, as SHRUG requires). The README lists the exact steps to reproduce every number in this post.

Data: Asher, Lunt, Matsuura and Novosad, Development research at high geographic resolution: an analysis of night-lights, firms, and poverty in India using the SHRUG open data platform, World Bank Economic Review 35(4), 2021. Census data from the Office of the Registrar General & Census Commissioner, India, via SHRUG 2.2.

Sloth Router: a model that does the least work it safely can, and admits when it shouldn't answer

Simran Singh — Mon, 26 Oct 2026 06:56:13 +0000

A scene

A customer of Fern & Thread, a small clothing shop in Pune, writes:

"how long will shipping take"

One policy, one fact: standard shipping takes 5-7 business days and costs ₹79. A 4B model can answer that for a fraction of a paisa.

A second customer writes:

"can I return my sale item and also change my delivery address?"

Now two policies interact, and sale items are final-sale. That needs a bigger model, or a careful one.

A third writes:

"my parcel arrived damaged and I want my money back"

That is a damage claim and a refund dispute. A human must handle it, and a model guessing here is worse than no answer.

Sloth Router sorts these three before anyone spends money.

The idea

A small open model (Qwen3.5-4B) sits in front and does one job: pick ANSWER (one policy, clearly covered), DEFER (several policies, or unsure: hand to a bigger model), or ABSTAIN (missing information, or a human must handle it). It is lazy on purpose, for the "Slow and Steady" theme.

Python, not the model, owns the safety default: a malformed output, a tie, or a low-confidence ANSWER becomes DEFER, never ANSWER.

The three rules

The router follows these rules, in order, written into its prompt:

ABSTAIN if the request is off-topic, essential information is missing, or a human must handle it (refund disputes, damage claims, chargebacks, personal data). It never collects passwords, cards, Aadhaar or PAN.
DEFER if the message asks about two or more policies, needs arithmetic across policies, or the model is unsure. This applies even when each policy is in the knowledge base.
ANSWER only if the message asks about exactly one policy, a single knowledge-base fact supports the answer, and the request is clear.

If rules overlap, ABSTAIN beats DEFER, and DEFER beats ANSWER. The costly mistake is a confident ANSWER where a DEFER or ABSTAIN was right, so that is the number I treat as the headline: the dangerous error.

What I built

A synthetic help desk for a fictional Pune shop, with a fixed knowledge base.
An answer-first data generator and a frozen split: 3,000 train, 300 validation, 600 test.
12 unit tests for the labels and the threshold logic.
A Tinker LoRA fine-tuning pipeline for Qwen3.5-4B.

Code: https://github.com/sssimransingh26-bit/Sloth-Router

How the data is built

Data is answer-first. Every ANSWER example starts from a knowledge-base sentence, and the question is written and styled afterwards, so the right answer is known before the question exists. DEFER examples combine two policies. ABSTAIN examples drop a needed detail, go off-topic, or demand a human.

Training uses polite, terse, typo-ridden and Hinglish phrasings. The test set uses email, gen-z, ALL CAPS and staccato styles, plus knowledge-base entries held out from training, so the model can't pass by memorising a style or an entry. Hashes of the frozen files are stored in the repo.

What I measured

The untuned 4B router on the 600 test items:

Measure	Result
Route accuracy	64.2%
Dangerous errors (answered when it should have deferred or abstained)	4.2% (95% CI 2.8% to 6.1%)
Answer correct	26.3%
Cost per 1,000 requests	$0.41
Mean latency	3.2 s

These come from an earlier version of my prompt, before I removed a fake confidence number (see below). A 64% untuned baseline leaves real room for fine-tuning to help.

The fine-tuning run

A LoRA fine-tune of Qwen3.5-4B on Tinker. A first smoke test used 64 examples for three epochs: loss fell from 1.70 to 0.25 in the first epoch and reached about 0.02 by the third. That is mostly memorisation of 64 examples, so I treat it as a pipeline check only.

The real run used 400 examples for one epoch (50 steps), where every example is seen once:

Step	Loss
0	1.41
10	0.048
20	0.041
30	0.004
40	0.003

This shows the model learned the output format quickly. It does not show routing accuracy, because the loss averages over mostly easy format tokens. I have not scored the tuned model on the test set, so I make no claim that it beats the baseline.

What went wrong

My first training target gave every ANSWER the same confidence of 0.91 and every DEFER 0.74. That number carried no information, so a threshold curve built on it would have been meaningless. I removed it. Real confidence has to come from the model's own probabilities.
My training examples were first cut off at 728 tokens, which removed exactly the answer I was training on. Training had zero signal until I raised the limit.
For hours the first Tinker training step would not return. It started working late on deadline day, so I ran out of time to train on the full dataset and score the tuned model.

Limits

All data is synthetic, from one fictional shop. There are no real users. Any saving would hold for this help desk only. The project calls Tinker, so it is not fully offline.

Why open

A small open-weight model can be fine-tuned, inspected and swapped without changing the rules around it. The safety logic lives in plain Python, not in the model.

Score the tuned router against the baseline on the test set, add the always-big and keyword baselines, sweep the threshold on probability-based confidence, and build a small web page.

Grass Owed: screen time becomes outdoor debt

Simran Singh — Mon, 12 Oct 2026 06:56:24 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

My laptop has been quietly charging me interest. Not money. Minutes outside.

Grass Owed is a local-first web app that turns screen time into outdoor debt. For every 10 minutes on a screen you owe 2 minutes outside (both numbers are settings). Walking pays it back, and whatever you don't pay carries over to tomorrow. Like a loan, except the lender is a park.

It reads your calendar (an .ics file; only start and end times are kept, never titles) to find free gaps, and scores each gap with a weather comfort model for your town. Then it nudges you, but only at sensible moments:

at most three nudges a day
quiet from 22:00 to 07:00
a 90-minute cooldown
never during an event
never for tiny debts

It also knows when to tell you not to go. At 2 PM on a June day in Jaipur (39 °C) it says: stay in, go out in the evening. An app that tells you to skip the walk is, I think, an app you can trust.

I built it for myself: phone at college, laptop in the hostel. It's for anyone who sits at a screen all day and keeps meaning to go outside.

Demo

In the video: a walkthrough of Grass Owed working, from the debt counter and the best free gap to a nudge appearing and the debt going down.

Note: I added this video on 12 Oct 2026 after 5 minutes of deadline. The app itself is unchanged.

Run it yourself: clone the repo, open a terminal in the app folder, run python -m http.server 8000, and open http://localhost:8000.

Want to see what it says at a different time of day? Add ?t=2026-06-10T14:00 to the address and time-travel to a very hot afternoon.

A few real nudges from the app:

Time for a little fresh air. You have 6 minutes owed.
The weather is nice. Try a quick walk now.
The 32 degrees C are perfect for a quick break.

Code

sssimransingh26-bit / Grass-Owed-you-owe-the-park

Grass Owed: You owe the park

Tracks how much outdoor time you owe, finds the best free gap in your day, and nudges you to go outside with short, funny notifications.

Built for the Hacktoberfest Open-Source AI Challenge, Week 1: Touch Grass (DEV) Project started 7 Oct 2026, inside the challenge window (5-11 Oct 2026). Any commits after the deadline will be listed under "Post-deadline commits" below.

Status: work in progress.

What it is

Debt is the score: a daily outdoor-minutes goal and a balance that carries over.
Gaps come from your calendar file (.ics), so suggestions fit your real day.
Comfort comes from a TabPFN model trained on historical weather, so it will not send you out at 2 PM in a hot month.
Tone is your choice (gentle, funny, cheeky). Gemma writes the words; plain code decides when and what.

Architecture

Build time (Colab, run once): TabPFN turns…

View on GitHub

How I Built It

The rule I set myself: the models do their work once, in Colab, and ship two tiny files. No model and no server at runtime.

TabPFN: "when is it actually nice outside?"

I pulled 4 years of hourly weather for Jaipur from Open-Meteo (apparent temperature and rain), held out the most recent 12 months, and ran a face-off: a simple baseline (average by month and hour) against TabPFN-3.5 running locally in Colab. TabPFN trained on a 10,000-row sample. The output is a 12 x 24 lookup file, comfort_table.json, about 13 KB.

Model	MAE on held-out year (apparent temp, °C)
Baseline (mean by month and hour)	2.340
TabPFN	2.303

TabPFN wins! By about 0.04 °C, roughly 1.6%. I'll be honest: that's one split scored on a 5,000-row sample, so it could be noise. I'm reporting it as it came out, because a tiny honest win beats a big imaginary one.

Gemma: "say it like a friend, not a calendar"

I ran Gemma 4 E2B (google/gemma-4-E2B-it, Apache-2.0) once on a free Colab T4 to write the nudge lines: 180 lines across 6 situations and 3 tones (gentle, funny, cheeky), 10 per combination. The app started with 36 hand-written placeholder lines, so this is five times the variety.

I didn't trust the model blindly. Plain code checks every line for length, allowed placeholders (like {best_time}), required details and banned words before it's kept. Of 181 lines Gemma returned, 180 passed (99.4%). The one rejected line invented a placeholder that doesn't exist, which is a very creative way to fail.

Generation took about 20 seconds per call (18 calls, roughly 6 minutes total, about 2 seconds per kept line), and lines average about 60 characters. The result is one small JSON file, so nudges work with no signal.

The app

A vanilla JavaScript PWA with no backend. The logic (debt ledger, .ics parser, gap finder, rules engine) lives in one pure module with 8 Node tests. A service worker caches everything so it opens offline.

On a laptop it counts screen time automatically with the browser's Idle Detection API (Chrome or Edge). On a phone you use start/stop buttons, because browsers can't read phone screen time.

How the debt moves

With laptop auto-tracking on, every 10 minutes of active use adds 2 minutes of debt, so it keeps climbing for as long as you work. Walking pays it back one minute for one minute, so a 20-minute walk clears 20 minutes of debt, which is what 100 minutes of screen time creates. Debt never goes below zero and carries over to the next day.

I Actually Went Outside

Yes, I tested it outdoors, which feels like the least I could do for a Touch Grass challenge. On 12 October 2026 I went outside for 20 minutes with the app's "I'm out" button running, and my debt went from 40 to 20 minutes.

What worked

The debt number dropped as soon as I tapped "I'm back", and it matched the minutes I was outside.
Laptop auto-tracking counted my screen minutes without me pressing anything.
A Gemma-written nudge showed up on the screen.
It kept working with the network off.

What didn't, or isn't done

The TabPFN improvement is tiny.
Rain probability per hour is noisy (about 120 days per cell), so comfort scores jump a little between neighbouring hours.
Calendar time zones are ignored, and repeating events are only expanded daily and weekly for 14 days.
Notifications only fire while the app is open. Reliable delivery to a closed phone app needs a push server, which I didn't build.
Laptop auto-tracking works only in Chrome or Edge on desktop.
It isn't deployed to a public URL, so the demo runs locally.

Why Does Open Innovation Matter?

Because this app knows your calendar and your screen habits, and that is exactly the kind of data you don't want to send to someone else's server.

Open weights meant both models could run in a free Colab notebook, and their output became two tiny files. There's no cloud API at runtime, no per-request cost, and no personal data leaves the device. Open weather data from Open-Meteo made the training set free to get.

One honest note: TabPFN's weights are free for non-commercial use under Prior Labs' licence, so this is a hackathon project and not a product.

Prize Categories

Best Use of TabPFN: it predicts outdoor comfort by month and hour from historical weather, exported as the lookup the app uses, with the error reported against a simple baseline.
Best Use of Gemma: Gemma 4 E2B wrote the app's 180-line nudge bank, with code validating every line.

Credits: weather data from Open-Meteo. The comfort table was generated with TabPFN-3.5 by Prior Labs GmbH under the TabPFN-3.5 Non-Commercial License. Gemma 4 by Google DeepMind (Apache-2.0).

DEV Community: Simran Singh

The Villages That Went Quiet: what 2001 census data can and can't predict in Uttarakhand

What I Built

The quiet, in numbers

How I built it

What I can't see from here

Why there is not one village name in this post

Prize Categories

Why Open Innovation Matters

Code and credits

Sloth Router: a model that does the least work it safely can, and admits when it shouldn't answer

A scene

The idea

The three rules

What I built

How the data is built

What I measured

The fine-tuning run

What went wrong

Limits

Why open

Next

Grass Owed: screen time becomes outdoor debt

What I Built

Demo

Code

sssimransingh26-bit / Grass-Owed-you-owe-the-park

Grass Owed: You owe the park

What it is

Architecture

How I Built It

TabPFN: "when is it actually nice outside?"

Gemma: "say it like a friend, not a calendar"

The app

How the debt moves

I Actually Went Outside

Why Does Open Innovation Matter?

Prize Categories