A self-updating open-data site with zero servers: GitHub Actions, Pages, and branches as a CDN

Village Finder serves interactive maps for over 78,000 Indian villages, refreshes its own geographic datasets daily, and streams live agricultural market prices.

It does all of this with no dedicated backend, no database server, and a monthly infrastructure bill of exactly ₹0.

Here is the exact architecture behind the project, including a highly effective pattern for zero-cost static architectures: weaponizing transient Git branches as a free, high-availability, CORS-enabled data CDN.

🛠️ The Serverless Toolkit

To achieve massive scale at absolute zero cost, the stack relies strictly on edge infrastructure:

GitHub Pages: Serves the frontend mapping applications (Vanilla JS + Leaflet.js, intentionally bypassing complex frontend frameworks for zero build-step latency).
GitHub Actions: Handles the entire continuous data integration workflow: daily upstream ingestion, polygon boundary simplification, release artifact compilation, and deployments.
GitHub Releases: Serves as the distribution network for versioned dataset downloads (.csv, .json, and .geojson splits).
Cloudflare R2: The single architecture exception. We stream gigabyte-scale cadastral land parcel maps that require absolute zero-egress pricing combined with reliable HTTP range-request support for geospatial vector formats.

🔀 The Core Insight: Two Classes of Data, Two Pipelines

When your application is completely database-less, every piece of incoming information has to live somewhere inside version control. Trying to treat volatile, real-time metrics the exact same way you treat structural administrative data will quickly ruin your git history.

We separated the data pipeline into two isolated lifecycles:

1. The Dataset of Record -> Reviewed Pull Requests

Core administrative village registries change infrequently but demand total structural integrity. When our scraper detects an upstream boundary change, the automation opens a structured Pull Request.

A comprehensive suite running over 90 validation checks via pytest validates referential tracking, and a custom changelog utility summarizes exactly which villages were shifted or reclassified. Consecutive nightly runs update the same PR branch in place, preventing bot spam.

When merged, the commit history acts as a transparent, version-controlled audit trail of government data changes.

2. Regenerable Artifacts -> Flat Data Branches

Daily agricultural market quotes (mandi prices), weekly welfare scheme updates, and monthly boundary vector tiles are highly volatile machine outputs. Having humans code-review a daily 10,000-row JSON diff is a waste of time.

Instead, the automation bypasses the PR loop entirely and writes directly to dedicated, isolated data branches using this isolated script routine:

# Compile and switch to an isolated, history-less data branch
work="$(mktemp -d)" 
cp out/*.json "$work" && cd "$work"
git init -q -b data/mandi-prices
git add . 
git commit -q -m "data: mandi prices $(date -u +%FT%RZ)"
git push --force "https://x-access-token:${TOKEN}@github.com/${REPO}.git" data/mandi-prices

By using git push --force with a history-less initial commit, the branch size never grows. We get infinite, automated data updates without inflating the base repository's .git footprint.

🌐 Branch Assets as a Live CDN

Here is where the magic happens: GitHub's asset proxy (raw.githubusercontent.com) serves any public repository file with an explicit Access-Control-Allow-Origin: * header.

Because of this, the frontend application doesn't need to redeploy when daily market figures shift. The client-side browser simply sends an asynchronous fetch directly to the head of the data branch:

// Fetch real-time market data straight from the branch asset
const targetStateUrl = `https://raw.githubusercontent.com/mchittineni/india-village-finder/refs/heads/data/mandi-prices/${stateName}.json`;
const response = await fetch(targetStateUrl);

For large spatial files that must match same-origin restrictions (like PMTiles requires for rendering vector graphics), we use a tiny GitHub Action step to overlay the static data branch files straight into the deployment workspace during the core production build window:

- name: Overlay Spatial Tile Artifacts
  run: git fetch -q --depth 1 origin "data/boundary-tiles" && git checkout -q FETCH_HEAD -- .

If the branch is completely absent on a developer's local fork, the step simply skips cleanly—ensuring the localized environment remains easy to run out-of-the-box.

☔ Designing to Survive Flaky Upstreams

Public sector APIs drop connections frequently. If a scheduled cron job fires an unexpected error alert every single time an upstream government gateway encounters an hour of maintenance downtime, it trains your engineering team to ignore CI alerts.

To resolve this, we mapped all connection timeouts and API drops to the standard Unix temporary failure contract: EX_TEMPFAIL (Status Code 75).

When our fetchers encounter a structural gateway issue after exhaustive retries, they exit with 75. The master GitHub Actions workflow catches this code, stops execution cleanly, keeps yesterday's working data snapshot intact, and writes a brief summary notice. The build stays green, preventing alert fatigue, while true codebase failures continue to ring alarms loudly.

💡 Hard-Learned Takeaways

If you are planning to leverage GitHub as an active data delivery network, watch out for these execution edge-cases:

Pathspecs Aren't Shell Globs: Inside Git operations, path formatting rules differ from traditional bash syntax. Writing */web/data maps to nothing, whereas explicitly writing */web/data/ targets nested subdirectories accurately. This single variance can silently eat data commits while keeping your workflows completely green.
Untracked Diffs: The standard utility git diff --quiet is blind to untracked files on initial initialization passes. Always verify pipeline variations by reading raw output lines from git status --porcelain.
Isolate Through Composites: Bundle identical setups, validation guards, and branch publication routines into modular custom Composite Actions. Keep your root workflow files incredibly readable, focusing only on core state execution loops.

📈 Open Code, Open Data

By decoupling static presentation logic from isolated data branches, you can build remarkably resilient, scalable open-data utilities that cost nothing to run.

The complete open-source pipeline architecture, testing frameworks, and spatial datasets are entirely free to explore and copy: