Katas Raj Heritage 3D: Building an AI-Powered Digital Preservation Tool

A PMW case study by Amna Hafeez · Team Indus

---

Problem

The Katas Raj Temples in Chakwal, Punjab, are a partially ruined but actively protected heritage site — a Hindu temple complex built around a sacred spring-fed pond. Documentation for sites like this is usually static: a handful of photos and a paragraph of history. That approach loses the spatial and material detail that actually matters for preservation — how large the structures are, what condition the domes and walls are in, and what the site looks like from angles a flat photo can't show. My goal was to build a tool that could classify, measure, and reconstruct the site digitally, and present it in a way that felt like a working survey record rather than a static gallery.

My Role

I was the sole developer and Team Indus lead on this build. I designed and implemented the full stack: the ML image classifier, the AR measurement overlay, the Three.js 3D viewer, and the site's information architecture and visual design (the "field survey record" aesthetic). I also ran and interpreted the site's own performance/accessibility audits and fixed what came out of them.

Live site: katas-raj-heritage-3d.vercel.app
Repository: github.com/Anfey-SE/katas-raj-heritage-3d
Commit history: github.com/Anfey-SE/katas-raj-heritage-3d/commits/main

What I Built

The site is organized as a "site survey record" with three exhibits per location:

• Exhibit A — Photograph: the base documentation shot, captioned like a field photo log (location, condition, protection status, capture angles logged).
• Exhibit B — AR Survey Lines: an HTML5 Canvas overlay drawn on top of the photograph showing measured distances across the site — diagonal spans, wall heights, and pond width (e.g., 21.1m diagonal, 18.7m and 12.4m spans, 9.2m elevation) — simulating how a real AR measurement tool would annotate the ruins.

• Exhibit C — Capture-to-3D: a Three.js-rendered depth relief generated from photo luminance, giving an orbit-able first-pass 3D read of the site's terrain and structures, with drag-to-orbit and pan controls.

Underneath this, I trained and integrated a MobileNetV2 image classifier to automatically categorize heritage site photos. Tested against three real Katas Raj photographs, it correctly identified the site type ("castle" — the closest ImageNet-class match to a fortified temple complex) with 74.7%, 88.4%, and 98.6% confidence respectively, with the next-closest guesses (monastery, cliff dwelling, palace) trailing far behind. That's a meaningful signal for a model that was never trained on Pakistani heritage architecture specifically.

Process

1. Photographed and sourced reference images of the Katas Raj complex.
2. Built the classifier pipeline in Google Colab (CPU-only local hardware made this the practical choice) and validated it against the three site photos, visualizing confidence scores per class.
3. Built the Exhibit A/B/C tabbed interface, including the Canvas-based AR measurement overlay and the Three.js depth-relief viewer.
4. Ran a Lighthouse audit and found the results were being skewed by Chrome extensions loading in the profile — the first pass showed misleadingly low Performance alongside perfect Accessibility/Best Practices/SEO. I re-ran the audit in a clean incognito window with no extensions to get a valid reading.
5. Documented the corrected results.

Result

Valid Lighthouse audit (incognito, no extensions):

Metric	Score
Performance	52
Accessibility	100
Best Practices	100
SEO	100

Accessibility, Best Practices, and SEO are all perfect. Performance sits at 52, held down mainly by the Three.js/Canvas-heavy Exhibit C view, which is the honest number rather than the artificially poor one caused by browser extensions on the first (invalid) run.

What I Learned

Running the same audit twice taught me something I wouldn't have learned from a single clean pass: tooling itself can lie to you. The first Lighthouse run reported a Performance score of 37 — a number I could have submitted as evidence without realizing it wasn't measuring my site at all, but the extensions in my browser profile. Catching that and re-running in incognito was a small thing, but it's the difference between real evidence and a false read. I also learned, more directly, how much weight a 3D canvas view carries on load performance versus a static image — the gap between my Performance score and my other three scores is basically a map of where the actual rendering cost lives in this app.

Next Improvements

• Lazy-load or defer the Three.js/Canvas exhibits so they don't block initial paint, to close the gap between Performance and the other three scores.
• Extend the classifier training set with more Pakistani heritage sites so classifications move from "closest ImageNet analog" (e.g., castle) toward more accurate custom labels (e.g., temple complex, stepped pond).
• Replace the simulated AR measurement overlay with actual computed distances from the COLMAP reconstruction pipeline once sparse reconstruction quality improves.

---

Evidence: live site · commit history · classifier confidence charts and Lighthouse audit screenshots above.