A safety alarm that people don't trust is worse than no alarm at all — because "no alarm" at least doesn't teach anyone to ignore warnings. The moment a hazard system cries wolf a few times, or fires without any explanation anyone can act on, people stop listening. And an ignored warning is a failed warning, no matter how accurate the model behind it is. That reality drove a lot of the thinking behind the rockfall prediction system.
A prediction nobody acts on didn't do its job
We tend to measure a predictive model by how correct it is. But for anything that issues a warning to a human, correctness is only half the job — the other half is being believed and acted on. A model can be right and still fail, if its output is a bare red light with no reason attached. The person on the receiving end has to decide: evacuate, inspect, or ignore? A number with no story behind it doesn't help them make that call.
So the real product isn't the prediction. It's a warning a human can trust enough to act on under pressure — and trust is earned by being explainable, not just by being right.
Design for the human on the other end
Building a hazard system means designing the trust, not only the accuracy:
- Explainability over black-box. A warning that can point at why — these sensors, this trend, this crossing of a threshold — lets a person sanity-check it against what they can see. That grounding is what turns an alert into an actionable decision instead of a mystery.
- Manage the false-alarm budget. Every unnecessary alarm spends trust you can't easily earn back. You'd rather tolerate a few false alarms than miss a real event — but you treat each one as a real cost, because enough of them and the whole system gets tuned out.
- The interface is part of the safety. How and when a warning is surfaced — clearly, early, with context — is as much a part of the engineering as the model. A correct prediction delivered badly is a wasted prediction.
The takeaway
Building anything that warns people taught me that the model is the easy half. The hard half is earning enough trust that the warning actually changes behavior — through explainability, restraint with false alarms, and an interface that respects the human's judgment. A prediction is only as valuable as the action it manages to trigger.
The full system — sensor fusion, the model, and how it surfaces a warning — is on the project page.
👉 See it: www.divyakush.com/projects/rockfall-prediction
Divyakush Punjabi — Full-Stack & AI Systems Engineer
🌐 https://www.divyakush.com · 💼 LinkedIn · 💻 GitHub
Top comments (0)