What if an AI system could look at an incident scene, identify the visible evidence, suggest what might have happened, and then evaluate those possible explanations based on the evidence?
That is the idea behind our hackathon project, AI “What Went Wrong?” — Incident Detective, built by **Team Tech Fusion.
What We Built
Our system takes two inputs:
- An image of an incident scene
- A short description or context about the incident
For example, the user could upload an image of a messy laboratory workspace and provide context such as:
“This incident happened in an electronics laboratory.”
The system then performs the analysis in two stages.
Stage 1 — Gemma 4: See and Understand
We use Gemma 4's multimodal capabilities to analyze the uploaded image together with the provided context.
Gemma extracts:
- Visible objects
- Visible conditions
- Evidence
- Possible explanations
- Uncertainties
The explanations are treated as hypotheses, not confirmed facts.
For example, Gemma might identify:
- Stains on a work surface
- A bottle lying near the stained area
- Protective gloves
- Residue on containers
and suggest possible explanations such as:
- Accidental spill
- Routine laboratory activity
- Material handling issue
The output is returned as structured JSON so that it can be passed directly to the next stage.
Stage 2 — Laya: Evaluate the Evidence
The structured output generated by Gemma is then passed to Laya, an open-source decision engine.
Laya evaluates the possible explanations against the evidence identified by Gemma and produces a confidence score for each hypothesis.
The dashboard then displays the explanations as cards, with the highest-confidence explanation highlighted as the most likely explanation.
The goal is not to claim certainty, but to make the reasoning process transparent:
Gemma sees → Evidence is extracted → Laya evaluates → Explanations are ranked
Architecture
text
User
│
├── Incident Image
└── Incident Context
│
▼
React Frontend
│
▼
FastAPI Backend
│
▼
Gemma 4
│
▼
Structured Evidence
- Hypotheses
- Uncertainties │ ▼ Laya │ ▼ Ranked Explanations
- Confidence Scores │ ▼ React Dashboard
Technology Stack
- Gemma 4 — multimodal incident scene analysis
- Laya — evidence-based hypothesis evaluation
- Python — AI/backend implementation
- FastAPI — backend API
- React + Vite — frontend dashboard
- Google GenAI SDK — Gemma API integration
Why This Approach?
A simple vision model can tell us what is visible in an image, but our goal is to go one step further.
We want to separate:
What can be seen from what might have happened.
Gemma handles scene understanding and hypothesis generation, while Laya independently evaluates those hypotheses using the evidence.
This separation gives us a clearer and more explainable pipeline instead of asking one model to directly decide what happened.
Responsible AI
The system is designed as an incident hypothesis and reasoning assistant, not as a forensic or safety authority.
The output represents possible explanations based on the available evidence. It should not be treated as definitive proof or as an accusation against a person.
We also explicitly preserve uncertainties so that the system can communicate what cannot be determined from the available information.
Links
GitHub Repository:
[Add your GitHub repository link here]
Laya:
https://github.com/NandhaKishorM/laya
Gemma:
https://ai.google.dev/gemma
Team: Tech Fusion
What's Next?
We are currently integrating the individual modules into a complete application.
Our next steps include:
- Connecting the React upload interface to the FastAPI backend
- Passing user-provided images and context to Gemma
- Connecting Gemma's structured output to Laya
- Building the final incident analysis dashboard
- Testing the system across different types of incident scenes
The core idea is simple:
Gemma sees. Laya decides. Our system explains.
Built as part of our hackathon project by Team Tech Fusion
Top comments (0)