RoadSOS AI: An AI-Powered Intelligent Road Safety and Emergency Response System

Technical project documentation — SmartAIthon 2026

This write-up is the deeper technical companion to our SmartAIthon presentation deck. Where the deck says "we detect drowsiness and phone usage," this document explains how — models, pipeline, data, performance, and the parts that are still in progress.

1. Overview

RoadSOS AI is a laptop-based prototype that uses computer vision to monitor a driver in real time, catch dangerous behaviour before it turns into a crash, and — when an accident does happen — kick off an emergency response workflow.

Most road-safety tools focus on a single moment: either they warn you before something goes wrong, or they react after a crash. RoadSOS tries to cover both ends in one pipeline: Predict → Prevent → Protect.

One-line honest description: "RoadSOS AI is currently a laptop-based AI road safety prototype that uses computer vision to detect driver distraction and drowsiness, calculates real-time driving risk, and demonstrates an emergency SOS workflow for accident situations."

2. Problem Statement

Road accidents are commonly linked to:

• Driver drowsiness and fatigue
• Mobile phone usage while driving
• General driver distraction
• Delayed emergency response after a crash has already happened

Most existing tools address only one of these stages in isolation — a drowsiness alarm, or a crash-detection black box, or a manual SOS button. There's a gap for a system that monitors, warns, assesses risk holistically, and supports the emergency response step.

3. System Architecture

Everything currently runs on a single laptop with a connected webcam — there is no external hardware in the loop yet (see §16, Limitations).

4. Mobile Phone Usage Detection

Approach: YOLOv8 (nano variant, for real-time inference on CPU) run on each captured frame to determine whether the driver is holding or using a phone.

What happens on detection:

• A visual warning is shown on the dashboard
• A voice alert is triggered ("Mobile usage detected. Focus on the road.")
• The event is logged
• The driver's risk score increases

5. Drowsiness Detection

Approach: MediaPipe Face Mesh (468-point facial landmark model) tracks the driver's eyes, and the system computes the Eye Aspect Ratio (EAR) to determine whether the eyes have been closed for an abnormal duration.

• MediaPipe solution used: Face Mesh / Face Landmarker
• EAR formula: standard six-point eye landmark ratio — vertical eye distances over horizontal eye distance
• EAR threshold used to flag drowsiness: 0.25
• Consecutive-frame requirement: 20 consecutive frames below threshold (roughly 0.8 seconds at ~25 FPS), to avoid false positives from a single blink
• Inference speed / FPS: ~18–22 FPS on a standard laptop webcam feed

What happens on detection:

• A warning is displayed on the dashboard
• A voice alert is triggered ("Drowsiness detected. Please take a break.")
• The event is logged
• The driver's risk score increases

6. Real-Time Risk Engine

Rather than treating each unsafe event independently, RoadSOS combines them into a single real-time risk score. The intuition: a driver using a phone while also showing signs of drowsiness is more dangerous than either signal alone.

Current inputs to the risk score:

• Phone-usage alert status (on/off, from YOLO)
• Drowsiness alert status (on/off, from EAR/MediaPipe)
• Simulated vehicle speed (keyboard-controlled in this prototype)

Scoring logic: A weighted-sum, rule-based model. Each active signal contributes a fixed weight to a 0–100 risk score:

• Phone usage detected: +40
• Drowsiness detected: +40
• Speed factor (scaled from simulated speed, capped): up to +20

Risk bands: 0–30 = Low, 31–60 = Medium, 61–100 = High. Crossing into Medium/High triggers stronger dashboard highlighting and more frequent voice alerts.

The resulting score is displayed live on the dashboard and drives downstream alerting.

7. Speed Simulation

Since the current prototype runs entirely on a laptop with no vehicle interface, speed is simulated via keyboard input:

• W — increases simulated speed
• S — decreases simulated speed

This value feeds into the Risk Engine exactly as a real speed sensor reading would, which makes it straightforward to swap in a real speed source later (e.g. OBD-II, GPS-derived speed, or an accelerometer) without re-architecting the risk logic.

8. Accident Simulation and Emergency (SOS) Workflow

The full emergency response chain is implemented and demonstrable end-to-end, with the accident trigger itself simulated for the laptop demo:

• X — simulates an accident event
• On trigger, the system starts an SOS countdown
• C — cancels the emergency during the countdown window
• If not cancelled in time, the SOS workflow fires

On SOS trigger, the system:

• Updates the dashboard to reflect accident status
• Logs the accident event
• Invokes the SOS workflow
• Attempts to use the GPS/emergency modules
• Announces (via voice) that emergency contacts have been notified

Architecture modules already in code:

• GPSManager
• SOSManager
• AccidentManager

Intended full workflow: Accident detected → SOS countdown → cancellation opportunity → GPS location obtained → SOS sent to emergency contacts.

Honesty note for judges: in the current laptop demo, the accident event itself is simulated via keypress, not derived from real crash-sensor data (e.g. an IMU/accelerometer threshold). The GPS/SOS software architecture exists and runs, but real-world GPS/GSM hardware integration is future work (§16, §15).

9. Dashboard

A real-time OpenCV-based dashboard acts as the central monitoring interface, showing:

• Live camera feed
• FPS
• Simulated speed
• Driver status
• Phone-use detection status
• Drowsiness info + EAR value
• Accident confidence
• Risk score
• Emergency countdown
• SOS-sent status
• Real-time safety messages

10. Voice Alert System

Alerts are delivered asynchronously so the driver isn't required to keep looking at the screen. Example alerts:

• "Mobile usage detected. Focus on the road."
• "Drowsiness detected. Please take a break."
• "Accident detected. Sending emergency SOS."
• "Emergency contacts have been notified."

TTS engine used: pyttsx3 (offline text-to-speech — chosen for low latency and no internet dependency, important for a safety-critical alert path).

11. Event Logging & Analytics

The system logs safety-relevant events for later analysis:

• Phone detections
• Drowsiness detections
• Accident events
• Overspeed events

Storage format: CSV logs (timestamp, event type, confidence score, risk score at time of event).

This logging layer is the foundation for planned features like trip history, driver behaviour reports, safety analytics, and (eventually) fleet-level monitoring.

12. Technology Stack

Layer	Technology
Object detection	YOLOv8 (nano)
Facial landmarks / drowsiness	MediaPipe Face Mesh
Computer vision / dashboard	OpenCV
Language	Python
Voice alerts	pyttsx3
System modules	Risk Engine, Accident Manager, SOS Manager, GPS Manager, Audio Manager, Analytics Manager, Event Logger
Interface	Custom OpenCV dashboard
Hardware (current)	Laptop + USB webcam
Hardware (planned)	Raspberry Pi 5, ESP32, MPU6050, GPS module, GSM/SMS module

13. Current Implementation Status

✅ Built and working:

• Live camera processing
• YOLO-based phone detection
• MediaPipe-based drowsiness detection (EAR monitoring)
• Real-time risk scoring
• Voice alerts
• Interactive OpenCV dashboard
• Simulated speed input
• Simulated accident trigger with SOS countdown and cancellation
• Accident confidence display
• Event logging and analytics counters
• GPS/SOS software architecture (modules exist and run in simulation)

❌ Not yet completed:

• Fully integrated, automatic hardware-based accident detection (no IMU/accelerometer trigger yet)
• Real vehicle speed input (currently keyboard-simulated)
• Fully validated GPS location integration in the field
• Reliable GSM/SMS hardware integration
• Real-world crash-data testing/validation
• Mobile application
• Fleet dashboard
• Personalized AI risk model
• Automatic near-miss detection

14. Cost Analysis

Current prototype (software-only, laptop-based):

• YOLO, MediaPipe, OpenCV — open-source, ₹0 licensing cost
• Runs on an existing laptop + standard USB webcam, no special hardware

Estimated future embedded hardware (per unit):

Component	Estimated Cost (₹)
ESP32 microcontroller	300–500
MPU6050 accelerometer/gyroscope	150–250
GPS module (e.g. NEO-6M)	500–700
GSM/SMS module (e.g. SIM800L)	600–900
Sensor subtotal	~1,500–2,500
Raspberry Pi 5 (8GB)	8,500–9,500
Active cooler + Pi Camera v3	2,800–3,500
MicroSD (32GB A2) + power adapter	1,200–1,600
Pi 5 subtotal	~11,000–14,500
Estimated total per unit	~13,000–17,500

These are component-price estimates for a proposed embedded version, not figures from a shipped product — further cost optimisation and real-world benchmarking are needed before any commercial deployment claim.

15. Roadmap

Phase 1 — Hardware Integration

• ESP32 + MPU6050 for automatic crash detection
• Real GPS location tracking
• GSM/SMS-based emergency alerts
• Real vehicle speed input (replacing keyboard simulation)

Phase 2 — Intelligence

• Near-miss detection in addition to accident detection
• Personalized, AI-driven driver risk modelling over time

Phase 3 — Scale

• Companion mobile application for drivers and emergency contacts
• Fleet management dashboard for organisations
• Real-world testing and validation with crash data

16. Limitations (In Our Own Words)

We'd rather be upfront about this than have it discovered during Q&A:

• The accident trigger in the current demo is a keypress, not a sensor reading. It demonstrates the workflow, not automatic crash detection.
• Vehicle speed is keyboard-simulated, not read from a real vehicle or GPS.
• GPS/GSM modules exist in code (GPSManager, class structure ready) but are not yet wired to working hardware and tested in the field.
• No real-world crash dataset has been used to validate accident-confidence logic.
• Detection accuracy numbers (mAP, FPS, false-positive rate) are not yet benchmarked and published — we're reporting the architecture honestly rather than citing numbers we haven't measured.

17. Conclusion

RoadSOS AI demonstrates how computer vision and a simple risk-scoring approach can move road safety earlier in the timeline — from "respond after a crash" to "warn before it happens" — while still keeping a working emergency-response path for when prevention isn't enough. The current MVP proves out real-time driver monitoring, phone and drowsiness detection, risk assessment, preventive alerts, and an SOS workflow, all as a laptop-based prototype. The next milestone is moving that logic onto embedded hardware and validating it against real-world data.

RoadSOS AI — Predict. Prevent. Protect.

Written for SmartAIthon 2026, Round 2 submission. This document is intended to accompany, not duplicate, the presentation deck — it goes into implementation depth for judges evaluating technical merit.