DEV Community

Abinivesh M
Abinivesh M

Posted on

"RoadSOS AI: An AI-Powered Intelligent Road Safety and Emergency Response System"

#ai #machinelearning #software

RoadSOS AI: An AI-Powered Intelligent Road Safety and Emergency Response System

Technical project documentation — SmartAIthon 2026

This write-up is the deeper technical companion to our SmartAIthon presentation deck. Where the deck says "we detect drowsiness and phone usage," this document explains how — models, pipeline, data, performance, and the parts that are still in progress.

1. Overview

RoadSOS AI is a laptop-based prototype that uses computer vision to monitor a driver in real time, catch dangerous behaviour before it turns into a crash, and — when an accident does happen — kick off an emergency response workflow.

Most road-safety tools focus on a single moment: either they warn you before something goes wrong, or they react after a crash. RoadSOS tries to cover both ends in one pipeline: Predict → Prevent → Protect.

One-line honest description: " RoadSOS AI is currently a laptop-based AI road safety prototype that uses computer vision to detect driver distraction and drowsiness, calculates real-time driving risk, and demonstrates an emergency SOS workflow for accident situations."

2. Problem Statement

Road accidents are commonly linked to:

  • Driver drowsiness and fatigue
  • Mobile phone usage while driving
  • General driver distraction
  • Delayed emergency response after a crash has already happened

Most existing tools address only one of these stages in isolation — a drowsiness alarm, or a crash-detection black box, or a manual SOS button. There's a gap for a system that monitors, warns, assesses risk holistically, and supports the emergency response step.

3. System Architecture 

Camera Input
     │
     ▼
OpenCV Processing (frame capture + preprocessing)
     │
     ├──────────────┬──────────────┐
     ▼                              ▼
YOLO-based Detection          MediaPipe Facial
(Mobile Phone Usage)          Landmark Detection
     │                        (Drowsiness / EAR)
     └──────────────┬──────────────┘
                     ▼
               Risk Engine
     (phone status + drowsiness status + simulated speed)
                     │
     ┌───────────────┼────────────────┐
     ▼               ▼                ▼
 Dashboard      Voice Alerts     Event Logging
 (OpenCV UI)    (async audio)    / Analytics
                     │
                     ▼
        Accident Manager (simulated trigger)
                     │
                     ▼
              SOS Manager → cancel window
                     │
                     ▼
              GPS Manager → Emergency Notification
Enter fullscreen mode Exit fullscreen mode

Everything currently runs on a single laptop with a connected webcam — there is no external hardware in the loop yet (see §9, Limitations).

4. Mobile Phone Usage Detection

Approach: YOLO-based object detection run on each captured frame to determine whether the driver is holding or using a phone.

  • YOLO version used: NA — fill in (e.g. YOLOv8n, YOLOv5s, etc.)
  • Dataset / training data: NA — fill in (e.g. custom-labelled dataset, COCO subset, public driver-distraction dataset, number of images/classes)
  • Preprocessing: NA — fill in (resize dimensions, normalization, augmentation used, if any)
  • Inference pipeline: frame captured via OpenCV → passed to YOLO model → bounding boxes filtered for phone class → confidence threshold applied → result fed to Risk Engine
  • Confidence threshold used: NA — fill in
  • Inference speed / FPS on this pipeline: NA — fill in (measured on your hardware)
  • Accuracy / mAP: NA — fill in if you've benchmarked it

What happens on detection:

  • A visual warning is shown on the dashboard
  • A voice alert is triggered ("Mobile usage detected. Focus on the road.")
  • The event is logged
  • The driver's risk score increases

5. Drowsiness Detection

Approach: MediaPipe facial landmark detection tracks the driver's eyes, and the system computes the Eye Aspect Ratio (EAR) to determine whether the eyes have been closed for an abnormal duration.

  • MediaPipe solution used: Face Mesh / Face Landmarker (NA — confirm exact solution + version)
  • EAR formula: standard six-point eye landmark ratio — vertical eye distances over horizontal eye distance
  • EAR threshold used to flag drowsiness: NA — fill in your calibrated threshold
  • Consecutive-frame requirement (to avoid false positives from a single blink): NA — fill in, e.g. "N consecutive frames below threshold"
  • Inference speed / FPS: NA — fill in

What happens on detection:

  • A warning is displayed on the dashboard
  • A voice alert is triggered ("Drowsiness detected. Please take a break.")
  • The event is logged
  • The driver's risk score increases

6. Real-Time Risk Engine

Rather than treating each unsafe event independently, RoadSOS combines them into a single real-time risk score. The intuition: a driver using a phone while also showing signs of drowsiness is more dangerous than either signal alone.

Current inputs to the risk score:

  1. Phone-usage alert status (on/off, from YOLO)
  2. Drowsiness alert status (on/off, from EAR/MediaPipe)
  3. Simulated vehicle speed (keyboard-controlled in this prototype)

Scoring logic: NA — fill in your actual weighting/formula, e.g. weighted sum, rule-based thresholds, or a simple state machine.

The resulting score is displayed live on the dashboard and drives downstream alerting.

7. Speed Simulation

Since the current prototype runs entirely on a laptop with no vehicle interface, speed is simulated via keyboard input:

  • W — increases simulated speed
  • S — decreases simulated speed

This value feeds into the Risk Engine exactly as a real speed sensor reading would, which makes it straightforward to swap in a real speed source later (e.g. OBD-II, GPS-derived speed, or an accelerometer) without re-architecting the risk logic.

8. Accident Simulation and Emergency (SOS) Workflow

The full emergency response chain is implemented and demonstrable end-to-end, with the accident trigger itself simulated for the laptop demo:

  • X — simulates an accident event
  • On trigger, the system starts an SOS countdown
  • C — cancels the emergency during the countdown window
  • If not cancelled in time, the SOS workflow fires

On SOS trigger, the system:

  1. Updates the dashboard to reflect accident status
  2. Logs the accident event
  3. Invokes the SOS workflow
  4. Attempts to use the GPS/emergency modules
  5. Announces (via voice) that emergency contacts have been notified

Architecture modules already in code:

  • GPSManager
  • SOSManager
  • AccidentManager

Intended full workflow: Accident detected → SOS countdown → cancellation opportunity → GPS location obtained → SOS sent to emergency contacts.

Honesty note for judges: in the current laptop demo, the accident event itself is simulated via keypress, not derived from real crash-sensor data (e.g. an IMU/accelerometer threshold). The GPS/SOS software architecture exists and runs, but real-world GPS/GSM hardware integration is future work (§9, §11).

9. Dashboard

A real-time OpenCV-based dashboard acts as the central monitoring interface, showing:

  • Live camera feed
  • FPS
  • Simulated speed
  • Driver status
  • Phone-use detection status
  • Drowsiness info + EAR value
  • Accident confidence
  • Risk score
  • Emergency countdown
  • SOS-sent status
  • Real-time safety messages

10. Voice Alert System

Alerts are delivered asynchronously so the driver isn't required to keep looking at the screen. Example alerts:

  • "Mobile usage detected. Focus on the road."
  • "Drowsiness detected. Please take a break."
  • "Accident detected. Sending emergency SOS."
  • "Emergency contacts have been notified."

TTS engine used: NA — fill in (e.g. pyttsx3, gTTS, platform TTS)

11. Event Logging & Analytics

The system logs safety-relevant events for later analysis:

  • Phone detections
  • Drowsiness detections
  • Accident events
  • Overspeed events

Storage format: NA — fill in (e.g. CSV, SQLite, JSON logs)

This logging layer is the foundation for planned features like trip history, driver behaviour reports, safety analytics, and (eventually) fleet-level monitoring.

12. Technology Stack

Layer Technology
Object detection YOLO (version — NA)
Facial landmarks / drowsiness MediaPipe
Computer vision / dashboard OpenCV
Language Python
Voice alerts TTS library — NA
System modules Risk Engine, Accident Manager, SOS Manager, GPS Manager, Audio Manager, Analytics Manager, Event Logger
Interface Custom OpenCV dashboard
Hardware (current) Laptop + USB webcam
Hardware (planned) Raspberry Pi 5, ESP32, MPU6050, GPS module, GSM/SMS module

13. Current Implementation Status

✅ Built and working:

  • Live camera processing
  • YOLO-based phone detection
  • MediaPipe-based drowsiness detection (EAR monitoring)
  • Real-time risk scoring
  • Voice alerts
  • Interactive OpenCV dashboard
  • Simulated speed input
  • Simulated accident trigger with SOS countdown and cancellation
  • Accident confidence display
  • Event logging and analytics counters
  • GPS/SOS software architecture (modules exist and run in simulation)

❌ Not yet completed:

  • Fully integrated, automatic hardware-based accident detection (no IMU/accelerometer trigger yet)
  • Real vehicle speed input (currently keyboard-simulated)
  • Fully validated GPS location integration in the field
  • Reliable GSM/SMS hardware integration
  • Real-world crash-data testing/validation
  • Mobile application
  • Fleet dashboard
  • Personalized AI risk model
  • Automatic near-miss detection

14. Cost Analysis

Current prototype (software-only, laptop-based):

  • YOLO, MediaPipe, OpenCV — open-source, ₹0 licensing cost
  • Runs on an existing laptop + standard USB webcam, no special hardware

Estimated future embedded hardware (per unit):

Component Estimated Cost (₹)
ESP32 microcontroller 300–500
MPU6050 accelerometer/gyroscope 150–250
GPS module (e.g. NEO-6M) 500–700
GSM/SMS module (e.g. SIM800L) 600–900
Sensor subtotal ~1,500–2,500
Raspberry Pi 5 (8GB) 8,500–9,500
Active cooler + Pi Camera v3 2,800–3,500
MicroSD (32GB A2) + power adapter 1,200–1,600
Pi 5 subtotal ~11,000–14,500
Estimated total per unit ~13,000–17,500

These are component-price estimates for a proposed embedded version, not figures from a shipped product — further cost optimisation and real-world benchmarking are needed before any commercial deployment claim.

15. Roadmap

Phase 1 — Hardware Integration

  • ESP32 + MPU6050 for automatic crash detection
  • Real GPS location tracking
  • GSM/SMS-based emergency alerts
  • Real vehicle speed input (replacing keyboard simulation)

Phase 2 — Intelligence

  • Near-miss detection in addition to accident detection
  • Personalized, AI-driven driver risk modelling over time

Phase 3 — Scale

  • Companion mobile application for drivers and emergency contacts
  • Fleet management dashboard for organisations
  • Real-world testing and validation with crash data

16. Limitations (In Our Own Words)

We'd rather be upfront about this than have it discovered during Q&A:

  • The accident trigger in the current demo is a keypress, not a sensor reading. It demonstrates the workflow, not automatic crash detection.
  • Vehicle speed is keyboard-simulated, not read from a real vehicle or GPS.
  • GPS/GSM modules exist in code (GPSManager, class structure ready) but are not yet wired to working hardware and tested in the field.
  • No real-world crash dataset has been used to validate accident-confidence logic.
  • Detection accuracy numbers (mAP, FPS, false-positive rate) are not yet benchmarked and published — we're reporting the architecture honestly rather than citing numbers we haven't measured.

17. Conclusion

RoadSOS AI demonstrates how computer vision and a simple risk-scoring approach can move road safety earlier in the timeline — from "respond after a crash" to "warn before it happens" — while still keeping a working emergency-response path for when prevention isn't enough. The current MVP proves out real-time driver monitoring, phone and drowsiness detection, risk assessment, preventive alerts, and an SOS workflow, all as a laptop-based prototype. The next milestone is moving that logic onto embedded hardware and validating it against real-world data.

RoadSOS AI — Predict. Prevent. Protect.

Written for SmartAIthon 2026, Round 2 submission. This document is intended to accompany, not duplicate, the presentation deck — it goes into implementation depth for judges evaluating technical merit.

Top comments (0)