DEV Community

vmodal_ai
vmodal_ai

Posted on

Object Detection on Android for Autonomous Robots

#android #kotlin #ai #robotics

Object Detection on Android for Autonomous Robots

Introduction

Autonomous robots need to recognize objects in their environment. Object detection can identify people, vehicles, tools, signs, and obstacles from camera frames.

Android can perform edge inference locally, reducing dependency on network connectivity.

Architecture 

CameraX
   |
Preprocessing
   |
Object Detection Model
   |
Postprocessing
   |
Detection Results
   |
Robot Perception Gateway
Enter fullscreen mode Exit fullscreen mode

Detection Model

Define a reusable result type:

data class Detection(
    val label: String,
    val confidence: Float,
    val left: Float,
    val top: Float,
    val right: Float,
    val bottom: Float
)
Enter fullscreen mode Exit fullscreen mode

This keeps the rest of the application independent from a particular model runtime.

CameraX Analysis

The camera analyzer should process frames asynchronously:

imageAnalysis.setAnalyzer(executor) { image ->
    detector.detect(image)
    image.close()
}
Enter fullscreen mode Exit fullscreen mode

Use a latest-frame strategy when real-time responsiveness is more important than processing every frame.

Model Runtime

The detector can be implemented behind an interface:

interface ObjectDetector {
    suspend fun detect(frame: ImageFrame): List<Detection>
}
Enter fullscreen mode Exit fullscreen mode

Possible mobile inference approaches include TensorFlow Lite or ONNX Runtime, depending on the model and deployment requirements.

Confidence Filtering

Not every prediction should be passed to the navigation system.

val valid = detections.filter {
    it.confidence >= 0.6f
}
Enter fullscreen mode Exit fullscreen mode

The threshold should be evaluated against the target environment rather than chosen arbitrarily.

Non-Maximum Suppression

Detection models may produce overlapping predictions.

Prediction A  ───────
Prediction B    ───────
        ↓
       NMS
        ↓
Single Detection
Enter fullscreen mode Exit fullscreen mode

Use the postprocessing method expected by your selected model.

Robot Integration

The Android device can send detections to the robot:

{
  "label": "person",
  "confidence": 0.94,
  "bbox": [120, 80, 350, 500]
}
Enter fullscreen mode Exit fullscreen mode

For autonomous navigation, the robot should combine this with physical measurements such as depth or LiDAR when distance matters.

Tracking

Instead of detecting every object from scratch at every stage, an additional tracking layer can maintain object identities between frames.

Detection
   ↓
Tracking
   ↓
Object ID
   ↓
Navigation / Behavior
Enter fullscreen mode Exit fullscreen mode

Tracking can reduce redundant processing and provide temporal context.

Performance Optimization

Important optimization techniques include:

  • Use an appropriately sized model
  • Reduce input resolution when acceptable
  • Reuse buffers
  • Avoid bitmap copies
  • Run inference off the main thread
  • Drop stale frames
  • Measure end-to-end latency

Autonomous Decision Making

Keep AI perception separate from robot control:

Camera
  ↓
Object Detection
  ↓
Perception State
  ↓
Navigation / Behavior
  ↓
Safety Controller
  ↓
Robot
Enter fullscreen mode Exit fullscreen mode

This separation makes the system easier to test and safer to operate.

Testing

Evaluate the system using representative scenarios:

  • Static objects
  • Moving people
  • Multiple objects
  • Low light
  • Bright light
  • Partial occlusion
  • Camera vibration
  • Device thermal throttling

Measure both detection accuracy and real-time performance.

Conclusion

Object detection on Android can provide useful edge perception for autonomous robots. Kotlin, CameraX, and a mobile inference runtime create a flexible foundation that can later be connected to ROS 2, sensor fusion, navigation, and Physical AI agents.

Useful Links

SDK Flutter: https://github.com/v-modal/vmodal_sdk_flutter

SDK Android: https://github.com/v-modal/vmodal_sdk_android

Discord: https://discord.gg/K72z28KUx

Top comments (0)