YOLO Object Detection on Android for Robotics
Introduction
Object detection is an important capability for autonomous robots. A robot can use detections to identify people, vehicles, tools, obstacles, and other objects in its environment.
YOLO-family models are widely used for real-time object detection. In this tutorial, we will design an Android application that captures camera frames and performs YOLO inference locally.
System Architecture
CameraX
|
Preprocessing
|
YOLO Model
|
Postprocessing
|
Bounding Boxes
|
Robot Perception Layer
The exact model format depends on the runtime you choose. For Android edge deployment, an exported model may be converted to a mobile-compatible format and executed using an appropriate inference runtime.
Android Project
Organize the project into separate layers:
vision/
├── CameraManager.kt
├── YoloDetector.kt
├── Detection.kt
└── DetectionOverlay.kt
This prevents camera handling, inference, and rendering from becoming tightly coupled.
Detection Model
Create a Kotlin model:
data class Detection(
val classId: Int,
val label: String,
val confidence: Float,
val boundingBox: RectF
)
Camera Pipeline
Use CameraX ImageAnalysis to obtain frames.
imageAnalysis.setAnalyzer(executor) { image ->
detector.process(image)
image.close()
}
For real-time robotics, use a backpressure strategy that drops stale frames rather than allowing an inference queue to grow indefinitely.
Preprocessing
Most object-detection models expect a fixed input size.
The preprocessing stage normally performs:
- Rotation correction
- Resize
- Color conversion
- Normalization
- Tensor creation
For example:
Camera Frame
↓
Resize
↓
Normalize
↓
Tensor
↓
YOLO
The preprocessing code must match the model's training/export requirements.
Running Inference
Create a detector abstraction:
class YoloDetector {
suspend fun detect(frame: ImageFrame): List<Detection> {
// preprocess
// inference
// postprocess
return emptyList()
}
}
Run inference outside the Android main thread.
Postprocessing
Object detectors can return multiple candidate boxes. Postprocessing commonly includes:
- Confidence filtering
- Class filtering
- Bounding-box conversion
- Non-Maximum Suppression
For example:
Raw Predictions
↓
Confidence Filter
↓
NMS
↓
Final Detections
Drawing Bounding Boxes
The Android UI can display detection results over the live camera preview.
+--------------------------+
| |
| +------------+ |
| | bottle | |
| | 92% | |
| +------------+ |
| |
+--------------------------+
Remember to map coordinates correctly when the preview and model input have different aspect ratios.
Using Detection for Robotics
Detection results can be passed to a robot control layer:
{
"object": "person",
"confidence": 0.92,
"bbox": [120, 80, 350, 500]
}
The robotics layer can combine this information with depth, odometry, LiDAR, or other sensors.
Do not treat a 2D bounding box as a physical distance measurement unless the system has additional calibration or depth information.
Performance Optimization
For edge robotics:
- Use a lightweight model where possible.
- Reduce inference resolution when acceptable.
- Reuse buffers.
- Avoid unnecessary bitmap allocations.
- Run inference off the UI thread.
- Process only the latest frame.
- Measure latency and thermal behavior.
A smaller model running consistently can be more useful for robotics than a larger model with unstable frame rates.
Testing
Test the detector with:
- Indoor scenes
- Outdoor scenes
- Low light
- Moving objects
- Multiple objects
- Camera rotation
- Network disconnected
For robotics, also measure end-to-end latency:
Capture → Inference → Decision → Robot Command
Conclusion
YOLO-style object detection can turn an Android device into a useful edge-vision component for robotics. Kotlin, CameraX, and a mobile inference runtime provide the foundation for building perception prototypes that can later integrate with ROS 2 and autonomous navigation.
Useful Links
SDK Flutter: https://github.com/v-modal/vmodal_sdk_flutter
SDK Android: https://github.com/v-modal/vmodal_sdk_android
Discord: https://discord.gg/K72z28KUx
Top comments (0)