Kotlin + NVIDIA Jetson: Building a Real-Time Robot Camera Dashboard

This tutorial shows how Kotlin can provide a responsive Android dashboard while NVIDIA Jetson performs camera inference.

Architecture

Robot Camera
     ↓
 NVIDIA Jetson
     ↓
AI inference
     ↓
Detection metadata
     ↓
Robot Gateway
     ↓
Android Kotlin

Sending detection metadata instead of every raw frame can reduce mobile bandwidth.

1. Define a detection model

data class Detection(
    val label: String,
    val confidence: Float,
    val x: Float,
    val y: Float,
    val width: Float,
    val height: Float
)

2. Represent perception state

data class PerceptionState(
    val detections: List<Detection> = emptyList(),
    val inferenceMs: Long = 0
)

Use StateFlow to expose state to Compose.

class PerceptionViewModel : ViewModel() {
    private val _state = kotlinx.coroutines.flow.MutableStateFlow(PerceptionState())
    val state = _state.asStateFlow()
}

3. Process frames on Jetson

A typical pipeline is:

Camera
  ↓
ROS 2 image topic
  ↓
AI inference
  ↓
Detection / tracking
  ↓
JSON or protobuf result
  ↓
Gateway

NVIDIA Isaac ROS includes GPU-accelerated perception packages designed for Jetson and ROS 2.

4. Display detections

Create an overlay composable using the normalized coordinates received from Jetson.

data class Box(
    val left: Float,
    val top: Float,
    val right: Float,
    val bottom: Float
)

Normalize coordinates to the displayed image size so the UI remains resolution-independent.

5. Keep AI off the UI thread

Use coroutines and flows for network and state updates.

viewModelScope.launch {
    gateway.observeDetections().collect { detections ->
        _state.update { it.copy(detections = detections) }
    }
}

6. Handle stale detections

Attach a timestamp or sequence number to every inference result. The Android client should discard old results rather than displaying stale robot perception.