DEV Community

vmodal_ai
vmodal_ai

Posted on

Android + NVIDIA Jetson + ROS 2: Building an AI Robot

#android #kotlin #jetson #ros2

Android + NVIDIA Jetson + ROS 2: Building an AI Robot

Introduction

A practical Physical AI system often separates the user interface, AI compute, robotics middleware, and hardware control.

In this architecture, Android provides the operator interface, an NVIDIA Jetson provides edge AI compute, and ROS 2 coordinates robotics workloads.

Architecture 

             Android / Kotlin
                    |
             Secure Gateway
                    |
                  ROS 2
             /      |              Vision     Nav       Control
          |
    NVIDIA Jetson
          |
     AI Inference
          |
    Robot Sensors
          |
     Robot Hardware
Enter fullscreen mode Exit fullscreen mode

This separation makes it possible to upgrade individual components without rebuilding the entire system.

Android Application

The Android application can provide:

  • Robot connection status
  • Camera stream
  • AI detections
  • Battery information
  • Navigation controls
  • Emergency stop
  • Robot diagnostics

Use Jetpack Compose to build the operator interface.

Jetson AI Computer

The Jetson can run computationally intensive workloads such as:

  • Object detection
  • Object tracking
  • Depth estimation
  • Visual SLAM
  • Navigation
  • Sensor fusion

The Android device does not need to perform every AI operation itself.

ROS 2 Layer

ROS 2 provides communication between robotics components.

A possible topic layout is:

/cmd_vel
/odom
/scan
/camera/image
/detections
/battery_state
/robot_status
Enter fullscreen mode Exit fullscreen mode

Keep the topic structure small and intentional for the mobile interface.

Android-to-ROS Gateway

Rather than making Android responsible for ROS 2 internals, use a gateway:

Android
   |
WebSocket / MQTT / ROS bridge
   |
ROS 2 Gateway
   |
ROS 2 Nodes
Enter fullscreen mode Exit fullscreen mode

The gateway can authenticate clients, validate commands, and expose only approved functionality.

AI Perception Pipeline

The Jetson can process camera frames:

Camera
   ↓
ROS 2 Image Topic
   ↓
Jetson AI Node
   ↓
Detection / Tracking
   ↓
ROS 2 Detection Topic
Enter fullscreen mode Exit fullscreen mode

The Android app can subscribe to summarized results rather than receiving raw sensor data when bandwidth is limited.

Android Dashboard

The dashboard can display:

Robot: ONLINE
Battery: 87%
Mode: AUTONOMOUS
Objects: 4
Position: X 2.3 / Y 4.8
Enter fullscreen mode Exit fullscreen mode

Compose state can be backed by Kotlin StateFlow.

Command Flow

For manual control:

Android
   ↓
Velocity Command
   ↓
Gateway
   ↓
ROS 2
   ↓
Safety Controller
   ↓
Robot Base
Enter fullscreen mode Exit fullscreen mode

The safety controller should remain authoritative over the physical robot.

Autonomous Mode

For autonomous operation:

Sensors
   ↓
Jetson Perception
   ↓
Localization
   ↓
Navigation
   ↓
Safety Controller
   ↓
Robot
Enter fullscreen mode Exit fullscreen mode

Android becomes a monitoring and supervisory interface rather than the primary controller.

Security

A production system should include:

  • Device authentication
  • Encrypted communication
  • Command authorization
  • Network segmentation
  • Rate limiting
  • Robot-side safety limits
  • Emergency stop

Never assume that a mobile application being inside the same Wi-Fi network makes the robot network trusted.

Testing

Use simulation before deploying to hardware.

Validate:

  • ROS 2 topic communication
  • AI inference
  • Android connectivity
  • Command timeouts
  • Network interruptions
  • Camera streaming
  • Safety behavior

Scaling to Robot Fleets

The same architecture can support multiple robots:

Android
   |
Fleet Gateway
   |
+--+---------+---------+
|            |         |
Robot 01   Robot 02  Robot 03
Enter fullscreen mode Exit fullscreen mode

Each robot can expose a controlled namespace and telemetry stream.

Conclusion

Android, NVIDIA Jetson, and ROS 2 form a strong architecture for Physical AI applications. Android handles human interaction, Jetson handles demanding edge AI workloads, and ROS 2 coordinates perception, navigation, and control.

This architecture can later be extended with LLM-based planning, voice interaction, computer vision, and autonomous task execution.

Useful Links

SDK Flutter: https://github.com/v-modal/vmodal_sdk_flutter

SDK Android: https://github.com/v-modal/vmodal_sdk_android

Discord: https://discord.gg/K72z28KUx

Top comments (0)