DEV Community

vmodal_ai
vmodal_ai

Posted on

Building a Remote Robot Control App with Flutter and NVIDIA Jetson

#nvidia #jetson #ros2 #robotics

Building a Remote Robot Control App with Flutter and NVIDIA Jetson

Build a remote-control architecture in which Flutter is the operator interface and Jetson is the edge computer.

The tutorial focuses on command acknowledgement, connection state, heartbeat messages, emergency-stop behavior, and safe handling of network loss.

What You Will Build

By the end of this tutorial, you will have:

  • A clear Jetson/ROS 2 architecture.
  • A working development workspace.
  • A small, testable robotics pipeline.
  • A path for connecting the system to Flutter where applicable.
  • Basic logging, testing, and troubleshooting practices.

Prerequisites

You should have:

  • An NVIDIA Jetson developer kit or compatible NVIDIA edge platform.
  • A stable Linux/Jetson software environment.
  • Basic Linux terminal knowledge.
  • Basic Python or C++ knowledge.
  • Familiarity with ROS 2 concepts such as nodes, topics, services, and actions.
  • A network connection between the robot computer and development machine.

Version note: NVIDIA Jetson, JetPack, CUDA, TensorRT, Isaac ROS, and ROS 2 compatibility changes over time. Check the current NVIDIA support matrix and the documentation for your exact board before installing packages. Do not blindly mix commands from different JetPack/ROS 2 releases.

Step 1: Prepare the Jetson

Start by confirming the device and installed software:

uname -a
cat /etc/os-release
Enter fullscreen mode Exit fullscreen mode

Then update package metadata:

sudo apt update
Enter fullscreen mode Exit fullscreen mode

Keep the base system consistent with the JetPack release supported by your target robotics stack.

Step 2: Install and Verify ROS 2

Install the ROS 2 distribution supported by your Jetson/Isaac ROS combination.

After installation, source ROS 2:

source /opt/ros/<ros-distro>/setup.bash
Enter fullscreen mode Exit fullscreen mode

Verify that ROS 2 is available:

ros2 --help
Enter fullscreen mode Exit fullscreen mode

Add the source command to your shell configuration if appropriate:

echo "source /opt/ros/<ros-distro>/setup.bash" >> ~/.bashrc
source ~/.bashrc
Enter fullscreen mode Exit fullscreen mode

Step 3: Create a ROS 2 Workspace 

mkdir -p ~/robot_ws/src
cd ~/robot_ws
colcon build
source install/setup.bash
Enter fullscreen mode Exit fullscreen mode

A typical workspace becomes:

robot_ws/
├── src/
├── build/
├── install/
└── log/
Enter fullscreen mode Exit fullscreen mode

Step 4: Create a Package

For Python:

cd ~/robot_ws/src
ros2 pkg create --build-type ament_python robot_ai_demo
Enter fullscreen mode Exit fullscreen mode

For C++:

ros2 pkg create --build-type ament_cmake robot_ai_demo_cpp
Enter fullscreen mode Exit fullscreen mode

Choose the language that best matches the latency and integration requirements of your application.

Step 5: Understand the Data Flow

A production robot should separate responsibilities.

Sensors
   |
   v
ROS 2 Drivers
   |
   v
Perception / Localization
   |
   v
Decision / Mission Logic
   |
   v
Safety Layer
   |
   v
Motor Controller
Enter fullscreen mode Exit fullscreen mode

For a Flutter operator application:

Flutter
   |
HTTPS / WebSocket
   |
Robot Gateway
   |
ROS 2
   |
Jetson
   |
Robot
Enter fullscreen mode Exit fullscreen mode

The Flutter application should normally communicate with a controlled gateway instead of directly exposing the ROS graph to the public internet.

Step 6: Publish a Simple ROS 2 Message

Create a small publisher and subscriber, then build the workspace:

cd ~/robot_ws
colcon build --symlink-install
source install/setup.bash
Enter fullscreen mode Exit fullscreen mode

Run the publisher:

ros2 run robot_ai_demo publisher
Enter fullscreen mode Exit fullscreen mode

In another terminal:

source ~/robot_ws/install/setup.bash
ros2 topic list
ros2 topic echo /robot_status
Enter fullscreen mode Exit fullscreen mode

This simple test proves that your ROS 2 environment is functioning before you add cameras, AI models, or motor controllers.

Step 7: Add the Main AI/Robot Component

For this tutorial, the main component is conceptually one of:

  • Camera and object detector
  • LiDAR and navigation stack
  • TensorRT inference node
  • Isaac ROS perception node
  • Robot telemetry collector
  • Fleet gateway
  • Voice/LLM intent service

Keep this component independent from the UI. Publish structured ROS 2 messages instead of UI-specific data.

Example:

camera/image
       |
       v
object_detector
       |
       v
/objects
       |
       +----> decision_node
       |
       +----> telemetry_gateway
Enter fullscreen mode Exit fullscreen mode

Step 8: Add Logging and Diagnostics

At minimum, log:

  • Node startup/shutdown.
  • Sensor connection failures.
  • Inference errors.
  • Network disconnects.
  • Safety-state changes.
  • Command acknowledgements.
  • Processing latency.

Useful ROS 2 commands include:

ros2 node list
ros2 topic list
ros2 topic info /robot_status
ros2 topic hz /robot_status
Enter fullscreen mode Exit fullscreen mode

Step 9: Add a Safety Layer

Never allow an AI model or remote UI to directly bypass safety logic.

A simple command path should be:

User/AI Intent
     |
     v
Command Validation
     |
     v
Robot State Check
     |
     v
Safety Rules
     |
     v
ROS 2 Command
Enter fullscreen mode Exit fullscreen mode

Examples of safety rules:

  • Stop if communication heartbeat expires.
  • Stop if a critical sensor fails.
  • Reject invalid velocity ranges.
  • Reject commands while the robot is in an unsafe state.
  • Give emergency stop the highest priority.

Step 10: Connect Flutter When Applicable

For Flutter projects, expose a small API such as:

GET  /api/robot/status
GET  /api/robot/telemetry
POST /api/robot/command
WS   /ws/robot
Enter fullscreen mode Exit fullscreen mode

Example WebSocket payload:

{
  "type": "command",
  "command": "stop",
  "sequence": 1024
}
Enter fullscreen mode Exit fullscreen mode

Flutter can then maintain:

ConnectionState
RobotState
TelemetryState
MissionState
AlertState
Enter fullscreen mode Exit fullscreen mode

Use BLoC, Riverpod, or another state-management approach to keep network events separate from presentation.

Step 11: Test the System

Test one layer at a time.

ROS 2 

ros2 topic list
ros2 topic echo /robot_status
Enter fullscreen mode Exit fullscreen mode

AI

Measure:

  • Model load time.
  • Preprocessing time.
  • Inference latency.
  • Postprocessing time.
  • End-to-end latency.

Network

Test:

  • Normal connection.
  • Temporary disconnect.
  • Reconnect.
  • Duplicate messages.
  • Delayed messages.

Safety

Verify:

  • Emergency stop.
  • Heartbeat timeout.
  • Sensor failure.
  • Invalid command.
  • Jetson restart.

Step 12: Optimize for Jetson

Do not optimize before measuring.

Record a baseline and then investigate:

  1. CPU utilization.
  2. GPU utilization.
  3. Memory consumption.
  4. Temperature.
  5. Power mode.
  6. Camera pipeline latency.
  7. AI inference latency.
  8. ROS 2 message latency.

For NVIDIA-accelerated applications, investigate TensorRT, DeepStream, and Isaac ROS where they match the workload.

Step 13: Make the Deployment Reproducible

Record:

Jetson model:
JetPack:
CUDA:
TensorRT:
ROS 2:
Isaac ROS:
Python:
Model:
Camera:
LiDAR:
Enter fullscreen mode Exit fullscreen mode

For serious deployments, containerize the application and keep configuration separate from application code.

Step 14: Troubleshooting

ROS 2 command not found 

source /opt/ros/<ros-distro>/setup.bash
Enter fullscreen mode Exit fullscreen mode

Package not found 

source ~/robot_ws/install/setup.bash
ros2 pkg list | grep robot
Enter fullscreen mode Exit fullscreen mode

Topic has no data

Check:

ros2 topic list
ros2 topic info /your_topic
ros2 topic hz /your_topic
Enter fullscreen mode Exit fullscreen mode

Then verify that the sensor publisher is actually running.

AI inference is too slow

Profile the complete pipeline. Do not assume the neural network is the only bottleneck. Camera conversion, memory copies, preprocessing, ROS serialization, and postprocessing can all contribute significant latency.

Flutter is disconnected

Implement:

  • reconnect with backoff,
  • heartbeat messages,
  • connection state,
  • command acknowledgement,
  • timeout handling.

Step 15: Production Checklist

Before deploying a robot, verify:

  • [ ] Hardware/software versions are documented.
  • [ ] ROS 2 nodes restart safely.
  • [ ] Sensor failures are detected.
  • [ ] Commands are validated.
  • [ ] Emergency stop works independently.
  • [ ] Network loss causes a safe state.
  • [ ] AI inference is monitored.
  • [ ] Logs are retained.
  • [ ] Telemetry is available.
  • [ ] The deployment can be reproduced.

Conclusion

NVIDIA Jetson is most useful when it is treated as an edge-computing platform inside a larger robotics architecture rather than simply as a small Linux computer. ROS 2 provides the communication and modularity layer, while NVIDIA acceleration can handle demanding perception workloads.

For Flutter-based robotics applications, a gateway between Flutter and ROS 2 creates a clean separation: the mobile application focuses on user experience, while Jetson and ROS 2 remain responsible for robot-side computation.

Useful Links

Top comments (0)