DEV Community

vmodal_ai
vmodal_ai

Posted on

Building a Kotlin Voice-Controlled NVIDIA Jetson Robot

Building a Kotlin Voice-Controlled NVIDIA Jetson Robot

Voice can provide a natural interface for Physical AI systems. The important design principle is to convert voice into validated high-level robot intents rather than allowing arbitrary text to directly control motors.

Architecture 

Android microphone
      ↓
Speech recognition
      ↓
Kotlin intent parser
      ↓
Validated command
      ↓
Jetson gateway
      ↓
ROS 2
      ↓
Robot
Enter fullscreen mode Exit fullscreen mode

1. Define robot intents 

sealed interface RobotIntent {
    data object Stop : RobotIntent
    data object StartPatrol : RobotIntent
    data class GoTo(val location: String) : RobotIntent
}
Enter fullscreen mode Exit fullscreen mode

2. Keep commands explicit

For example:

"Start patrol" → START_PATROL
"Stop robot"   → STOP
"Go to lab"    → GO_TO("lab")
Enter fullscreen mode Exit fullscreen mode

Avoid translating arbitrary natural-language text directly into low-level velocity commands.

3. Add confirmation for risky actions 

data class PendingAction(
    val intent: RobotIntent,
    val requiresConfirmation: Boolean
)
Enter fullscreen mode Exit fullscreen mode

For consequential actions, show a confirmation dialog before execution.

4. Send to Jetson

The Jetson gateway validates the intent and translates it into the appropriate ROS 2 action or service.

Kotlin intent
     ↓
Authentication
     ↓
Authorization
     ↓
Validation
     ↓
ROS 2 action/service
Enter fullscreen mode Exit fullscreen mode

5. Handle feedback

ROS 2 actions are useful for long-running tasks because Android can receive progress and completion state.

Example UI:

Task: Navigate to Lab
Status: MOVING
Progress: 64%
Enter fullscreen mode Exit fullscreen mode

6. Add a stop path

Voice recognition should never be the only emergency mechanism. A dedicated stop control should remain available.

Conclusion

Kotlin can provide a clean voice interface while NVIDIA Jetson and ROS 2 remain responsible for the physical execution layer.

References

Useful Links

Website: www.v-modal.com

SDK Flutter: https://github.com/v-modal/vmodal_sdk_flutter

SDK Android: https://github.com/v-modal/vmodal_sdk_android

Discord: https://discord.gg/K72z28KUx

Reddit: https://www.reddit.com/r/v_modal/

Top comments (0)