DEV Community

vmodal_ai
vmodal_ai

Posted on

Kotlin + NVIDIA Jetson: Building an AI Task Interface for Robots

Kotlin + NVIDIA Jetson: Building an AI Task Interface for Robots

Large language models can provide a natural-language task interface, but the LLM should not directly control motors. A safer architecture converts natural language into a constrained task representation.

Architecture 

User
 ↓
Kotlin Android
 ↓
AI task planner
 ↓
Structured robot task
 ↓
Policy / validator
 ↓
Jetson gateway
 ↓
ROS 2 actions
Enter fullscreen mode Exit fullscreen mode

1. Define a structured task 

data class RobotTask(
    val action: String,
    val target: String?,
    val parameters: Map<String, String> = emptyMap()
)
Enter fullscreen mode Exit fullscreen mode

Example:

{
  "action": "navigate",
  "target": "warehouse_a"
}
Enter fullscreen mode Exit fullscreen mode

2. Validate the task 

fun isAllowed(task: RobotTask): Boolean {
    return task.action in setOf(
        "navigate",
        "inspect",
        "return_to_base",
        "stop"
    )
}
Enter fullscreen mode Exit fullscreen mode

The actual production validator should be stricter and run close to the robot.

3. Map tasks to ROS 2 

navigate → Navigation action
inspect → Inspection workflow
return_to_base → Navigation goal
stop → Safety stop command
Enter fullscreen mode Exit fullscreen mode

4. Show the plan to the operator

Before execution:

Requested:
"Inspect the loading area"

Plan:
1. Navigate to loading area
2. Start camera inspection
3. Report detected objects
4. Return to base
Enter fullscreen mode Exit fullscreen mode

The operator can approve or cancel the task.

5. Stream task progress 

data class TaskProgress(
    val taskId: String,
    val step: String,
    val progress: Int
)
Enter fullscreen mode Exit fullscreen mode

6. Keep physical safety local

AI-generated plans should never bypass:

  • Collision avoidance
  • Velocity limits
  • Geofencing
  • Hardware safety controllers
  • Emergency-stop systems

Conclusion

Kotlin can make advanced Physical AI capabilities accessible to operators without putting an LLM directly in the physical control loop.

References

Useful Links

Website: www.v-modal.com

SDK Flutter: https://github.com/v-modal/vmodal_sdk_flutter

SDK Android: https://github.com/v-modal/vmodal_sdk_android

Discord: https://discord.gg/K72z28KUx

Reddit: https://www.reddit.com/r/v_modal/

Top comments (0)