From AI Action Tokens to Physical Robot Motor Commands

Modern AI systems can generate high-level actions, structured commands, or action tokens. A physical robot, however, ultimately needs bounded actuator commands.

The engineering challenge is building a safe translation layer between these two worlds.

The Complete Pipeline

AI Model
   ↓
Action Tokens / Structured Intent
   ↓
Action Parser
   ↓
Task / Motion Planner
   ↓
Trajectory Generator
   ↓
Controller
   ↓
ros2_control
   ↓
Hardware Interface
   ↓
Motor Driver
   ↓
Physical Robot

The AI should not normally have direct access to raw motor commands.

1. Define a Robot Action Vocabulary

Start with a constrained action space:

MOVE
TURN
GRASP
RELEASE
LOOK
NAVIGATE
STOP

An AI model might produce:

{
  "action": "MOVE",
  "linear_velocity": 0.3,
  "duration": 2.0
}

The parser validates the structure before anything reaches the controller.

2. Parse and Validate

def validate_action(action):
    allowed = {
        "MOVE",
        "TURN",
        "STOP"
    }

    if action["type"] not in allowed:
        return False

    if action["type"] == "MOVE":
        if abs(action["velocity"]) > MAX_VELOCITY:
            return False

    return True

Never treat model output as trusted hardware input.

3. Convert Intent to a Trajectory

Suppose AI requests:

Move forward at 0.3 m/s for 2 seconds.

A trajectory generator can create:

t=0.0 → v=0.0
t=0.2 → v=0.3
...
t=1.8 → v=0.3
t=2.0 → v=0.0

The controller then tracks this trajectory.

4. Use Robot Kinematics

For a mobile robot, desired linear and angular velocities can be converted to wheel velocities.

For a differential-drive robot:

v_left  = (v - ωL/2) / r
v_right = (v + ωL/2) / r

where:

• v = linear velocity
• ω = angular velocity
• L = wheel separation
• r = wheel radius

The resulting wheel targets are still passed through limits and controllers.

5. Use Controllers for Physical Execution

AI Intent
   ↓
Trajectory
   ↓
Controller
   ↓
Command Interface
   ↓
Hardware

ros2_control provides the abstraction between controllers and hardware using state and command interfaces. ros2_control

Its architecture is based around a read-update-write loop:

read()
  ↓
controller update
  ↓
write()

6. Add a Safety Gate

A safety gate can inspect every action:

def safety_gate(command, state):
    if state.emergency_stop:
        return StopCommand()

    if command.velocity > state.max_velocity:
        command.velocity = state.max_velocity

    if obstacle_too_close(state):
        return StopCommand()

    return command

This creates an explicit boundary:

AI
 ↓
Safety Gate
 ↓
Controller

7. Handle Action Tokens

If an AI model outputs discrete tokens such as:

<MOVE>
<FORWARD>
<SPEED_03>
<DURATION_2>

a decoder converts them into structured data:

Token Sequence
      ↓
Action Decoder
      ↓
Structured Command

The token vocabulary should be intentionally constrained.

8. Add State Feedback

The robot should continuously feed state back into the system.

Sensors
   ↓
State Estimator
   ↓
Robot State
   ↓
Planner / AI
   ↓
Next Action

This creates a closed loop rather than a one-shot command generator.

9. Verify Action Results

Suppose the AI requests:

GRASP red_box

The robot should verify:

Gripper closed?
Object detected?
Object lifted?

If verification fails:

Retry
   or
Recover
   or
Request Human Help

10. Handle Latency and Stale Commands

AI inference can be slow.

If a command was generated 2 seconds ago, it may no longer be valid.

Attach:

timestamp
sequence number
expiration time

to action messages.

For example:

{
  "command": "MOVE",
  "velocity": 0.3,
  "created_at": 1720000000,
  "expires_at": 1720000000.5
}

The control system should reject expired commands.

11. Build a Layered Safety Architecture

A robust system can use several boundaries:

AI Policy
   ↓
Semantic Validation
   ↓
Task Planner
   ↓
Motion Constraints
   ↓
Safety Supervisor
   ↓
Low-Level Controller
   ↓
Hardware Limits
   ↓
Motor

No single AI component should be trusted with unrestricted physical authority.

12. Test the Translation Layer

Create tests for:

Valid AI action
Invalid action
Missing fields
Out-of-range velocity
Expired command
Emergency stop
Obstacle detected
Controller failure
Hardware communication failure

Also test adversarial or unexpected model outputs.

13. Example End-to-End Flow

User:

"Move toward the charging station."

AI:

{
  "action": "NAVIGATE",
  "target": "charging_station"
}

Planner:

Charging station
      ↓
Navigation Goal
      ↓
Trajectory

Controller:

Trajectory
    ↓
Velocity commands

Hardware:

Velocity
   ↓
Wheel commands
   ↓
Motors

Feedback:

Encoders + Sensors
        ↓
Robot State
        ↓
Planner

The process continues until the goal is verified or the system enters a safe recovery state.

Conclusion

The difficult part of connecting AI to robotics is not converting text into a motor command. It is creating the layers that make that conversion safe, deterministic, observable, and reversible.

AI should generate intentions or structured actions. Planning should convert those intentions into feasible motion. Controllers should track that motion, while ros2_control and hardware interfaces provide the final boundary to physical actuators.

That architecture allows increasingly capable AI systems to control robots without removing the engineering safeguards required by physical systems.

Useful Links

Website: www.v-modal.com
SDK Flutter: https://github.com/v-modal/vmodal_sdk_flutter
SDK Android: https://github.com/v-modal/vmodal_sdk_android
Discord: https://discord.gg/K72z28KUx