Domain Randomization for Robust Robot Learning

Domain randomization is the idea that, instead of trying to perfectly match simulation to reality, you randomize simulation parameters widely enough that reality just looks like "one more variation" the policy has already seen. It's one of the highest-leverage techniques in sim-to-real robot learning, and it's straightforward to bolt onto an existing MuJoCo (or other simulator) pipeline.

The Core Idea

If you train a policy across a wide distribution of:

• lighting conditions
• object colors/textures
• camera positions and lens distortion
• friction, mass, and damping values
• sensor noise levels

...then the real world simply becomes one more sample from a distribution the policy has already learned to handle, rather than an out-of-distribution shock.

Categories of Randomization

Visual Randomization

Targets the visual gap between rendered and real images.

import numpy as np

def randomize_visuals(model, data):
    # Randomize object color
    obj_geom_id = model.geom("target_object_geom").id
    model.geom_rgba[obj_geom_id][:3] = np.random.uniform(0, 1, 3)

    # Randomize lighting
    light_id = model.light("main_light").id
    model.light_diffuse[light_id] = np.random.uniform(0.4, 1.0, 3)
    model.light_pos[light_id] += np.random.uniform(-0.3, 0.3, 3)

    # Randomize table/background texture (if using textured materials)
    table_mat_id = model.material("table_material").id
    model.mat_rgba[table_mat_id][:3] = np.random.uniform(0.2, 0.8, 3)

Dynamics Randomization

Targets the physics/dynamics gap.

def randomize_dynamics(model, data):
    # Randomize friction
    for geom_id in [model.geom("target_object_geom").id, model.geom("table").id]:
        model.geom_friction[geom_id][0] = np.random.uniform(0.4, 1.2)

    # Randomize object mass
    obj_body_id = model.body("target_object").id
    model.body_mass[obj_body_id] = np.random.uniform(0.05, 0.3)

    # Randomize actuator gains
    for i in range(model.nu):
        model.actuator_gainprm[i][0] *= np.random.uniform(0.85, 1.15)

Sensor and Latency Randomization

Targets the sensing/timing gap, which is often overlooked but very impactful.

def randomize_observation(obs, image_noise_std=0.02, state_noise_std=0.01, latency_steps=(0, 3)):
    noisy_obs = obs.copy()

    noisy_obs["image"] = np.clip(
        obs["image"] + np.random.normal(0, image_noise_std, obs["image"].shape), 0, 1
    )
    noisy_obs["joint_positions"] = obs["joint_positions"] + np.random.normal(
        0, state_noise_std, obs["joint_positions"].shape
    )

    # Simulate variable sensor latency by optionally returning a delayed observation
    delay = np.random.randint(*latency_steps)
    noisy_obs["_simulated_delay"] = delay

    return noisy_obs

Camera Pose Randomization

Small camera placement errors are inevitable when mounting real hardware — simulate that variance too.

def randomize_camera_pose(model, camera_name, pos_noise=0.01, angle_noise_deg=2.0):
    cam_id = model.camera(camera_name).id
    model.cam_pos[cam_id] += np.random.uniform(-pos_noise, pos_noise, 3)

    angle_rad = np.radians(np.random.uniform(-angle_noise_deg, angle_noise_deg, 3))
    # Apply small rotation perturbation to cam_quat via your rotation utility of choice
    model.cam_quat[cam_id] = perturb_quaternion(model.cam_quat[cam_id], angle_rad)

Wiring Randomization into the Environment

Apply randomization at reset() time so every episode is a fresh sample from the randomized distribution:

class DomainRandomizedEnv(RenderedMuJoCoEnv):
    def __init__(self, xml_path, randomize=True, **kwargs):
        super().__init__(xml_path, **kwargs)
        self.randomize = randomize

    def reset(self):
        obs = super().reset()
        if self.randomize:
            randomize_visuals(self.model, self.data)
            randomize_dynamics(self.model, self.data)
            mujoco.mj_forward(self.model, self.data)
            obs = self._get_obs()
            obs["image"] = self.render()
        return obs

    def step(self, action):
        obs, reward, done, info = super().step(action)
        if self.randomize:
            obs = randomize_observation(obs)
        return obs, reward, done, info

How Much Randomization Is Too Much?

This is the central tuning problem of domain randomization: too little and you don't cover the real-world distribution; too much and the task becomes so variable the policy fails to learn anything useful (or requires far more capacity and data).

A practical approach:

1. Start with a narrow randomization range around your best system-identified parameters
2. Train and evaluate in sim; confirm the policy still solves the task reliably across the randomized range
3. Gradually widen ranges, re-training and re-evaluating, until sim performance starts to degrade meaningfully
4. Test the resulting policy on real hardware and compare against a non-randomized baseline

def randomization_sweep(base_ranges, scale_factors, env_factory, train_fn, eval_fn):
    results = {}
    for scale in scale_factors:
        scaled_ranges = {k: (lo * scale, hi * scale) for k, (lo, hi) in base_ranges.items()}
        env = env_factory(randomization_ranges=scaled_ranges)
        policy = train_fn(env)
        success_rate = eval_fn(env, policy)
        results[scale] = success_rate
        print(f"Randomization scale {scale}: {success_rate:.1%} sim success")
    return results

Curriculum-Style Randomization

Rather than fixed randomization ranges for the entire training run, many pipelines ramp randomization up over training — starting narrow (easier to learn) and widening as the policy improves:

def get_randomization_scale(training_step, total_steps, start_scale=0.2, end_scale=1.0):
    progress = min(training_step / total_steps, 1.0)
    return start_scale + progress * (end_scale - start_scale)

Practical Tips

• Randomize what actually varies in your deployment environment. If your robot always operates under fixed studio lighting, heavy lighting randomization may cost more in sample efficiency than it buys in robustness — profile your real environment first.
• Track per-parameter ablations. When a policy fails to transfer, disable randomization dimensions one at a time in evaluation to isolate which one the policy is actually sensitive to.
• Combine with the fine-tuning strategy from the sim-to-real tutorial — domain randomization gets you a robust starting policy; a small amount of real data closes the remaining gap.

What's Next

Domain randomization pairs naturally with generating large volumes of synthetic training data — the subject of the final tutorial in this series.

Useful Links

Website: www.v-modal.com
SDK Flutter: v-modal/vmodal_sdk_flutter
SDK Android: v-modal/vmodal_sdk_android
Discord: https://discord.gg/K72z28KUx