3 min read

RoboThoughts #4: Dive into RoboVerse - Domain Randomization

Source: http://x.com/i/article/2028477325462122496

In robotic Reinforcement Learning (RL) and Imitation Learning (IL), policies often overfit to specific visual or physical parameters in simulation. As a core component of Axis Robotics’ data infrastructure, the RoboVerse Domain Randomization (DR) system addresses this problem by introducing controlled variations to reduce the model’s reliance on non-essential environment features.

By injecting diversity across scene composition, material properties, lighting conditions, and camera parameters, the system forces the model to learn generalized, underlying physical and geometric features, enabling a robust transfer from pure simulation to the physical world.

Design Philosophy: Separation of Concerns

In traditional simulators, object creation and property modification are often entangled, leading to an exponential explosion in code complexity. The central design principle of the RoboVerse DR system is the Separation of Concerns. It strictly divides simulation variations into two distinct operations, handled by separate modules:

Lifecycle Management: Handled primarily by the SceneRandomizer, which directly manipulates the USD (Universal Scene Description) Stage. It focuses purely on the creation, deletion, and switching of "dynamic objects."

Property Editing: Handled by specialized randomizers (e.g., MaterialRandomizer, LightRandomizer). It focuses on modifying the attributes of "existing objects" (like altering material roughness or adjusting light intensity) without caring about the object's creation or destruction.

In the code implementation, this decoupling is reflected in extremely clean component initialization:

python

# 1. Lifecycle Management: Handles "life and death" (e.g., spawning different tables, backgrounds)
scene_rand = SceneRandomizer(scene_cfg, seed=42)

# 2. Property Editing: Handles "feature modification" (e.g., changing table material or friction without altering geometry)
mat_rand = MaterialRandomizer(mat_cfg, seed=43)
light_rand = LightRandomizer(light_cfg, seed=44)

System Framework: Object Abstraction and Hierarchical Architecture

To support this logic, RoboVerse establishes a rigorous object taxonomy and a hierarchical spatial framework.

Unified Object Registry and Object Dualism

Constrained by the underlying physics engine, objects are divided into two categories: Static Objects (exist at initialization, participate in physics, e.g., robots) and Dynamic Objects (generated at runtime by the SceneRandomizer, purely visual, e.g., background walls).

To bridge this divide, RoboVerse introduces ObjectRegistry. All randomizers can query target objects through a unified interface, eliminating the burden of manually managing complex USD Prim paths.

python

# Fetch the global registry singleton
registry = ObjectRegistry.get_instance()

# Unified querying: Static objects managed by the physics engine (e.g., robotic arms)
static_only = registry.list_objects(lifecycle='static')
# Unified querying: Dynamically loaded visual distractors
dynamic_only = registry.list_objects(lifecycle='dynamic')

# Randomizers can directly fetch the underlying USD path for property manipulation
meta = registry.get_object_metadata("table")
prim_paths = meta.prim_paths

Three-Layer Scene Hierarchy

For spatial organization, the system discards imperative setup scripts in favor of highly readable Declarative Configuration, semantically divided into three layers:

python

from metasim.randomization import (
    SceneRandomCfg,
    EnvironmentLayerCfg,
    WorkspaceLayerCfg,
    ObjectsLayerCfg,
    ManualGeometryCfg,   
    USDAssetPoolCfg
)

scene_cfg = SceneRandomCfg(
    # Layer 0 - Environment: Macro background (e.g., complete Kujiale interior room layouts)
    environment_layer=EnvironmentLayerCfg(
        elements=[ManualGeometryCfg(name="floor", ...),
            ManualGeometryCfg(name="walls", ...),],
        shared=True,  # Single instance for all environments
        z_offset=0.0
    ),
    # Layer 1 - Workspace: Robotic manipulation surface (e.g., various tables from the EmbodiedGen dataset)
    workspace_layer=WorkspaceLayerCfg(
        elements=[USDAssetPoolCfg(name="table", usd_paths=table_paths, selection_strategy="random")],
        shared=True
    ),
    # Layer 2 - Objects: Visual diversity distractors (e.g., randomly scattered desktop clutter)
    objects_layer=ObjectsLayerCfg(
         elements=[
            USDAssetPoolCfg(name="distractors", usd_paths=[...])
        ],
        shared=False  # Different objects per environment
    ) 
)

Progressive Strategy and System Advantages

To ensure academic rigor and controllability, the system employs a Multi-level Progressive Randomization Strategy: Level 0 (Baseline) -> Level 1 (Scene & Materials) -> Level 2 (+ Lighting) -> Level 3 (+ Camera Perturbation). Researchers can systematically isolate and quantify the impact of each randomization factor on the policy's generalization.

In the RL environment reset() workflow, this is reflected in a strict execution order:

python

# Level 0: Baseline (deterministic)
scene_rand()  # Fixed materials

# Level 1: Add material randomization
scene_rand()
mat_rand()

# Level 2: Add lighting
scene_rand()
mat_rand()
light_rand()

# Level 3: Add camera
scene_rand()
mat_rand()
light_rand()
cam_rand()

Beyond strict logic, the RoboVerse DR architecture offers several key advantages:

  • Strict Reproducibility:

All randomizers inherit from a unified base class, BaseRandomizerType, which maintains completely independent random number generators. By passing a Seed (e.g., seed=42), the system allocates independent random streams, ensuring absolute consistency in scene perturbations even during distributed training.

  • Seamless Hybrid Simulation Support:

Randomizers automatically bind to IsaacLab (physics) or IsaacSim (rendering) based on their specific API needs, remaining completely transparent to the developer.

  • Automated Asset Management:

Combined with USDAssetPoolCfg, the system natively integrates on-demand downloading and caching of cloud assets. It automatically converts URDF models to USD format on the fly, achieving high memory efficiency through USD idempotent operations.

The RoboVerse Domain Randomization system is fundamentally a scene generation middleware deeply rebuilt upon the physics engine's underlying logic and USD features. It provides an academically rigorous and engineering-efficient proving ground for physical AI to transition into reality.

Read More

https://roboverse.wiki/metasim/concept/randomization