Real2Gym
Real-World Videos to
Executable Robot Simulations
From a human gesture or a robot demonstration to a world that robots can act in.
Reconstruct in Blender. Execute in MuJoCo. Refine with GPT6 Astra.
A mug. A microwave. A robot.
Place the mug inside, withdraw the hand, and close the door. A human demonstration retargeted to dual FR3 + Franka Hand.
Native task completedAbout this run 374 frames · 30 fps
Different demonstrations.
One path to simulation.
Six source scenes. Eight robot embodiments and demonstrations.
Frame-matched Real RGB / Blender RGB / MuJoCo RGB.
Showing all 8 demonstrations.
One change.
A visibly different world.
Geometry, position, orientation, table height, textures, and distractors.
Plus three background settings. Compare each one with the original.
Microwave scaled to 75%
The microwave is 25% smaller in all three dimensions. The action is adapted to its smaller cavity.
WITHDRAW → CLOSE DOOR
One category changes in each example. Both views are MuJoCo, with the same camera and event index; playback follows source timing (3× native speed). Augmented scenes have no paired real recording.
Bring your own
real-world demonstration.
Explore the workflow, try a scene, or build on the framework.
Explore the code@software{real2gym,
author = {{Real2Gym Contributors}},
title = {{Real2Gym: Real-World Videos to
Executable Robot Simulations}},
year = {2026},
url = {https://github.com/cskrren/Real2Gym}
}