Explore Labs

High-Fidelity Digital Twin Lab

Explore Labs

Enter Lab Back to overview

Simulation is usually a way of making a problem easier: strip out the noise, hand the planner a clean world, study the algorithm in isolation. This laboratory uses it the other way round — at a fidelity that puts the difficulty back, so that a plan which succeeds here has had to earn it. What is tested is not whether the reasoning is right in principle, but how much of it survives an encounter with a body.

It is also a laboratory anyone can walk into. High-fidelity embodied research is normally gated by hardware and by weeks of integration; here it is a link. A GPU-backed session opens in your browser with everything already assembled, so an experiment becomes something another group can rerun rather than rebuild.

Inside, you are equipped with high-fidelity digital-twin assets — the robots and the domestic environments they work in — and with open-source CRAM plans that use them to carry out a range of everyday household tasks.

Example Videos

Kitchen Manipulation with GARMI

GARMI opens a drawer, perceives the worktop, transports an object into the drawer, and opens and closes a cabinet door under whole-body control.

Multiple Robots in the AICOR Apartment

Different robots at work in the same AICOR apartment, each paired with the environment at run time.

Generating Demonstration Data at Scale

The twin is a source of training data as well as a place to run plans: 100 episodes of "put the cup into the sink" are generated with the cup placed at random and recorded from four cameras in LeRobot format. An ACT policy trained on them succeeds in 14 of 50 evaluation rollouts, which is what makes the data worth generating.

Browse the episodes in the LeRobot dataset visualiser.

Source Code

The laboratory is open source and developed at github.com/yxzhan/cram-vrb-lab.

Runs this lab on your own machine — an NVIDIA RTX GPU with Docker and the NVIDIA Container Toolkit — with a single command:

curl -fsSL https://raw.githubusercontent.com/yxzhan/cram-vrb-lab/dev/install.sh | bash

 

It pulls the latest pre-built docker image and starts vscode at http://localhost:8888/vscode.

Architecture

The plan, the world model and the controller sit above a ROS 2 interface; the Isaac Sim scene sits below it, in the position a physical robot would occupy.

Software Components

CRAM: A software toolbox for implementing autonomous robots.

Semantic Digital Twin: A unified world model bridging geometry, kinematics and asserted meaning, shared by the planner and the perception pipeline.

GISKARD: A framework for constraint- and optimization-based robot motion and planning control.

ROBOKUDO: A perception framework targeted for robot manipulation tasks.

NVIDIA Isaac Sim: A GPU-accelerated robotics simulator providing the photorealistic rendering and PhysX contact simulation this laboratory is built on.

Back to top