Ropedia has introduced Xperience 2, a Physical AI dataset that combines first-person recordings of human activity with reconstructed 3D environments to generate robot training data and simulation scenarios. The system is designed to help robotics developers reuse human demonstrations across different training and evaluation tasks.
Collecting training data using physical robots remains resource-intensive, particularly as developers work towards systems capable of handling increasingly complex tasks. By linking recorded human movements to the environments where they occur, Xperience 2 addresses this challenge and allows the same data to support robot motion trajectories, simulated interactions and multi-step task training without requiring physical robots at every stage.
Connecting human movements with 3D environments
Capture begins with Ropedia’s HOMIE head-mounted device, whose recordings Xperience 2 pairs with detailed 3D reconstructions of the spaces where those activities take place. Data can be collected in homes, restaurants, retail stores, offices and outdoor environments, covering human actions across different physical settings. Ropedia’s processing pipeline then aligns camera trajectories, hand and body movements, and other recorded signals with the corresponding 3D environment, preserving both the actions being performed and their surrounding spatial context so developers can use the resulting data for simulation, training and evaluation.
In its current form, the Xperience dataset comprises 1,812 reconstructed scenes from 596 collection locations alongside 4,230 task recordings covering activities such as object manipulation, cleaning and household tasks. Developers can convert these environments into reusable simulation scenes where robots practise tasks with different layouts, object arrangements and starting conditions.
Within Ropedia’s wider data infrastructure, the dataset sits alongside HOMIE capture hardware, the Foundry data processing platform and Physical data-as-a-Service. According to the company, Xperience now covers 10 million real-world interaction clips, representing approximately 100,000 hours of data, and has accumulated more than 2.7 million downloads on Hugging Face.
Four methods for generating robot training data
To turn captured human activity and reconstructed environments into training resources for robots, Xperience 2 supports four distinct approaches. The first, Real-to-Sim, reconstructs objects from real-world scans and adds collision properties, joints and physical parameters so they can be manipulated in simulation. In one demonstration involving a single home, that process generated 107 objects, including 73 that could be opened, pressed or deformed.
Human-Robot Retargeting converts recorded human movements into robot joint trajectories, adapting demonstrations to different robotic configurations so the same human activity recordings can support training across different robot designs. For creating training experiences directly using reconstructed environments, Virtual Teleoperation generates robot interaction demonstrations within simulated environments without requiring physical robots or manual resets.
The fourth approach, Long-Horizon Trajectory Generation, targets tasks that require several connected operations by creating extended sequences of robot actions involving multiple objects and rooms. Ropedia demonstrated this capability using a simulated wheeled dual-arm robot that performed eight consecutive subtasks in a reconstructed environment, including retrieving a cup, operating a microwave and placing the cup in another room.
Across 30 executions involving different starting configurations, the simulated robot completed all eight subtasks without intermediate resets. Each trajectory averaged approximately 2.2 minutes, with generation taking between 10 and 16 minutes on a single simulation worker.
Using simulation to improve robot training and evaluation
Once an environment and task have been configured, Xperience 2 can use computing resources to generate additional robot training trajectories without requiring new human demonstrations or physical robot data collection. Existing recordings and reconstructed environments can therefore support multiple training scenarios.
Beyond trajectory generation, the infrastructure also allows researchers to evaluate different robotic models under consistent conditions. By using the same tasks and starting configurations, developers can compare completion rates, examine failure patterns and identify where additional training data may be required. Researchers can also vary object positions, scene layouts and robot configurations to examine how models perform under different conditions, with failures observed during these simulations informing subsequent data collection by helping identify which objects, actions or environmental conditions need further attention.
“To scale Physical AI, we need infrastructure that makes every real-world experience more valuable by connecting it to the environments, actions, and outcomes that robots need to learn from. With Xperience 2, we are building towards a data infrastructure where human experience, 3D environments, simulation, and evaluation work together, allowing a single demonstration to support multiple training and development workflows,” said Zhaoxi Chen, Co-Founder and CEO of Ropedia.
Xperience 2 builds on Ropedia’s earlier Xperience-10M work as the company continues developing its Physical AI data infrastructure. Ropedia currently serves more than 20 robotics and foundation model companies and operates offices in four locations worldwide.
