SpatialDex: Does 3D Video Pretraining Help Bimanual Manipulation?

Can ordinary RGB human videos teach a robot to coordinate two dexterous hands? This episode explains how SpatialDex reconstructs spatial context from video and uses it to train bimanual manipulation policies. We cover three findings: why a shared video-level scale matters for aligning hands with their surroundings; how geometry enters both the policy's observations and its prediction targets; and what the matched pretraining experiment reveals beyond the headline baseline comparison. SpatialDex reports a 38.3-percentage-point real-robot improvement over its strongest tested baseline and a 14.2-point simulated improvement from geometry-rich versus motion-only pretraining. ([arxiv.org](https://arxiv.org/abs/2610.29716)) The evaluation includes simulated Allegro–Franka and Wuji–Tianji systems, plus physical Wuji hands on Tianji arms performing tasks such as handovers, spraying, unzipping, and sweeping. Human pretraining videos are RGB-only; robot execution uses RGB-D observations. The central limitation is causal attribution: the matched geometry-pretraining controls are evaluated in simulation, while the physical results compare complete systems. ([arxiv.org](https://arxiv.org/html/2610.29716v1)) Listen for a technical account of what reconstruction contributes—and why the relationship between a hand trajectory and its surroundings may be the most valuable thing to recover from human video.