Fuhao Li, Wenxuan Song, Han Zhao +5 authors
Vision-language-action (VLA) models have recently shown strong potential in
enabling robots to follow language instructions and execute precise actions.
However, most VLAs are built upon vision-language models pretrained solely on
2D data, which lack accurate spatial awareness and hinder their ability to
operate in the 3D physical world. Existing solutions attempt to incorporate
explicit 3D sensor inputs such as depth maps or point clouds, but these
approaches face challenges due to sensor noise, hardware heterogeneity, and
incomplete depth coverage in existing datasets. Alternative methods that
estimate 3D cues from 2D images also suffer from the limited performance of
depth estimators.We propose Spatial Forcing (SF), a simple yet effective
alignment strategy that implicitly forces VLA models to develop spatial
comprehension capabilities without relying on explicit 3D inputs or depth
estimators. SF aligns intermediate visual embeddings of VLAs with geometric
representations produced by pretrained 3D foundation models. By enforcing
alignment at intermediate layers, SF guides VLAs to encode richer spatial
representations that enhance action precision.Extensive experiments in
simulation and real-world environments demonstrate that SF achieves
state-of-the-art results, surpassing both 2D- and 3D-based VLAs. SF further
accelerates training by up to 3.8x and improves data efficiency across diverse
robotic tasks. Project page is at https://spatial-forcing.github.io/