Less Decoder is More Encoder: Geometric Representation Learning from Novel View Synthesis

Novel view synthesis should teach models about 3D structure, yet encoderbased approaches produce weak representations. The authors blame two architectural choices: decoders that are too spatially expressive, which dilute the encoder's role, and pixel-space targets, which hinder feature learning. SNAP, a self-supervised transformer, uses a pose-conditioned local decoder and a latent-space reconstruction objective. It competes with geometry-supervised methods across localization, pose estimation, correspondence, depth estimation, and robot manipulation. Its features show emergent viewpoint invariance and degrade gracefully under camera shifts. Authors: Keerthi Kaashyap, Dennis Anthony, Akshay Krishnan, Nhi Ngoc Nguyen, Jeremy Collins, James Hays, Shreyas Kousik, Animesh Garg Paper: https://arxiv.org/abs/2610.03717v1