Geometry Matters: 3D Foundation Priors for Learning Semantic Correspondence

The paper introduces a post-training framework for 3D-aware semantic correspondence using reconstructed object geometry instead of coarse spherical priors.

It estimates pose and shape with SAM3D, then refines pose through render-and-compare optimization before projecting PartField descriptors into the image.

Matches are filtered with geodesic distances on the reconstructed surface, helping reject errors on symmetric or repeated parts.

A lightweight adapter is then trained on DINO and Stable Diffusion features using the filtered correspondences, without pose annotations.

The method reports stronger semantic correspondence than prior approaches while reducing reliance on manual supervision.