How can embedding models bind concepts?

Researchers found that CLIP-like vision-language embeddings represent scenes as additive object pieces, but the model still needs a complex binding function to link concepts to the right objects.

That binding mechanism does not generalize well to unseen object-concept combinations, helping explain common failures in multi-object understanding.

In controlled transformer models trained from scratch, binding generalization emerged once training data covered enough concept combinations.

Those simpler models learned binding functions with multiplicative interactions, suggesting better concept binding can arise with sufficient data and the right architecture.