LatentOmni: Rethinking Omni-Modal Understanding via Unified Audio-Visual Latent Reasoning

Researchers introduced LatentOmni, a cross-modal framework for audio-visual reasoning that interleaves text with latent sensory states.

It adds feature-level supervision and temporal consistency embedding to better preserve fine-grained evidence across modalities.

The method, paired with components like Omni-Sync Position Embedding and LatentOmni-Instruct-35K, outperforms explicit text-based chain-of-thought on several benchmarks.

The results suggest latent-space reasoning can improve temporal grounding and overall performance in multimodal large language models.