Salient Knowledge Pathways: Sparse Cross-Modal Routing for Efficient Knowledge-Intensive Multimodal Question Answering

Researchers introduced SKIP, a sparse inference architecture for knowledge-intensive multimodal QA.

It uses question-guided visual pruning, sparse retrieval, sparse cross-attention, and adaptive budget control.

On five benchmarks, SKIP matches or outperforms strong dense baselines while using far fewer FLOPs and lower latency.

The team also presents an information-bottleneck analysis to explain optimal sparsity for efficient multimodal reasoning.