OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models

OmniScope is a training-free token compression method that handles audio and video separately for omnimodal LLMs.

Using the query as a shared anchor, it estimates modality-specific relevance, prunes visual tokens with an anchor-delta method, and merges audio tokens within each second.

It outperformed baselines on four benchmarks and two Qwen2.5-Omni scales, delivering up to 3.53x faster prefill and over 15% lower GPU memory use at 25% token retention.

The work tackles cross-modal salience mismatch, offering a practical way to cut inference cost without sacrificing much accuracy.