ReMoE: Boosting Expert Reuse through Router Fine-Tuning in Memory-Constrained MoE LLM Inference

Researchers introduced ReMoE, a router fine-tuning method for MoE LLM inference that makes token routing more stable.

By biasing routing toward recently used experts, ReMoE raises expert reuse by 26% and improves cache locality.

The result is fewer expert fetches from slow storage, leading to higher throughput and lower time per output token.

The method showed real-system gains on models and platforms including DeepSeek, Qwen, vLLM, and Jetson Orin NX without extra runtime compute.