Researchers train AI model that achieves near-full performance with just 12.5 percent of its experts

AI2 and UC Berkeley introduced EMO, a mixture-of-experts language model trained to use document boundaries to drive cleaner expert specialization.

EMO matched a standard MoE model at full size, but unlike the baseline it kept nearly all of its performance after pruning to 25% and even 12.5% of its experts.

The standard MoE degraded sharply under the same pruning, suggesting EMO is much more modular.

The findings point to more memory-efficient MoE systems and make it easier to extract domain-specific submodels with minimal loss.