Scaling Continual Learning to 300+ Tasks with Bi-Level Routing Mixture-of-Experts

Researchers introduced CaRE, a continual learning framework built on a bi-level routing mixture-of-experts design.

CaRE activates task-relevant routers and experts in two stages, helping preserve performance across long task sequences.

The method shows strong results on standard class-incremental learning benchmarks and on a new large-scale testbed.

The new OmniBenchmark-1K benchmark is designed to evaluate continual learning across hundreds of tasks at scale.