Guiding LLM Post-training Data Engineering with Model Internals from Sparse Autoencoders

Researchers introduced SAERL, a post-training data engineering framework that uses sparse autoencoder features from LLM internals to guide data selection.

SAERL controls batch diversity, builds difficulty-aware curricula, and filters low-quality data to make RL post-training more efficient.

On Qwen2.5-Math-1.5B, it improved average accuracy by 3.00% over vanilla GRPO and reached target performance with 20% fewer training steps.

The gains held across model scales and reinforcement learning methods, showing that internal sparse autoencoder signals can improve LLM post-training.