DenoiseRL: Bootstrapping Reasoning Models to Recover from Noisy Prefixes

Researchers introduced DenoiseRL, a reinforcement learning framework that improves large language model reasoning by learning from failed traces.

The method uses incorrect reasoning from weaker models to train recovery, exploration, and self-correction, reducing dependence on stronger teacher models and curated data.

DenoiseRL outperforms strong on-policy RL baselines on math and general reasoning benchmarks.

The approach offers a more scalable path to better reasoning models by learning from mistakes instead of only successful examples.