GDSD: Reinforcement Learning as Guided Denoiser Self-Distillation for Diffusion Language Models

Researchers introduced Guided Denoiser Self-Distillation (GDSD) for training diffusion language models with reinforcement learning.

GDSD matches denoiser logits to a self-teacher derived from reverse-KL regularized RL, avoiding biased ELBO-based likelihood surrogates.

On planning, math, and coding benchmarks, LLaDA-8B and Dream-7B trained with GDSD showed higher accuracy and more stable training.

The reported gains reached up to 19.6%, suggesting a stronger path for improving diffusion LLMs without train-inference mismatch.