Trust the Batch, On- or Off-Policy: Adaptive Policy Optimization for RL Post-Training

Researchers proposed a batch-adaptive RL post-training objective that uses policy-ratio statistics to balance trust-region updates and off-policy data.

Instead of fixed clipping, the method normalizes effective sample size to automatically adjust update strength and regularization.

The approach reduces sensitivity to hyperparameters and training settings, which can improve stability in mismatched or scaled systems.

Experiments showed competitive or better performance across benchmarks.