Collaborative Weighting with Pessimistic Critic for Mitigating Overestimation in Off-Policy Reinforcement Learning

Researchers introduced Collaborative Weighting Actor-Critic (CWAC), a new framework to reduce overestimation bias in off-policy reinforcement learning.

CWAC combines distributional critics, collaborative reweighting of TD errors and uncertainty, and stochastic pessimistic value estimation to stabilize training.

The method is designed to plug into existing algorithms such as SAC, TD3, and DDPG with little extra cost.

The authors report stronger performance on simulated control tasks, suggesting more reliable value learning.