NVIDIA AI Introduces PivotRL: A New AI Framework Achieving High Agentic Accuracy With 4x Fewer Rollout Turns Efficiently

PivotRL introduces a framework that uses SFT trajectories and turn-level updates to train agentic models framework that trains on SFT trajectories with turn-level updates.

Pivot Filtering selects pivot turns where empirical reward variance is nonzero and reward mean is below a threshold filters pivot candidate turns using a frozen reference policy and two statistical conditions.

Functional Rewards assigns reward 1 to actions that belong to a verifier-defined set of acceptable actions for a state rewards actions that pass a domain-specific verifier rather than requiring exact string matches.

Theorem 3.2 shows the Fisher norm of the natural gradient of the statewise reward objective scales with the reward standard deviation relates GRPO gradient signal to reward dispersion.

Theorem 3.3 shows functional-reward RL shifts probability mass toward acceptable actions while preserving relative ordering of unrelated actions addresses policy change under functional rewards.