When Does Multi-Agent RL Improve LLM Workflows? Workflow, Scale, and Policy-Sharing Tradeoffs

Researchers evaluated end-to-end RL for multi-agent LLM workflows across roles, tasks, and model sizes.

RL usually improved over base models, but gains varied widely by workflow design, task type, and scale.

Isolated-policy training could reach higher peaks, but it was more likely to suffer abrupt collapse.

Shared-policy training changed the failure mode rather than eliminating it, reflecting role-specific gradient dynamics and workflow routing.