Study Finds Multi-Turn RL Gains for LLM Agents Depend on Turn Budgets

A new arXiv paper from researchers studying agentic reinforcement learning examines what multi-turn training actually improves in LLM agents. Evaluating models with Pass@(K,T), which varies sampling breadth K and interaction depth T, the authors report that trajectory-level RL advantages require sufficient turn budgets and shrink noticeably when those budgets are tightened. The gains also do not evenly raise single-step decision quality; instead they concentrate in error recovery and long-horizon resilience. To address the deployment risk of budget dependence, the paper discusses process supervision signals and introduces BudgetMix RL, which varies interaction budgets during training. Experiments reported by the authors show it holding up across different execution settings. A project page accompanies the abstract.