Fewer Clarifications, Better Code: Benchmarking Cross-Session Personalized Ambiguity Adaptation in Coding Assistants

The CAPA benchmark tests whether coding assistants can use a user’s past sessions to resolve recurring ambiguities more effectively.

It evaluates 12 LLMs on code generation across 600 sessions and 300 held-out evaluations, comparing performance with and without same-user history.

The work targets a common failure mode: assistants repeatedly asking the same clarifying questions to the same user across sessions.

To reduce that friction, it proposes same-user history gating, a lightweight inference-time strategy for personalized disambiguation.