LLMs Get Lost in Evolving User Intent

Researchers turned single-turn benchmarks into multi-turn conversations where user intent can change over time.

Across multiple tasks and model families, strong static benchmark performers often lost accuracy when intent was revealed, revised, or redirected.

The study highlights a gap between benchmark results and real-world conversational agents, which must track shifting user goals.