SCICONVBENCH: Benchmarking LLMs on Multi-Turn Clarification for Task Formulation in Computational Science

Researchers introduced SCICONVBENCH, a benchmark for testing how LLMs clarify ambiguous or inconsistent computational science requests in multi-turn dialogs.

The benchmark uses a task ontology and rubric-based scoring across four domains: fluid mechanics, solid mechanics, materials science, and partial differential equations.

Results show frontier models are better at resolving contradictions than asking for missing information, and they often make unsupported assumptions.

The work highlights a key gap in scientific AI assistants: reliably grounding task specifications before computation begins.