Would You Walk to the Car Wash? Revealing the Salience Bias of Large Language Models in Commonsense Reasoning

A new study finds that large language models often show salience bias in commonsense reasoning, getting led astray by explicit but irrelevant cues.

Using the SaliTrap benchmark, the researchers tested 12 state-of-the-art models and found that distractors like numbers frequently disrupted their answers.

The paper argues these errors are mostly caused by knowledge suppression under misleading framing, not by a lack of commonsense knowledge.

It also shows that simple inference-time prompting can recover much of the lost performance, suggesting a low-cost fix and a useful diagnostic benchmark.