Revealing Algorithmic Deductive Circuits for Logical Reasoning

Researchers mapped LLM reasoning steps to token logits and traced them with causal mediation analysis.

A small set of attention heads appears to fetch factual and rule-based details for local sub-tasks.

Higher layers then combine these signals into larger reasoning strategies, including graph traversal-like algorithms.

The findings clarify how LLMs do step-level inference and could guide more interpretable, controllable model design.