Code-Guided Reasoning for Small Language Models: Evaluating Executable MCQA Scaffolds

Researchers proposed Code-Guided Reasoning, a standardized evaluation framework and generated-program set for testing small language models on MCQA.

Across 20,000+ results from six models, assisted reasoning often beat direct answering in the main benchmark split.

The gains were not universal: some tasks regressed, answer extraction was brittle, and generated code sometimes ignored instructions.

The study highlights both the promise and the practical costs of external tools and executable prompts for benchmarking and deployment.