SaaSBench: Exploring the Boundaries of Coding Agents in Long-Horizon Enterprise SaaS Engineering

Researchers introduced SaaSBench, a benchmark for evaluating AI coding agents on realistic enterprise SaaS engineering tasks.

The benchmark includes 30 tasks across 6 SaaS domains, 5,370 validation nodes, and heterogeneous software stacks.

Experiments show most agent failures occur during system configuration and integration, not in core logic generation.

The findings highlight a major gap between coding agents’ current abilities and real-world enterprise software readiness.