Beyond Prefill-Decode Disaggregation: Dissecting LLM Inference for Heterogeneous Platforms via Dynamic Operator Scheduling

Researchers introduced DOPS, a closed-loop, hardware-aware framework for heterogeneous LLM inference.

DOPS combines a stage-aware DAG with Bifocal for dynamic operator-to-device placement and WLA for blockwise weight-layout selection under memory limits.

On NPU+PIM systems, it outperformed prefill-decode disaggregation baselines and delivered stronger latency and scalability.

The work suggests heterogeneous LLM serving needs dynamic scheduling and weight-layout decisions, not just prefill-decode splitting.