Relay-style orchestration revitalizes national computing power, and PD separation finally breaks through: latency cut in half, costs down nearly 40%!

At WAIC 2026, Infinigence AI released the full technical report for PDD, a cross-cluster heterogeneous LLM inference architecture.

PDD splits inference into Prefill, RelayDecode, and MainDecode: a local relay starts decoding first, then a remote cluster takes over.

The design reduces WAN KV Cache transfer latency, while RadixCache further cuts cross-cluster data movement by reusing prefixes.

In testing, PDD cut first-token latency by 51.5% and lowered per-token cost by 37.5%, making distributed GPU clusters more practical for serving.