Mix-Quant: Quantized Prefilling, Precise Decoding for Agentic LLMs

Mix-Quant introduces phase-aware quantization for agentic LLM inference.

It quantizes the compute-heavy prefilling stage with NVFP4 while keeping decoding in BF16.

Experiments show prefilling can handle stronger quantization with little accuracy loss.

The approach reduces latency and improves hardware efficiency for long-context, multi-step AI agents.