Orthrus-Qwen3: up to 7.8×tokens/forward on Qwen3, identical output distribution | Hacker News

Orthrus-Qwen3 boosts Qwen3 inference by predicting multiple tokens in parallel and then verifying them.

The method reports up to 7.8× more tokens per forward pass while preserving the same output distribution.

This can make local LLM inference much faster, especially on GPU setups limited by memory bandwidth and latency.

It is a speculative decoding-style approach aimed at higher throughput without changing model behavior.