Gated DeltaNet-2: Decoupling Erase and Write in Linear Attention

Gated DeltaNet-2 is a new linear attention model for long-context language modeling and retrieval.

It separates memory erasure and writing with channel-wise erase and write gates, plus adaptive forgetting and channel-wise decay.

This decoupling improves accuracy on retrieval-heavy tasks while keeping training and decoding efficient.

On 1.3B-parameter long-context benchmarks, it reports state-of-the-art results among the compared efficient sequence models.