Balancing Fidelity and Diversity in Diffusion Models via Symmetric Attention Decomposition: Hopfield Perspective

Researchers decompose the pre-softmax attention matrix into symmetric and skew-symmetric parts to study controllable diffusion generation.

They interpret the symmetric part as an energy-landscape stability signal and the skew-symmetric part as circulation dynamics in associative memory.

These stability measures correlate with the fidelity-diversity trade-off, suggesting a principled way to steer generation quality.

The paper proposes a knob for adjusting that trade-off and provides a GitHub repository for the method.