Spectral Transformation for Layer-wise Global Rank Discovery in Federated LoRA for Vision Transformers

Researchers introduced SpecTraL, a federated LoRA framework for Vision Transformers that improves federated fine-tuning without dense adapter reconstruction.

It stacks client LoRA adapters and applies latent-space spectral transforms, including Householder-based operations, to reduce aggregation error and server overhead.

SpecTraL also uses a spiked covariance model from random matrix theory to infer optimal layer-wise global ranks automatically.

A padding-aware initialization lets clients keep training without repeatedly merging adapters into the base model.

Experiments on federated ViT benchmarks such as DomainNet and NICO++ show better accuracy-communication trade-offs for ViT-B/16 and ViT-L/16.