A Guide to Choosing AWS Compute for Distributed Training (Part 1: Model Scale and Hardware Selection) | Amazon Web Services

AWS released a guide for choosing compute for distributed training, covering scale-up vs. scale-out, GPU instance selection, networking, and data pipeline bottlenecks.

It recommends single-instance or tightly coupled setups for 10M–10B models, UltraClusters with EFA for 10B–100B, and specialized architectures like UltraServer for 100B–1T+.

The guide highlights the impact of parallelism choices, memory limits, and storage throughput, including FSx for Lustre, data sharding, and data loader tuning.

AWS also notes key differences between CPU-based HPC and GPU ML clusters, plus the importance of avoiding EFA/TCP fallback for performance.