Microsoft’s Azure Maia chief on the complex future of AI compute

Microsoft developed Maia 200 to reduce AI inference cost on Azure by focusing on latency and on-chip capacity.

Maia 200 offers 216GB HBM3e, 272MB on-die SRAM, and about 7TB/s memory bandwidth to keep models on-chip.

Azure will present Maia 200 through abstraction layers and allow other chips for workloads that need them.

Maia 200 is designed for inference rather than training and is available via SDKs with Triton and PyTorch in preview.