How NetEase Games cut LLM cold starts from 42 minutes to 30 seconds

NetEase Games found that model loading, not GPU scheduling, was the real bottleneck for serverless LLM inference.

By replacing direct cross-region storage access with a cache layer and Fluid prefetching on Kubernetes, it cut model load time from 42 minutes to 30 seconds.

The improvement made autoscaling and multi-tenant model serving much more practical for GPU infrastructure.

The case shows that elastic AI systems need fast, manageable model data access as much as compute availability.