“Corporate GPU utilization is only 5%”... Elice, Nota, and Lablup present solutions

Cast AI found that in poorly optimized enterprise Kubernetes environments, average GPU utilization was only about 5%.

The findings underscore that simply buying more GPUs does not improve AI competitiveness unless operational efficiency is raised.

Korean AI infrastructure firms are proposing fixes such as GPU virtualization, resource scheduling, model slimming and quantization, and inference optimization.

They also point to integrated monitoring and observability as key tools to remove bottlenecks and boost effective GPU use.