I Tried Running LLMs on Intel's NPU. Here's What Actually Happened.

Local LLM inference was tested on a Lenovo ThinkPad T14 Gen 5.

The laptop had an Intel Core Ultra 7 155U, 32 GB DDR5, Windows 11, and Intel AI Boost NPU 3720.

The article compared NPU, CPU, and llama.cpp performance.

Qwen2.5-7B-Instruct was exported to OpenVINO IR with int4 weights and loaded on the NPU.

The NPU compiler hit "LLVM ERROR" because it requires static tensor shapes; Qwen2.5-1.5B-Instruct also crashed.