I picked up a Surface with a Snapdragon X Elite chip partly because of the on-device AI story. The NPU has 45 TOPS of compute, and Microsoft markets these devices specifically around local AI workloads. What I found when I actually tried to run a local LLM is that most of the popular tools completely ignore the NPU. Ollama runs on CPU only on ARM Windows. LM Studio does too, despite advertising GPU acceleration. The NPU just sits idle. ...
When trying to set up my machine to run Azure Functions locally using Visual Studio Code, I ran across an error. After generating an empty HTTP-triggered Azure Function and running it, a big red error popped up in the terminal: func : File C:\Users\Hendr\AppData\Roaming\npm\func.ps1 cannot be loaded because running scripts is disabled on this system. For more information, see about_Execution_Policies at https:/go.microsoft.com/fwlink/?LinkID=135170. I happened to try the same thing on a different machine, and got exactly the same behavior. It seems that this is something that you need to resolve yourself on most Windows machines. Browsing to the URL provided in the error message, we get an explanation of what an Execution Policy is exactly: ...