Windows ML Provides Developers with a New Runtime Platform for Local AI Model Processing on Windows 11 Devices
Microsoft launches Windows ML. It is a new runtime platform focused on local AI inference and simplified model deployment on Windows 11 devices. The technology is part of Windows AI Foundry and is designed to allow code to optimally utilize available client hardware, such as CPUs, GPUs, and NPUs. This is relevant, as the lightweight NPU in Windows 11 computers currently has little work to do.
One Platform for all Hardware
Windows ML supports a wide range of Windows hardware, from entry-level laptops and Copilot+ PCs to truly powerful workstation systems built for AI. Developers can run models on the hardware most suitable for their workload. For example, an NPU is efficient for energy-efficient processing of light and optimized AI workloads. GPUs provide more computing power, while CPUs ensure flexibility and broad compatibility.
The platform is based on ONNX Runtime (ORT) and uses existing ORT APIs. Thanks to this choice, developers can use existing models and workflows without having to rebuild everything.
The system optimizes workloads based on the present chips via Execution Providers (EPs), developed in collaboration with hardware partners such as AMD, Intel, NVIDIA, and Qualcomm. This eliminates the need for separate builds for different hardware configurations.
API Layer
Windows ML provides a unified API layer with two levels: a higher-level ML layer with runtime management tools and a lower layer that provides direct access to ONNX Runtime APIs. This allows for both simple integration and detailed control.
The AI Toolkit for Visual Studio Code helps developers with model conversion, optimization, quantization, and profiling. Windows ML supports on-device ahead-of-time model compilation (AOT), enabling applications to start faster and run more efficiently.
According to Microsoft, models perform up to 20 percent better on Windows ML compared to other model formats. Moreover, the platform provides compatibility across different Windows builds.
Important Step
The introduction of Windows ML is an important step towards the deployment of AI on laptops. Although laptop manufacturers, including Microsoft, have been talking about AI PCs for over a year, there are still not many AI-driven applications available. This runtime will certainly make it easier to efficiently use the included NPU, alongside other present components.
Windows ML is now available in public preview for all Windows 11 devices worldwide. Developers can get started immediately via Microsoft Learn and the AI Dev Gallery.