Windows ML Standardizes Machine Learning and AI for Developers

on March 8, 2018
Reading Time: 3 minutes

During its Windows Developer Day this week, Microsoft took the covers off of its plans to help accelerate and dominate in the world of machine learning. Windows ML is a new API that Microsoft will be including in the RS4 release of Windows 10 this year, enabling a new class of developer to access the power and capability of machine learning for their software.

Microsoft already uses machine learning and AI in Windows 10 and on its Azure cloud infrastructure. This ranges from analyzing live camera feeds to AI for game engines and even indexing for search functionality on your local machine. Cortana is the most explicit and public example of what Microsoft has built today, with the Photo-app based facial recognition and image classification being a close second.

Windows ML allows software developers to utilize pre-trained machine learning models to power new experiences and classifications of apps. The API allows for simple integration with existing Microsoft development tools like Visual Studio. Windows ML supports direct importing of ONNX (Open Neural Network Exchange) formatted files that represent deep learning compute models, allowing for easy transferal and sharing between application environments. This format was introduced by Microsoft and Facebook back in September of last year. Frameworks like Caffe2, PyTorch, and Cognitive support ONNX export, so models that are trained in them can utilize inference through any system that integrates ONNX.

To be clear, Windows ML isn’t intended to replace the training activity that you would run on larger, high-performance server clusters. Microsoft still touts its Azure Cloud infrastructure for that, but it does see benefits to pairing that with the Windows ML enabled software ecosystem on edge devices. Software that wants to support updating training models with end-user input can do so with significantly less bandwidth required, as only the much smaller, pre-defined Windows ML result would need to be returned.

With Windows ML, an entire new class of developer will be able to utilize machine learning and AI systems to improve the consumer experience. We will see spikes in AI-driven applications for image recognition, automated text generation, gaming, motion tracking, and so much more. There is a huge potential to be fulfilled by simply getting the power of machine learning into the hands of as many software developers as possible, and no one can offer that better than Microsoft.

Maybe the most exciting part about Windows ML to me is the support for hardware acceleration. The API will be able to run on CPUs, GPUs, and even newer AI-specific add-in hardware like the upcoming Intel Movidius chip. Using DirectX 12 hardware acceleration and DX12 compute capabilities that were expanded with Windows 10, Microsoft will allow application developers to write applications that don’t need to worry about code changes for the underlying hardware in a system to ensure compatibility. While performance will obviously scale from processor to processor, as will user experiences based on that, Windows ML aims to create the same kind of API layer advantages for machine learning as DirectX has done for gaming and graphics.

Microsoft not only will support discrete graphics solutions but also integrated graphics from Intel (and AMD I assume). Windows ML will be one of the first major users of Intel’s AVX-512 capabilities (vector extensions added to consumer hardware with Skylake-X) and Movidius dedicated AI processor. Qualcomm will also support the new API on its upcoming Always Connected PCs using the Snapdragon 835 platform, possibly opening us up to the first use case for the company’s dedicated on-chip AI Engine.

This new API will be supported with both Windows UWP apps (Windows Store) and Win32 apps (classic desktop apps).

It is still in the early phases of development when it comes to the true AI-driven future of computing. Microsoft has been a player in the consumer market with the Cortana integration on Windows, but it has seen limited success compared to the popularity of Google, Amazon, and even Apple systems. By enabling every Windows application developer to take advantage of machine learning with Windows ML, Microsoft will see significant movement in the space, much of it likely using its Azure cloud systems for training and management. And for consumers, the age of artificial intelligence ubiquity looks closer than ever.