Meet Mu, the small language mannequin in command of Microsoft’s Settings AI agent

In short: Small language fashions are typically extra compact and environment friendly than LLMs, as they’re designed to run on native {hardware} or edge units. Microsoft is now bringing one more SLM to Home windows 11, as customers apparently want a number of AI-powered hints to assist them discover particular OS settings and customise their PC expertise.

Microsoft just lately introduced Mu, a brand new small language mannequin designed to combine with the Home windows 11 UI expertise. Mu will work alongside Phi Silica – the language mannequin beforehand launched on Copilot+ PCs – to energy the AI agent characteristic in Home windows Settings that the corporate is at present growing.

Like Phi Silica, Mu will run totally on the system’s neural processing unit, delivering responses at over 100 tokens per second, which ought to present a easy person expertise in Settings.

Microsoft describes Mu as an environment friendly 330-million-parameter mannequin optimized for small-scale deployments. It’s based mostly on a transformer encoder-decoder structure, the place the encoder converts person enter right into a fixed-length latent illustration, and the decoder generates output tokens from that illustration.

Microsoft designed Mu as a extremely optimized “sibling” to Phi Silica, utilizing a number of methods to cut back the parameter rely and enhance effectivity. The SLM was skilled utilizing an unspecified variety of A100 GPUs on the Azure Machine Studying platform, progressing by way of a number of phases and making use of methods that Redmond developed throughout work on the Phi fashions.

After fine-tuning the mannequin, the builders discovered that Mu delivers efficiency practically similar to Phi-3.5-mini, regardless of being simply one-tenth its measurement. Moreover, Microsoft collaborated with silicon companions Intel, AMD, and Qualcomm to make sure that Mu is totally optimized for operating on native NPU-powered machines.

What was the purpose of all these optimization efforts? In accordance with Redmond, Mu will energy the brand new “agentic” expertise within the upcoming AI-enhanced Settings app. The AI and cloud big goals to offer a brand new “chat” characteristic that lets Home windows customers ask settings-related questions in pure language and obtain efficient solutions, even once they barely know what they’re searching for.

Agentic AI within the Settings app is meant to enhance Home windows’ ease of use. Microsoft acknowledged that coaching an SLM able to dealing with (and modifying) lots of of system settings was a problem, and the corporate says it welcomes person suggestions on the brand new characteristic.

Personally, I by no means skilled any usability or discoverability points with Settings throughout eight years of utilizing Home windows 10, but Microsoft stays bullish about remodeling Home windows into an “AI-first” platform.