Microsoft is expanding its AI capabilities across all Windows 11 PCs by rolling out Copilot Vision and Actions, alongside voice interaction features. Previously reserved for premium users, these AI tools are now accessible to all, aiming to redefine how users interact with their devices.
Enhanced AI Features
With the new update, Windows 11 users can now use the wake word “Hey, Copilot” to interact directly with the Copilot. This voice functionality is intended to complement the mouse and keyboard as primary input methods. Microsoft notes an increased Copilot usage when voice commands are utilized, highlighting its potential to simplify user interactions.
Copilot Vision allows AI to view the screen and offer context-sensitive assistance, using a floating toolbar called Highlights to help users navigate interface elements or execute tasks. This feature, previously limited to select devices with an NPU, will now be available broadly, enhancing interactions across various applications, including gaming and productivity tools.
Copilot Actions and Connectors
Copilot Actions enable users to perform tasks within apps and the operating system through natural language prompts. Tasks such as toggling settings, batch editing photos, or extracting data from documents can now be executed with simple commands. This functionality, similar to Apple's Spotlight Actions, extends Microsoft's goal of integrating AI as a primary interface for the OS.
Additionally, Microsoft has previewed Connectors for Windows Insiders, allowing Copilot to access third-party apps like Google Calendar and Gmail. Through Connectors, Copilot can open files, manage emails, and create presentations, demonstrating its potential to streamline workflows significantly.
Looking Ahead
While Copilot Actions will roll out to mainstream Windows users in the coming months, further advancements remain exclusive to devices with more powerful NPUs. These include features like Click to Do and enhanced on-device processing.
Despite these advancements, the adoption rate of these features by users might vary, as the migration from Windows 10 to Windows 11 has been gradual. Microsoft's ambition is for Copilot to evolve into a primary interface, similar to how users utilize shortcuts and scripts in other ecosystems.
