An Innovative AI Tool for Windows PCs
Click To Do, Microsoft’s promising new AI feature, emerges as the latest innovation for the company’s Copilot+ PCs. This integrated solution aims to enhance user experience by enabling simple, actionable commands through intuitive interactions.
To activate Click To Do, users can either hold down the Windows key and click once with the left mouse button or use the keyboard shortcut Windows key + Q. If these actions do not trigger any response, it indicates the absence of a Copilot+ PC environment, since the feature is exclusive to such devices.
Technological Integration and Features
Upon activation, Click To Do highlights on-screen text and images, framing them within a selectable outline. This feature, which originated as part of the Windows Recall tool, offers a variety of contextual actions, making it a significant step forward from its predecessor. What's noteworthy is that Click To Do does not run continuously in the background, but instead operates on-demand, thus optimizing system resources effectively.
Click To Do captures a screenshot of the current screen and processes many commands on-device. The Copilot+ PCs utilize a specialized NPU-powered Phi Silica language model to execute complex text operations like summarizing content, generating bulleted lists, or rewriting text in varying tones. Meanwhile, actions requiring external data interaction, such as “Search the web” or “Visual search with Bing,” communicate with Microsoft servers, ensuring data privacy unless users opt to share information.
The integration of Optical Character Recognition (OCR) technology allows for selectable screen text, unlocking potential for various immediate actions like sending an email when selecting an address, or opening a website when a URL is chosen. The feature expands significantly when selecting over ten words, enabling diverse AI-driven actions.
AI-Driven Image Manipulation
In addition to text-based actions, Click To Do offers powerful AI tools for image manipulation. Users can perform actions such as “Blur background,” “Erase objects,” or “Remove background,” streamlining complex photo editing tasks within Windows without additional software. For more sophisticated image queries, the “Ask Copilot” option allows users to seamlessly send images to Microsoft’s cloud-based AI chatbot.
Customization and Expansion Plans
Although Click To Do is active by default on supported PCs, it can be disabled at any time through Settings > Privacy & security > Click to Do, offering users complete control over its operation.
Microsoft is actively expanding the capabilities of Click To Do. Future enhancements include an integrated Copilot prompt box, support for mixed text and image selection, on-device image description with a large language model, and table recognition features for Excel, illustrating Microsoft's commitment to making Click To Do a comprehensive AI gateway within Windows.
Despite its advanced capabilities and potential to streamline productivity, some users, including the author, still prefer traditional tools like copy-pasting and the Snipping Tool for certain tasks. However, Microsoft’s strategic shift from Recall to Click To Do reflects its forward-thinking approach in AI innovation.
