Microsoft is infusing Windows search with AI for its next upgrade

AI-powered search revolutionizes Windows user experience: Microsoft is set to introduce significant improvements to Windows search functionality on its new Copilot Plus PCs, leveraging AI models to enhance user interaction with various types of content.

Key features and improvements:

• The new search capabilities will allow users to find images using natural language descriptions, even if the search terms don’t match file names or metadata.
• Windows will better understand user intent, making it easier to locate documents, images, files, and emails without the need to remember specific file names or locations.
• The improved search will be available in File Explorer, the main Windows search interface, and the Settings search box.
• Local search functionality for OneDrive content will be enabled on Copilot Plus PCs, utilizing NPU chips to work offline.

Click to Do: A new interactive feature:

• Microsoft is introducing Click to Do, a feature similar to Google’s Circle to Search.
• Users can activate Click to Do by pressing the Windows key and left-clicking the mouse.
• An interactive overlay appears, allowing users to select images or text for various actions.
• The feature works across windows, documents, images, and videos, offering context-aware functionality.

Practical applications of Click to Do:

• Visual search on items in YouTube videos using Bing.
• Text-related actions such as rewriting, summarizing documents, explaining text, and sending emails.
• Shopping and learning capabilities based on screen content.

Rollout timeline and availability:

• Testing with Windows Insiders on Copilot Plus PCs begins in October.
• Gradual rollout of new search and Click to Do features starts in November.
• The previously announced Recall feature will be tested on Qualcomm-powered devices in October and on Intel- or AMD-powered Copilot Plus PCs in November.

Additional AI-powered enhancements:

• Microsoft Paint will receive generative fill and erase features as part of the new Windows AI capabilities.

Potential impact on user productivity:

• These AI-powered improvements aim to streamline user workflows and enhance productivity by simplifying content discovery and interaction.
• The natural language search capabilities could significantly reduce the time spent locating files and information within the Windows ecosystem.

Industry implications: Microsoft’s AI-driven enhancements to Windows search and interaction methods signal a broader trend in the tech industry towards more intuitive and powerful user interfaces.

• This move may pressure other operating system developers to incorporate similar AI-powered features to remain competitive.
• The integration of AI into core operating system functions could lead to new paradigms in human-computer interaction, potentially influencing future software and hardware design.

Looking ahead: As these features roll out and evolve, it will be interesting to observe how users adapt to and utilize these AI-powered tools in their daily computing tasks, and how Microsoft continues to refine and expand upon these capabilities in future Windows updates.