Apple Announces Visual Intelligence, a Direct Challenge to Google Lens

Apple introduces Visual Intelligence: Apple has unveiled a new AI-powered feature called Visual Intelligence, set to be part of the iOS 18 Apple Intelligence suite later this year.

• Visual Intelligence is comparable to similar features offered by other multimodal AI systems, such as Google Lens or OpenAI’s image recognition capabilities.
• The feature aims to provide instant information about objects or scenes captured by the iPhone’s camera.
• Visual Intelligence is enabled by a new hardware feature called Camera Control, which is a capacitive camera button on the side of iPhone 16 and 16 Pro models.

How Visual Intelligence works: Users can activate the feature by pressing and holding the Camera Control button and pointing their iPhone’s camera at an object of interest.

• The system uses a combination of on-device intelligence and Apple services to analyze the image without storing users’ photos.
• Visual Intelligence can extract information from various sources, such as restaurant hours from a storefront image or event details from a flyer.
• The feature also supports integration with third-party models, allowing users to search Google for similar items or get help with study concepts from photographed notes.

Privacy and processing: Apple emphasizes that Visual Intelligence prioritizes user privacy by leveraging on-device processing and secure Apple services.

• The company states that images processed through Visual Intelligence are not stored, addressing potential privacy concerns.
• This approach aligns with Apple’s longstanding commitment to user data protection and privacy-focused technology development.

Release timeline and availability: While Apple has announced Visual Intelligence, the exact release date remains unspecified.

• The feature is slated to be part of the iOS 18 update, which is expected to be released “later this year.”
• Visual Intelligence will be available on iPhone 16 and 16 Pro models equipped with the new Camera Control button.

Broader implications: Apple’s entry into the visual recognition AI space signifies the growing importance of multimodal AI in consumer technology.

• This move puts Apple in direct competition with Google’s well-established Lens feature, potentially reshaping the landscape of visual search and recognition tools in mobile devices.
• The integration of Visual Intelligence with third-party models suggests a more open approach from Apple, which could lead to a rich ecosystem of AI-powered visual applications on iOS devices.