This Chinese vacuum records video and audio to train its AI

Smart home privacy concerns: The popular Ecovacs Deebot robot vacuums are collecting sensitive user data, including photos, videos, and audio recordings from inside homes, to train the company’s AI models.

• Ecovacs, a Chinese home robotics company, offers a “product improvement program” through its smartphone app, which users can opt into without clear information about the data being collected.
• The company’s privacy policy allows for broad collection of user data, including 2D or 3D maps of homes, voice recordings, and photos or videos captured by the device.
• Even when users delete recordings, photos, or videos through the app, Ecovacs may continue to retain and use this data.

Cybersecurity vulnerabilities: Critical flaws in some Ecovacs models have raised concerns about the company’s ability to protect user information.

• Cybersecurity researcher Dennis Giese discovered basic errors that put Ecovacs customers’ privacy at risk, allowing some models to be hacked remotely.
• Giese questioned the security of Ecovacs’ back-end servers, highlighting potential vulnerabilities to corporate espionage or nation-state actors.
• Ecovacs, valued at $4.6 billion, has committed to fixing security issues in its flagship robot vacuum by November and is exploring more comprehensive testing methods.

AI training and data usage: Ecovacs confirms that data collected from users participating in the product improvement program is used to train its AI models.

• The company claims to anonymize user information at the machine level before uploading it to servers.
• Ecovacs states it has implemented strict access management protocols for viewing and utilizing anonymized user data.
• Two Ecovacs Robotics AI department engineers previously described the need for large amounts of data to build deep learning models, citing cooperation with institutions worldwide to collect data.

Industry precedents: Previous incidents involving robot vacuum data leaks highlight the potential risks associated with collecting sensitive information.

• In 2022, intimate photos taken by iRobot devices, including one of a person on a toilet, were shared on Facebook.
• The leaked images came from special development robots with modifications not present in consumer products, and users had consented to data collection for research purposes.
• iRobot had contracted Scale AI, an AI training data company, to analyze raw footage for object detection algorithm training.

Data labeling industry: The incident sheds light on the broader ecosystem of AI training data preparation and its potential pitfalls.

• Scale AI, valued at $20 billion, claims to generate nearly all the data needed to fuel leading large language models.
• Contract workers for companies like Scale AI perform tasks such as differentiating objects in videos, labeling images, and editing text for AI model training.
• iRobot terminated its relationship with Scale AI after contractors leaked photos on social media.

Alternative technologies: Researchers are developing privacy-preserving solutions to address concerns about data collection by smart home devices.

• The Australian Centre for Robotics has created technology that scrambles images before digitization, preventing remote attacks from accessing raw imagery while still allowing robots to navigate effectively.
• This “privacy-preserving” camera approach could potentially be commercialized and adopted by tech companies in the future.

Broader implications: The Ecovacs case highlights the ongoing tension between technological advancement and user privacy in the smart home industry.

• As AI-powered devices become more prevalent in homes, the collection and use of sensitive data for training purposes raise significant privacy and security concerns.
• The incident underscores the need for greater transparency from companies about data collection practices and improved security measures to protect user information.
• Balancing the benefits of AI-driven product improvements with user privacy rights remains a critical challenge for the smart home industry and regulators.