A face only AI could love: Can a synthetic visage solve facial recognition’s privacy problem?

Researchers are exploring the use of synthetic faces—computer-generated images that don’t belong to real people—to train facial recognition AI systems, potentially solving major privacy concerns while maintaining fairness across demographic groups. This approach could eliminate the need for scraping millions of real photos from the internet without consent, addressing both ethical data collection issues and the risk of identity theft or surveillance overreach.

The big picture: Facial recognition technology has achieved near-perfect accuracy rates of 99.9 percent across different skin tones, ages, and genders, but this success came at the cost of individual privacy through massive data collection from real faces.

Why this matters: Current training methods involve collecting millions of real photos without consent, creating significant privacy risks and potential for identity theft, while synthetic data could provide the same training benefits without compromising personal information.

How synthetic face training works: The process requires two main steps to create effective training datasets.
• First, researchers generate unique fake faces using the same technology behind deepfakes.
• Then, they create variations of each synthetic face under different lighting conditions, angles, and with various accessories.
• The generators still need training on thousands of real images initially, but far fewer than the millions required for direct recognition model training.

Current performance challenges: Models trained on synthetic data still lag behind those using real-world faces in terms of accuracy.
• A recent study found that while real-face-trained models achieved 85 percent average accuracy, synthetic-face-trained models reached only 75 percent.
• However, the synthetic models showed significantly less bias, with only one-third the variability between demographic groups compared to real-data models.
• The accuracy gap stems from generators’ limited ability to create unique identities and their tendency to produce “pretty, studio-like pictures” that don’t reflect real-world image variety.

What the research shows: A July 2024 study demonstrated that demographically balanced synthetic datasets can reduce racial bias more effectively than real datasets of similar size.
• When tested on African, Asian, Caucasian, and Indian faces, the real-data model showed 90 percent accuracy for Caucasian faces but only 81 percent for African faces.
• The synthetic-data model, despite lower overall accuracy, performed far more consistently across all racial groups.

The privacy problem: Major tech companies have scraped billions of images without permission, creating massive ethical and legal concerns.
• IBM’s “Diversity in Faces” dataset contained over 1 million images taken from Flickr without owner consent.
• Clearview AI, a vendor used by law enforcement, has gathered an estimated 60 billion images from Instagram and Facebook without permission.
• These practices have triggered significant backlash over privacy violations and potential misuse.

What experts are saying: Researchers emphasize the balance between accuracy and ethical considerations in facial recognition development.
• “Every person, irrespective of their skin color or their gender or their age, should have an equal chance of being correctly recognized,” says Ketan Kotwal of the Idiap Research Institute in Switzerland.
• “If you use a less accurate system, you are likely to track the wrong people,” Kotwal adds, arguing for highly accurate systems if they’re to be used at all.

Next steps: Researchers plan to explore hybrid approaches that combine synthetic and real data to improve accuracy while maintaining ethical standards.
• The hybrid method would use synthetic data to teach facial features and demographic variations, then fine-tune with consensually obtained real-world data.
• The field is advancing rapidly, with first proposals for synthetic data in facial recognition emerging only in 2023.