New research results in AI that associates colors, shapes and sounds with flavors

Microsoft and the University of Oslo researchers have discovered that artificial intelligence systems can form sensory associations similar to humans, linking different colors, shapes, and sounds with specific flavors and tastes.

The human connection: The human brain naturally creates connections between different sensory experiences, known as cross-modal correspondences, which influence how we perceive tastes, sounds, and colors.

• Research shows that colors like red and pink are commonly associated with sweetness, while yellow or green are linked to sourness
• The color of a wine glass or background music can significantly impact taste perception
• In extreme cases, some individuals experience synaesthesia, where sensory experiences become intensely interconnected

Scientific foundation: Cross-modal research has established clear patterns in how humans associate sensory experiences across different cultures.

• Studies from the 1970s first documented these sensory connections
• A multinational study led by Xiaoang Wang found similar patterns across Chinese, Indian, and Malaysian participants
• Shape also influences taste perception, with round shapes generally associated with sweetness and angular shapes with sourness or bitterness

AI implications: Recent research led by Carlos Velasco tested ChatGPT‘s ability to recognize and replicate human sensory associations.

• The AI system demonstrated similar cross-modal associations to humans when asked about relationships between shapes, colors, and tastes
• ChatGPT-4 showed more reliable human-like associations compared to earlier versions
• Other AI models, like Google’s Gemini, also demonstrated these capabilities, though some responses appeared to be drawn directly from scientific literature

Practical applications: The discovery opens new possibilities for product design and marketing.

• AI could help identify previously undocumented sensory connections
• Marketers could leverage these insights for more effective product packaging and design
• The technology could assist in creating more engaging multisensory experiences

Limitations and considerations: The research team acknowledges several important caveats in applying AI to sensory associations.

• AI systems can sometimes generate unreliable or “hallucinated” responses
• Machine learning models may lack the nuanced understanding that humans naturally possess
• Human creativity and verification remain essential in applying these insights

Looking forward: The intersection of AI and sensory perception research presents both opportunities and challenges for understanding how humans process multisensory experiences, though careful validation of AI-generated insights remains crucial.