MIT students push boundaries of human-AI teamwork

Students from MIT’s Interaction Intelligence course showcased innovative human-AI collaboration projects at the 2023 NeurIPS conference, demonstrating new ways for artificial intelligence to enhance creative expression and learning.

Event Context; The 38th Neural Information Processing Systems (NeurIPS) conference in Vancouver attracted over 16,000 attendees to witness cutting-edge developments in artificial intelligence and machine learning.

• The projects emerged from MIT course 4.043/4.044 (Interaction Intelligence), taught by Marcelo Coelho in the Department of Architecture
• The course explores large language objects and how AI can extend into physical applications
• The conference served as a prestigious venue for showcasing these innovative student projects

Project Highlights: Five distinct projects demonstrated novel approaches to integrating AI with human creativity and learning.

• “Be the Beat” created an AI-powered boombox that analyzes dance movements using PoseNet technology to suggest matching music, inverting the traditional relationship between dance and music
• “A Mystery for You” developed an educational game using large language models to help young learners develop critical thinking and fact-checking skills through tactile interaction
• “Memorscope” designed a device that creates shared memories between two users by combining face-to-face interaction with AI image generation from OpenAI and Midjourney
• “Narratron” built an interactive shadow puppetry system that uses AI to co-create children’s stories based on hand shadow inputs
• “Perfect Syntax” explored AI’s ability to manipulate video fragments, examining the relationship between technology and motion perception

Technical Implementation: Each project leveraged different AI technologies to create unique interactive experiences.

• The projects incorporated various AI models including PoseNet, large language models, OpenAI, and Midjourney
• Physical interfaces ranged from boomboxes to projectors to tactile game components
• The implementations focused on creating intuitive human-computer interactions that feel natural and engaging

Educational Impact: These projects demonstrate how AI can enhance learning and creativity in unexpected ways.

• Students developed critical thinking skills through investigative gameplay
• Creative expression was augmented through AI-assisted dance and storytelling
• Memory creation and preservation were reimagined through collective AI-enhanced experiences

Future Implications: These student projects suggest a promising direction for AI that emphasizes augmenting human capabilities rather than replacing them, while raising important questions about how technology will continue to reshape creative expression and learning in the years ahead.