Meta is releasing new research to advance embodied AI and robotics

Advancing embodied AI: Meta FAIR is making significant strides in robotics research, aiming to develop AI systems that can perceive and interact with their surroundings as effectively as humans.

• Meta is releasing several new research artifacts that focus on touch perception, robot dexterity, and human-robot interaction, which are crucial for achieving advanced machine intelligence (AMI).
• The company is partnering with GelSight Inc and Wonik Robotics to develop and commercialize tactile sensing innovations, fostering an open ecosystem for AI research.

Breakthrough in touch perception: Meta Sparsh, a new general-purpose touch representation, is being released to enable AI to perceive what’s inaccessible through vision alone.

• Sparsh is the first encoder that works across various vision-based tactile sensors and tasks, leveraging self-supervised learning to avoid the need for labels.
• The technology outperforms task- and sensor-specific models by an average of over 95% on a novel benchmark consisting of six touch-centric tasks.

Advanced tactile sensing: Meta Digit 360, a revolutionary artificial fingertip, delivers human-level tactile sensing capabilities.

• Equipped with over 18 sensing features, Digit 360 can detect miniature spatial changes and capture forces as small as 1 millinewton.
• The sensor incorporates a wide-field-of-view optical system with over 8 million taxels and multiple sensing modalities to perceive vibrations, heat, and even odors.
• GelSight Inc will manufacture and distribute Digit 360, making it available for purchase next year.

Integrating tactile sensing: Meta Digit Plexus provides a standardized platform for integrating various tactile sensors on a single robot hand.

• The platform combines vision-based and skin-based tactile sensors across fingertips, fingers, and palm, enabling seamless data collection, control, and analysis.
• Wonik Robotics will use Meta Digit Plexus to create a new advanced dexterous robot hand, the next generation of the Allegro Hand.

Enhancing human-robot collaboration: The PARTNR benchmark introduces a framework for evaluating planning and reasoning in human-robot collaboration scenarios.

• PARTNR comprises 100,000 natural language tasks across 60 houses and over 5,800 unique objects, built on top of the Habitat 3.0 simulator.
• The benchmark aims to drive advancements in human-robot interaction and collaborative intelligence, transforming AI models from “agents” to “partners.”

Industry partnerships and community engagement: Meta is collaborating with industry leaders to commercialize and distribute its innovations.

• GelSight Inc will manufacture and distribute Digit 360, with early access available through a call for proposals.
• Wonik Robotics will develop and distribute the next generation of the Allegro Hand, integrating Meta’s tactile sensing technologies.

Broader implications: Meta’s advancements in robotics and tactile sensing have the potential to impact various fields significantly.

• These innovations could revolutionize healthcare, manufacturing, supply chains, and energy sectors by enabling more sophisticated and capable robotic systems.
• The open-source approach and industry partnerships aim to foster community-driven research and accelerate progress in robotics and AI.
• As AI and robotics continue to evolve, these technologies may pave the way for more efficient, reliable, and adaptable systems that can better serve human needs and preferences.