MIT startup invents new breed of AI model and it’s already state of the art

Liquid AI unveils groundbreaking non-transformer AI models: Liquid AI, a startup with roots in MIT’s CSAIL, has introduced a new class of AI models that challenge the dominance of transformer-based architectures in the field of artificial intelligence.

Revolutionary approach to AI model design: Liquid AI’s new Liquid Foundation Models (LFMs) are built from first principles, eschewing the transformer architecture that has been the cornerstone of most recent AI advancements.

• The company’s goal is to explore alternative methods for building foundation models beyond Generative Pre-trained Transformers (GPTs).
• LFMs are based on computational units grounded in dynamical systems theory, signal processing, and numerical linear algebra.
• This novel approach allows for the creation of general-purpose AI models capable of handling various types of sequential data, including video, audio, text, time series, and signals.

Impressive performance and efficiency: Early benchmarks suggest that LFMs are already outperforming comparable transformer-based models while using significantly less memory.

• The smallest model, LFM 1.3B, has surpassed Meta’s Llama 3.2-1.2B and Microsoft’s Phi-1.5 on several third-party benchmarks, including the Massive Multitask Language Understanding (MMLU) test.
• LFM-3B requires only 16 GB of memory, compared to the 48 GB needed by Meta’s Llama-3.2-3B model, showcasing superior memory efficiency.
• The largest model, LFM 40B MoE, employs a Mixture-of-Experts approach similar to Mistral’s Mixtral.

Multimodal capabilities and industry applications: Liquid AI has designed its foundation models to be versatile across multiple data modalities, positioning them for use in various industries.

• The models can handle audio, video, and text data, making them suitable for a wide range of applications.
• Target industries include financial services, biotechnology, and consumer electronics.
• The models’ efficiency makes them ideal for deployment on edge devices and in enterprise-level applications.

Technical innovations and adaptability: LFMs build upon Liquid Neural Networks (LNNs), an architecture developed at CSAIL that aims to make artificial neurons more efficient and adaptable.

• LNNs demonstrate that fewer neurons, combined with innovative mathematical formulations, can achieve results comparable to traditional deep learning models with thousands of neurons.
• This approach allows for real-time adjustments during inference without the computational overhead associated with traditional models.
• LFMs can handle up to 1 million tokens efficiently while maintaining minimal memory usage.

Accessibility and future developments: While not open-source, Liquid AI is making its models available through various platforms and is actively seeking feedback from early adopters.

• Users can access the models through Liquid’s inference playground, Lambda Chat, or Perplexity AI.
• The company is optimizing its models for deployment on hardware from NVIDIA, AMD, Apple, Qualcomm, and Cerebras.
• Liquid AI plans to release technical blog posts and engage in red-teaming efforts to improve future iterations of the models.

Anticipation builds for official launch: Liquid AI is preparing for a full launch event on October 23, 2024, at MIT’s Kresge Auditorium in Cambridge, MA.

• The company is accepting RSVPs for in-person attendance at the event.
• This launch will mark Liquid AI’s official entry as a key player in the foundation model space.

Potential industry impact: The introduction of Liquid Foundation Models represents a significant development in AI technology, offering a compelling alternative to traditional transformer-based models.

• By combining state-of-the-art performance with unprecedented memory efficiency, LFMs could potentially disrupt the current AI landscape.
• The success of these models may encourage further research into non-transformer architectures, potentially leading to new breakthroughs in AI capabilities and efficiency.