×
Written by
Published on
Written by
Published on
Join our daily newsletter for breaking news, product launches and deals, research breakdowns, and other industry-leading AI coverage
Join Now

Liquid AI unveils groundbreaking non-transformer AI models: Liquid AI, a startup with roots in MIT’s CSAIL, has introduced a new class of AI models that challenge the dominance of transformer-based architectures in the field of artificial intelligence.

Revolutionary approach to AI model design: Liquid AI’s new Liquid Foundation Models (LFMs) are built from first principles, eschewing the transformer architecture that has been the cornerstone of most recent AI advancements.

  • The company’s goal is to explore alternative methods for building foundation models beyond Generative Pre-trained Transformers (GPTs).
  • LFMs are based on computational units grounded in dynamical systems theory, signal processing, and numerical linear algebra.
  • This novel approach allows for the creation of general-purpose AI models capable of handling various types of sequential data, including video, audio, text, time series, and signals.

Impressive performance and efficiency: Early benchmarks suggest that LFMs are already outperforming comparable transformer-based models while using significantly less memory.

  • The smallest model, LFM 1.3B, has surpassed Meta’s Llama 3.2-1.2B and Microsoft’s Phi-1.5 on several third-party benchmarks, including the Massive Multitask Language Understanding (MMLU) test.
  • LFM-3B requires only 16 GB of memory, compared to the 48 GB needed by Meta’s Llama-3.2-3B model, showcasing superior memory efficiency.
  • The largest model, LFM 40B MoE, employs a Mixture-of-Experts approach similar to Mistral’s Mixtral.

Multimodal capabilities and industry applications: Liquid AI has designed its foundation models to be versatile across multiple data modalities, positioning them for use in various industries.

  • The models can handle audio, video, and text data, making them suitable for a wide range of applications.
  • Target industries include financial services, biotechnology, and consumer electronics.
  • The models’ efficiency makes them ideal for deployment on edge devices and in enterprise-level applications.

Technical innovations and adaptability: LFMs build upon Liquid Neural Networks (LNNs), an architecture developed at CSAIL that aims to make artificial neurons more efficient and adaptable.

  • LNNs demonstrate that fewer neurons, combined with innovative mathematical formulations, can achieve results comparable to traditional deep learning models with thousands of neurons.
  • This approach allows for real-time adjustments during inference without the computational overhead associated with traditional models.
  • LFMs can handle up to 1 million tokens efficiently while maintaining minimal memory usage.

Accessibility and future developments: While not open-source, Liquid AI is making its models available through various platforms and is actively seeking feedback from early adopters.

  • Users can access the models through Liquid’s inference playground, Lambda Chat, or Perplexity AI.
  • The company is optimizing its models for deployment on hardware from NVIDIA, AMD, Apple, Qualcomm, and Cerebras.
  • Liquid AI plans to release technical blog posts and engage in red-teaming efforts to improve future iterations of the models.

Anticipation builds for official launch: Liquid AI is preparing for a full launch event on October 23, 2024, at MIT’s Kresge Auditorium in Cambridge, MA.

  • The company is accepting RSVPs for in-person attendance at the event.
  • This launch will mark Liquid AI’s official entry as a key player in the foundation model space.

Potential industry impact: The introduction of Liquid Foundation Models represents a significant development in AI technology, offering a compelling alternative to traditional transformer-based models.

  • By combining state-of-the-art performance with unprecedented memory efficiency, LFMs could potentially disrupt the current AI landscape.
  • The success of these models may encourage further research into non-transformer architectures, potentially leading to new breakthroughs in AI capabilities and efficiency.
MIT spinoff Liquid debuts non-transformer AI models and they’re already state-of-the-art

Recent News

CERN is training an AI model to revolutionize cancer treatment

CERN applies its AI expertise to healthcare, improving cancer treatment, stroke management, and brain abnormality detection.

AI climate gamble: Eric Schmidt’s bold $125M venture

The debate highlights the complex balance between AI's potential to solve climate issues and its significant energy consumption, calling for a more nuanced approach to development and deployment.

Google stuffs ads into AI search, frustrating users

Google's AI-generated search summaries now include ads, raising concerns about user experience and content quality while showcasing AI's revenue potential in search.