×
What exactly are ‘foundation models’
Written by
Published on
Join our daily newsletter for breaking news, product launches and deals, research breakdowns, and other industry-leading AI coverage
Join Now

The concept of foundation models emerged in 2021 as researchers identified a new category of AI neural networks capable of handling diverse tasks after being trained on massive unlabeled datasets. These models represent a significant shift from earlier AI systems that were narrowly focused on specific tasks, as they can be adapted for various applications ranging from language processing to image analysis.

Key characteristics and capabilities: Foundation models represent a breakthrough in AI architecture by combining massive-scale training with adaptability across multiple domains.

  • These AI systems learn from unlabeled datasets, eliminating the need for time-consuming manual data labeling
  • Through fine-tuning, they can perform diverse tasks from text translation to medical image analysis
  • The number of published foundation models more than doubled from 2022 to 2023, with 149 models released in 2023 alone

Technical evolution: The development of foundation models traces back to the breakthrough transformer architecture introduced in 2017, which sparked rapid advancement in the field.

  • Google’s BERT model became open-source software, initiating a race to build increasingly powerful language models
  • GPT-3, released in 2020, demonstrated unprecedented capabilities with 175 billion parameters
  • Google’s 2024 Gemini Ultra represents the current state-of-the-art, requiring 50 billion petaflops of computing power

Multimodal capabilities: Foundation models have expanded beyond text to handle multiple types of data simultaneously.

  • Vision Language Models (VLMs) can process and generate text, images, audio, and video
  • The Cosmos Nemotron 34B model, trained on 355,000 videos and 2.8 million images, exemplifies advanced multimodal capabilities
  • Diffusion models have enabled text-to-image generation, attracting millions of users to services like Midjourney

Real-world applications: These models are finding practical applications across various industries and use cases.

  • Businesses are customizing pre-trained foundation models rather than building from scratch
  • Digital twins powered by foundation models help optimize factory and warehouse operations
  • The NVIDIA NeMo framework enables companies to create custom chatbots and AI assistants

Challenges and concerns: The widespread adoption of foundation models has raised important considerations about their responsible development and use.

  • Models can amplify biases present in training data
  • There are risks of generating inaccurate or misleading information
  • Intellectual property rights violations remain a concern
  • Researchers are developing safeguards including prompt filtering and dataset scrubbing

Looking ahead: While foundation models have demonstrated remarkable capabilities, researchers believe we’ve only scratched the surface of their potential applications and impact on various industries.

What Are Foundation Models?

Recent News

Eurasian AI-liance: BMW to add DeepSeek to China vehicles by end of 2025

BMW aims to enhance its position in China's competitive market by integrating locally developed AI from startup DeepSeek into vehicles sold there.

AI-powered authentication framework launched by Descope

The platform addresses critical security infrastructure for enterprise AI agents that access multiple systems with user credentials, filling a gap many companies discover too late.

Deboost the boast: Apple urged to temper AI claims about Siri capabilities

Advertising watchdog calls out Apple for suggesting AI features were immediately available when they were actually released over several months.