×
Why enterprises are increasingly using small language models
Written by
Published on
Join our daily newsletter for breaking news, product launches and deals, research breakdowns, and other industry-leading AI coverage
Join Now

The growing prominence of smaller AI models in enterprise applications is reshaping how businesses approach artificial intelligence implementation, with a focus on efficiency and cost-effectiveness.

Key findings from industry research: Databricks’ State of Data + AI report reveals that 77% of enterprise AI implementations utilize smaller models with less than 13 billion parameters, while large models exceeding 100 billion parameters account for only 15% of deployments.

  • Enterprise buyers are increasingly scrutinizing the return on investment of larger AI models, particularly in production environments
  • The cost differential between small and large models is significant, with pricing increasing geometrically as parameter counts rise
  • This trend reflects a broader shift toward practical, cost-effective AI solutions in business settings

Performance advantages of smaller models: Recent advancements have significantly improved the capabilities of smaller AI models, making them increasingly attractive alternatives to their larger counterparts.

  • Smaller models now approach the performance levels of larger models in many applications
  • The reduced cost allows organizations to run multiple iterations for verification purposes, similar to using multiple human reviewers
  • This redundancy capability enhances accuracy and reliability while maintaining cost advantages

Latency considerations: Response time measurements reveal substantial performance advantages for smaller AI models.

  • 7 billion parameter models demonstrate an 18ms latency per token
  • 13 billion parameter models show 21ms latency per token
  • 70 billion parameter models require 47ms per token
  • Larger 405 billion parameter models range from 70-750ms per token
  • These differences significantly impact user experience, as faster response times lead to better engagement

Business implications: The combination of cost savings and performance benefits makes smaller AI models particularly attractive for enterprise deployment.

  • Organizations can achieve comparable results at a fraction of the cost of larger models
  • Reduced latency translates to improved user experiences and higher productivity
  • The efficiency gains allow for broader implementation across various business functions

Looking ahead: The trend toward smaller, more efficient AI models suggests a maturing market where practical considerations are beginning to outweigh the pursuit of ever-larger models, potentially signaling a new phase in enterprise AI adoption focused on optimization rather than scale.

Small but Mighty AI by @ttunguz

Recent News

Grok stands alone as X restricts AI training on posts in new policy update

X explicitly bans third-party AI companies from using tweets for model training while still preserving access for its own Grok AI.

Coming out of the dark: Shadow AI usage surges in enterprise IT

IT leaders report 90% concern over unauthorized AI tools, with most organizations already suffering negative consequences including data leaks and financial losses.

Anthropic CEO opposes 10-year AI regulation ban in NYT op-ed

As AI capabilities rapidly accelerate, Anthropic's chief executive argues for targeted federal transparency standards rather than blocking state-level regulation for a decade.