Why CIOs Are Considering Small Language Models

The integration of generative AI in enterprise products is prompting CIOs to carefully evaluate the technology’s potential and risks, with a focus on sustainability, governance, and building internal AI expertise.

Balancing technological excitement and commercial propositions: CIOs are studying generative AI to stay current while maintaining a pragmatic approach:

• The rapid advancements in AI, such as GPT-4 passing the Turing test and the availability of AI assistants like Microsoft Copilot and Gemini Advanced, are driving CIOs to closely monitor the technology.
• However, CIOs are cautious about adopting generative AI without proper considerations, recognizing the risks of overestimating its capabilities and placing excessive trust in the technology.

Sustainability concerns and the appeal of small language models: The energy intensity of large AI models is raising sustainability concerns, making small language models (SLMs) an attractive alternative for CIOs:

• SLMs trained on specific company data are seen as more efficient, cost-effective, and accurate for business tasks compared to large language models from big tech providers.
• By using SLMs, companies can maintain greater control over their AI systems and data, addressing concerns about relying on external providers.

Emphasis on governance and internal AI expertise: CIOs are prioritizing data and AI governance, as well as building in-house AI competence, to effectively leverage and impact AI products and services:

• Careful evaluation of costs and capabilities is crucial when deciding between using large models via APIs or developing customized small models in-house.
• While big tech companies dominate the AI space, CIOs aim to steer the application of the technology to meet specific business objectives.
• Having a capable IT team with competence in AI is considered essential for CIOs to effectively manage and influence the adoption of AI in their organizations.

Navigating the AI landscape: As CIOs navigate the rapidly evolving AI landscape, they must strike a balance between embracing the technology’s potential and mitigating its risks:

• CIOs are tasked with making informed decisions about AI adoption, considering factors such as sustainability, cost-effectiveness, data control, and alignment with business goals.
• Building internal AI expertise and establishing robust governance frameworks will be critical for CIOs to successfully integrate AI into their organizations while maintaining a strategic and pragmatic approach.