Why Analysts Predict an End to AI’s Over-Reliance on GPUs

AI’s brute force era nears its end: Gartner analysts predict a shift away from specialized AI hardware, including GPUs, as more efficient programming techniques emerge.

The big picture: Gartner’s chief of research for AI, Erick Brethenoux, argues that the current reliance on powerful hardware for AI workloads is temporary, with generative AI applications likely to follow historical patterns of optimization.

• Brethenoux draws on 45 years of AI observation, noting that specialized AI hardware has consistently been rendered obsolete as standard machines become capable of handling AI tasks.
• The current “brute force” phase of AI is characterized by unrefined programming techniques requiring powerful hardware, which Brethenoux suggests is unsustainable in the long term.

Generative AI’s limited scope: Despite dominating discussions, generative AI is applicable to only a small fraction of use cases, according to Gartner’s analysis.

• Brethenoux estimates that generative AI accounts for 90% of AI-related discourse but only 5% of actual use cases.
• Many organizations have reverted to established AI techniques or are exploring “composite AI” solutions that combine generative AI with traditional methods like machine learning and knowledge graphs.

Reframing AI’s business value: Companies are reassessing the role of AI in their operations, recognizing the importance of existing AI applications that may have been overlooked.

• The period from late 2022 to early 2024 saw many IT departments shift focus from profit-generating activities to exploring generative AI.
• Organizations are now realizing that AI may already be contributing significantly to their business through less flashy but crucial applications, such as predictive maintenance.

Composite AI as a preferred approach: Gartner analysts recommend combining generative AI with established AI techniques for more reliable and efficient outcomes.

• Examples of composite AI include using generative AI to create text descriptions for outputs from predictive maintenance applications or to generate prose recommendations based on firewall log analysis.
• Bern Elliot, a Gartner vice president and distinguished analyst, emphasizes the importance of using generative AI alongside other AI methods to compensate for its limitations.

Cautionary notes on generative AI: Gartner experts highlight the technology’s limitations and potential risks.

• Elliot points out that generative AI lacks reasoning capabilities and produces only probabilistic content sequences.
• Despite improvements reducing “hallucinations” to 1-2% of outputs, the sheer volume of interactions in production environments can still result in millions of erroneous responses.
• Experts recommend implementing guardrails using non-generative AI techniques to validate generative AI outputs.

Analyzing deeper: As the AI landscape evolves, organizations must carefully evaluate the appropriate applications for generative AI while leveraging existing AI investments. The shift away from specialized hardware suggests a future where AI becomes more integrated into standard computing infrastructure, potentially democratizing access to AI capabilities. However, the challenges of reliability and accuracy in generative AI outputs underscore the need for continued innovation in AI development and implementation strategies.