Enterprises Invest Heavily in Generative AI Despite Integration Challenges and Infrastructure Barriers

A recent survey conducted by Dataiku and Cognizant reveals that enterprises are making significant investments in generative AI, but challenges persist in fully realizing its potential and integrating it into their operations.

Significant financial commitments: Nearly three-quarters of the surveyed organizations plan to spend over $500,000 on generative AI in the next year, with almost half allocating more than $1 million, highlighting the growing interest and investment in this technology.

• Despite the substantial spending, only one-third of the organizations have a dedicated budget for generative AI initiatives, with the majority funding these projects from other sources like IT, data science, or analytics budgets.
• The return on investment for these expenditures remains unclear, but there is optimism that the added value will justify the costs as advances in large language models (LLMs) and other generative models continue.

Integration challenges and infrastructure barriers: Enterprises face various hurdles in implementing generative AI, including infrastructure limitations, regulatory compliance, and internal policy challenges.

• Most respondents reported having infrastructure barriers that prevent them from using LLMs as desired, while operational costs of generative models also remain a significant obstacle.
• Hosted LLM services like Microsoft Azure ML, Amazon Bedrock, and OpenAI API are popular choices for exploring and producing generative AI, but their token-based pricing model makes it difficult to manage costs at scale.
• Self-hosted open-source LLMs can meet enterprise needs and reduce inference costs, but they require upfront investment and in-house technical expertise that many organizations lack.

Data quality and usability concerns: Data challenges continue to hinder the adoption of generative AI, with data quality and usability being the biggest data infrastructure issues faced by IT leaders.

• Most organizations have rich data resources, but their data infrastructure was not designed with machine learning in mind, leading to data silos and incompatible formats that require preprocessing and consolidation before use.
• Data engineering and data ownership management remain important challenges for most machine learning and AI projects, despite the advent of generative AI.

Opportunities for service providers and enterprises: As generative AI transitions from exploratory projects to the foundation of scalable operations, companies providing generative AI services can support enterprises and developers with better tools and platforms to simplify integration and reduce complexity.

• Enterprises can prepare for the wave of generative AI technologies by running small pilot projects, experimenting with new technologies, and identifying pain points in their data infrastructure and policies.
• Building in-house skills can give organizations more options and better position them to harness the full potential of generative AI and drive innovation in their respective industries.

Analyzing deeper: While the survey highlights the growing interest and investment in generative AI among enterprises, it also underscores the significant challenges that organizations face in fully capitalizing on this technology. The lack of dedicated budgets, infrastructure barriers, and data quality issues suggest that many enterprises are still in the early stages of adopting generative AI and have yet to develop comprehensive strategies for its implementation. As the technology continues to evolve rapidly, organizations will need to remain agile and adaptable to stay competitive. Service providers that can offer simplified solutions and support enterprises in overcoming these challenges will likely play a crucial role in driving the widespread adoption and success of generative AI in the enterprise landscape.