Veritone’s Data Refinery aims to tackle AI’s data drought

Veritone has launched Data Refinery, a new tool designed to transform unstructured data into AI-ready assets, addressing the growing challenge of data scarcity in artificial intelligence development.

Market context and critical need: The artificial intelligence industry faces a looming shortage of high-quality training data, with experts predicting a crisis as early as 2026.

• Industry analysts, including CB Insights, warn about the diminishing availability of accessible text data for AI training
• Organizations are struggling to process vast quantities of unstructured data into actionable insights
• Poor-quality or synthetic data can result in AI “hallucinations” – instances where systems generate inaccurate or nonsensical information

Core functionality and capabilities: Veritone Data Refinery converts unstructured data into structured formats suitable for advanced AI applications, including Large Language Models (LLMs) and Large Multimodal Models (LMMs).

• The platform processes various data types including text, audio, and images
• Built on Veritone’s aiWARE™ platform, the system unifies disparate data silos into a cohesive repository
• Organizations can use refined data internally or monetize it through licensing to third parties

Company expertise and track record: Veritone’s experience in data management demonstrates significant capability in handling large-scale information processing.

• In 2023, the company managed 11.38 petabytes of data
• Processed 64.5 million cognitive media hours for major clients including the NCAA
• Maintains strong security protocols with GDPR compliance and SOC2 certification

Expert perspectives: Industry leaders emphasize the importance of data processing capabilities in the current AI landscape.

• Dr. Christos Makridis, associate professor at Arizona State University and CEO of Dainamic, highlights the critical need for platforms that help organizations understand their unstructured data
• Ryan Steelberg, CEO of Veritone, emphasizes the company’s role in bridging the emerging gap in the AI ecosystem through secure and ethical data transformation

Strategic implications: The introduction of Data Refinery represents a significant development in addressing AI’s data challenges while raising important considerations about the future of AI development.

• The platform’s dual functionality for internal innovation and data monetization reflects evolving market needs
• Security and privacy features indicate growing awareness of data protection requirements
• The solution’s focus on high-quality data processing suggests a shift toward more reliable AI model training methods