AI PCs fall short of performance expectations: Recent benchmarks reveal that AI-powered PCs are struggling to deliver on their promised computational capabilities, particularly in the realm of neural processing units (NPUs).
Qualcomm’s NPU technology under scrutiny: Pete Warden, a long-time advocate of Qualcomm’s NPU technology, has expressed disappointment with the performance of these chips in Windows tablets, specifically the Microsoft Surface Pro running on Arm.
- Warden’s history with Qualcomm includes collaborating on experimental support for their HVX DSP in TensorFlow back in 2017.
- The promise of up to 45 trillion operations per second on Windows tablets equipped with Qualcomm’s NPUs initially generated significant excitement.
Benchmarking reveals significant performance gaps: Warden’s open-source benchmark, focusing on matrix multiplication as a fundamental AI operation, exposed a substantial discrepancy between advertised and actual performance.
- The NPU achieved only 573 billion operations per second, less than 1.3% of the advertised 45 trillion operations per second.
- This performance was even lower than that of the CPU and significantly behind the 2.16 teraops achieved by an Nvidia RTX 4080 in a gaming laptop using the same benchmark.
Potential causes for underperformance: While the exact reason for the poor performance remains unclear, several factors could be contributing to the issue.
- Software stack limitations, including the Onnx runtime, drivers, and on-chip code, may not be fully optimized yet.
- The inability to compile and run custom operations on the DSP in Windows further limits potential workarounds.
- It’s possible that the method of calling the code could be a factor, though Warden claims to have followed the documentation closely.
Industry implications: The underperformance of AI PCs could have broader implications for the tech industry and consumer expectations.
- This situation highlights the challenges in translating theoretical hardware capabilities into real-world performance gains.
- It raises questions about the readiness of AI acceleration hardware for mainstream consumer devices.
- The discrepancy between advertised and actual performance could potentially impact consumer trust in AI PC marketing claims.
Looking ahead: Despite the current disappointment, there’s still hope for improvement in AI PC performance.
- Many of the potential issues identified could be addressed through software updates, suggesting that performance could improve over time.
- The open-sourcing of the benchmark invites collaboration and further investigation from the tech community.
- Warden remains optimistic about the hardware’s potential, pending resolution of the current performance bottlenecks.
Broader context: This situation underscores the complexities involved in bringing cutting-edge AI technologies to consumer-grade devices.
- It highlights the gap between theoretical capabilities and practical implementation in real-world scenarios.
- The challenges faced by Qualcomm and Microsoft in this instance may offer valuable lessons for other companies working on AI acceleration in consumer electronics.
- This experience emphasizes the importance of robust testing and optimization before bringing AI-powered devices to market.
Recent Stories
DOE fusion roadmap targets 2030s commercial deployment as AI drives $9B investment
The Department of Energy has released a new roadmap targeting commercial-scale fusion power deployment by the mid-2030s, though the plan lacks specific funding commitments and relies on scientific breakthroughs that have eluded researchers for decades. The strategy emphasizes public-private partnerships and positions AI as both a research tool and motivation for developing fusion energy to meet data centers' growing electricity demands. The big picture: The DOE's roadmap aims to "deliver the public infrastructure that supports the fusion private sector scale up in the 2030s," but acknowledges it cannot commit to specific funding levels and remains subject to Congressional appropriations. Why...
Oct 17, 2025Tying it all together: Credo’s purple cables power the $4B AI data center boom
Credo, a Silicon Valley semiconductor company specializing in data center cables and chips, has seen its stock price more than double this year to $143.61, following a 245% surge in 2024. The company's signature purple cables, which cost between $300-$500 each, have become essential infrastructure for AI data centers, positioning Credo to capitalize on the trillion-dollar AI infrastructure expansion as hyperscalers like Amazon, Microsoft, and Elon Musk's xAI rapidly build out massive computing facilities. What you should know: Credo's active electrical cables (AECs) are becoming indispensable for connecting the massive GPU clusters required for AI training and inference. The company...
Oct 17, 2025Vatican launches Latin American AI network for human development
The Vatican hosted a two-day conference bringing together 50 global experts to explore how artificial intelligence can advance peace, social justice, and human development. The event launched the Latin American AI Network for Integral Human Development and established principles for ethical AI governance that prioritize human dignity over technological advancement. What you should know: The Pontifical Academy of Social Sciences, the Vatican's research body for social issues, organized the "Digital Rerum Novarum" conference on October 16-17, combining academic research with practical AI applications. Participants included leading experts from MIT, Microsoft, Columbia University, the UN, and major European institutions. The conference...