Far from benched: Nvidia GPUs maintain benchmark top spot in generative AI performance tests

Nvidia’s GPU dominance in generative AI benchmarks underscores the company’s continued leadership position in the artificial intelligence hardware market. The latest MLPerf benchmark results reveal Nvidia’s commanding performance across multiple generative AI tests, with only limited competition from rivals AMD and Google. This benchmark serves as a critical industry measure, offering insights into which chips can best handle the computationally intensive demands of today’s most advanced AI applications.

The big picture: Nvidia’s general-purpose GPU chips have maintained their leadership position in the latest MLPerf benchmark tests, which now include specific measurements for generative AI applications such as large language models.

• Systems built with multiple Nvidia chips from vendors like SuperMicro, HPE, and Lenovo claimed most of the top positions in the benchmark’s fifth installment.
• The benchmarks have evolved to focus on measuring inference speed – how fast machines can produce tokens, process queries, or output data samples – critical metrics for generative AI performance.

Key additions: The MLCommons consortium updated their tests to include new benchmarks specifically targeting generative AI technologies.

• The benchmark now evaluates performance on Meta‘s open-source LLM Llama 3.1 405b and an interactive version of the smaller Llama 2 70b.
• Additional new tests measure the processing speed of graph neural networks and the assembly of LiDAR sensing data for automobile road mapping.

Competitive landscape: While Nvidia dominated the closed division tests, the benchmark revealed limited but noteworthy competition from other chip manufacturers.

• AMD’s MI300X GPU managed to achieve the top score in two Llama 2 70b tests, representing the most significant challenge to Nvidia’s dominance.
• Google participated with its Trillium chip, though it couldn’t match the performance of Nvidia’s Blackwell architecture.
• The benchmark saw fewer competitors than previous iterations, with notable absences from Intel and Qualcomm in the GPU category.

Behind the numbers: Despite not competing in the GPU category, Intel still maintained relevance in the data center space.

• Intel’s Xeon microprocessors powered seven of the top 11 systems in the datacenter closed division, highlighting the company’s continued importance in AI infrastructure.