Building Colossus: Inside the AI supercomputer Supermicro is building for xAI

A cutting-edge AI supercomputer, claimed to be the world’s largest liquid-cooled GPU cluster, is being built through a partnership between xAI, Supermicro, and NVIDIA in Memphis, Tennessee.

Project scale and specifications: The supercomputer installation represents an unprecedented deployment of computing power in terms of both size and speed.

• The system incorporates over 100,000 NVIDIA HGX H100 GPUs across four compute halls, with each hall housing approximately 25,000 GPUs
• The facility includes exabytes of storage capacity (an exabyte is equivalent to one billion gigabytes)
• The entire system was transformed from an empty building to a functioning AI supercomputer in just 122 days

Technical architecture: The supercomputer employs a modular design centered around liquid-cooled server racks for maximum efficiency and performance.

• Each rack contains eight 4U Universal GPU systems, with each system equipped with 8 NVIDIA HGX H100 GPUs
• The innovative design allows for server maintenance without removing units from the rack, reducing downtime and improving serviceability
• An advanced liquid cooling system manages the intense heat generated by the densely packed GPUs

Networking capabilities: The system utilizes state-of-the-art networking infrastructure to enable rapid data processing and communication between components.

• The NVIDIA Spectrum-X Ethernet platform and Bluefield-3 SuperNICs provide 400Gbps networking speeds
• NVIDIA Spectrum SN5600 switches support 128 400-gigabit ethernet links per unit
• This high-speed networking infrastructure enables efficient communication between the thousands of GPUs working in parallel

Future implications: While this supercomputer demonstrates remarkable technical achievements in scale and deployment speed, questions remain about its energy consumption, environmental impact, and whether such massive computing resources concentrated in private hands could affect the broader AI development landscape.