NVIDIA Ethernet Networking Accelerates World’s Largest AI Supercomputer, Built by xAI

/in /by

October 29, 2024

NVIDIA today announced that xAI’s Colossus supercomputer cluster comprising 100,000 NVIDIA Hopper GPUs, achieved this massive scale by using the NVIDIA Spectrum-X Ethernet networking platform, which is designed to deliver superior performance to multi-tenant, hyperscale AI factories using standards-based Ethernet, for its Remote Direct Memory Access network.

Colossus, the world’s largest AI supercomputer, is being used to train xAI’s Grok family of large language models, with chatbots offered as a feature for X Premium subscribers. xAI is in the process of doubling the size of Colossus to a combined total of 200,000 NVIDIA Hopper GPUs.

“AI is becoming mission-critical and requires increased performance, security, scalability and cost-efficiency,” said Gilad Shainer, senior vice president of networking at NVIDIA.

The NVIDIA Spectrum-X Ethernet networking platform is designed to provide innovators such as xAI with faster processing, analysis and execution of AI workloads, and in turn accelerates the development, deployment and time to market of AI solutions.

“Colossus is the most powerful training system in the world,” said Elon Musk. “Nice work by the xAI team, NVIDIA and our many partners/suppliers.”

Some of the key highlights from the announcement include:

  • Spectrum X delivers up to 400 Gbps of bandwidth, significantly improving data transfer rates and reducing latency. This is crucial for organizations that rely on rapid data processing and real-time analytics.
  • The platform is optimized for AI applications, leveraging NVIDIA’s expertise in artificial intelligence to provide intelligent networking capabilities. This integration allows for smarter data routing and management, enhancing overall system performance.
  • It is built on NVIDIA’s Colossus architecture, which supports a wide range of applications from cloud computing to enterprise networking. This architecture is designed to scale efficiently, accommodating the increasing volume of data generated by modern applications.
  • With a focus on sustainability, Spectrum X aims to reduce energy consumption in data centers while maintaining high performance. This is increasingly important as organizations seek to minimize their carbon footprint.

The supporting facility and state-of-the-art supercomputer was built by xAI and NVIDIA in just 122 days, instead of the typical timeframe for systems of this size that can take many months to years. It took 19 days from the time the first rack rolled onto the floor until training began.

Source:NVIDIA Ethernet Networking Accelerates World’s Largest AI Supercomputer, Built by xAI