"Superhuman": Nvidia CEO Blown Away After Musk Sets Up 100,000 GPUs In 19 Days
Nvidia CEO Jensen Huang says Elon Musk pulled off a "superhuman" feat by setting up a supercluster of 100,000 H200 Blackwell GPUs in just 19 days - a process he says takes everyone else "one year to accomplish."
According to Tom's Hardware, the xAI team reportedly went from "concept" to full-ready compatibility with Nvidia's "gear" in less than three weeks - including running xAI's first AI training run.
From start to finish, the process involved building the massive X factory where the GPUs would reside and equipping the entire factory with liquid cooling and power to make all 200,000 GPUs operational. That's not to mention all of the coordination between Nvidia's and Elon Musk's engineering teams to get all of the hardware and infrastructure shipped and installed precisely and in a coordinated manner.
What's more, Huang says that networking Nvidia's gear isn't as simple as networking traditional data center servers. "The number of wires that goes in one node...the back of a computer is all wires," he said, adding that Musk's integration of 100,000 H200 GPUs has "never been done before," and probably won't be repeated by anyone else anytime soon.
Watch:
Nvidia CEO Jensen Huang on Elon Musk and @xAI
— ALEX (@ajtourville) October 13, 2024
“Never been done before – xAI did in 19 days what everyone else needs one year to accomplish.
That is superhuman – There's only one person in the world who could do that – Elon Musk is singular in his understanding of engineering.” pic.twitter.com/oNZuH97RhJ