Among the many announcements NVIDIA CEO Jensen Huang made during his keynote at COMPUTEX 2023, one of the highlights is the next-gen supercomputer of NVIDIA called the DGX GH200 that’s aimed towards generative AI.

This supercomputer has 256 GH200 Grace Hopper Superchips, each having their own Arm-based Grace GPU and H100 Tensor Core GPU, that are combined together with a new NVLink Switch System to make them function as a large single GPU. All of those chips help the DGX GH200 deliver up to an insane 1 exaflop of performance, which is paired with 144TB of shared memory. NVIDIA claims that the GH200 has around 500x more memory than its A100 system, along with as much as 10x more bandwidth to offer.

This makes the DGX GH200 one of the most powerful supercomputers on the planet, and NVIDIA already has a number of interested customers that include Google Cloud, Meta, and Microsoft.

“Training large AI models is traditionally a resource- and time-intensive task,” Microsoft Azure Infrastructure Corporate Vice President Girish Bablani said. “The potential for DGX GH200 to work with terabyte-sized datasets would allow developers to conduct advanced research at a larger scale and accelerated speeds.”

If that’s not enough, NVIDIA themselves are building an even more powerful supercomputer with the Helios that combines four DGX GH200 systems. Both the single DGX GH200 and the larger Helios are set to be made available by the end of 2023.