Eight quadrillion calculations per second: The new GH200 Grace Hopper computer chip performs more than 4,000 smartphones combined: portrait of a high performer.
The GH200 Grace Hopper computer chip is more powerful than 4,000 smartphone processors combined.
Illustration Andrin Engel
The hourglass turns. And spins. And spins. Anyone who uses artificial intelligence to generate an image or text must be patient. The waiting time is nothing compared to the time it takes to train the models behind the AI generators. Chat-GPT, for example, took 34 days and during this time used about as much electricity as a thousand households per year – for training alone.
So it’s no wonder that engineers and programmers around the world are trying to make AI models more efficient. Nvidia, the Silicon Valley hardware giant, will launch a processor called the GH200 Grace Hopper in 2024 that will accelerate AI more than any that came before it.
In order to train an AI, computers have to carry out an extremely large number of calculations – not one after the other, but preferably at the same time, in parallel. This requires a lot of computing power – and memory, i.e. storage capacity. That’s exactly what Grace Hopper is optimized for.
What’s inside the new “superchip”?
200 billion times one or zero
The heart of every computer is a plate of transistors. They act as switches and switch between two states: on and off, one and zero. Computers can calculate with this, i.e. process, display and store data.
What seems like magic to most people is something Frank Gürkaynak, who researches computer chips at ETH Zurich, deals with in everyday life. He says: “The more transistors, the faster a computer chip – in theory.”
Two key points determine whether Grace Hopper can actually calculate faster with 200 billion transistors than a modern smartphone with around 16 billion: on the one hand, the efficiency of the many programs that are interposed between the hardware and the end user, i.e. the algorithms with which the Chip is asked to do the math.
And on the other hand, the data structure of the raw data: only if it is cleanly processed and fed into the chip in a well-organized manner can it utilize its capacity.
To what extent this actually works in the new Grace Hopper is still unclear. No independent analyzes can be made before the chips are delivered in 2024.
Jensen Huang, founder and CEO of Nvidia, will introduce the GH200 Grace Hopper at an IT conference in Taipei, Taiwan in late May 2023. On the left is a cluster of chips that work together like a supercomputer.
Walid Berrazeg / Getty
Computing power: 4,000 times as fast as a smartphone
The information on computing power shows what the many billions of transistors are capable of. Grace Hopper calculates at a speed of 8 petaflops. This means the chip can perform 8 quadrillion calculations in one second.
A thought experiment shows how large this number is: let’s say that everyone in Switzerland, from infants to residents of old people’s homes, would each have to solve a billion arithmetic problems. This is impossible within the human lifetime. But if it still succeeded, Switzerland would have a computing power of 8 petaflops: 8 million people times a billion calculations, that’s 8 quadrillion calculations. Grace Hopper can do it in a second.
This makes Grace Hopper around 4,000 times as fast as today’s smartphone. Or to put it another way: If you wanted to express Grace Hopper’s computing power in smartphones, you would have to connect 4,000 of the fastest devices currently commercially available. If you removed the processors and laid them out on the floor, you would have an area of over two square meters. Grace Hopper, on the other hand, is about the size of a hand.
Storage capacity: Like watching TV for 17 days
But just being fast is not enough in the AI age – you also need a particularly large storage capacity in order to be able to process the large amounts of data that are necessary for training and using AI algorithms.
Grace Hopper has a storage capacity (memory) of 1.2 terabytes. This can be used to store around 400 hours of video data, i.e. around 200 films. If these were put together and played without a break, you would have to watch TV for 17 days and nights.
Compared to its previous version, the new Grace Hopper has three times more memory. That’s why, according to Nvidia, it’s three times as fast in AI applications.
But hardware researcher Gürkaynak also says: “Just more memory and more flops do not automatically lead to higher performance.” What really makes computer chips faster is the ability to efficiently route data to the processors that perform calculations. “This is the big challenge for all hardware manufacturers.”
CPU and GPU together: a “super chip”
In order to make the data flow as fast as possible, at least within the chip, Nvidia has come up with a lot of ideas. Grace Hopper combines a CPU, a central processing unit, with a graphics processor, a GPU. GPUs were once developed for graphics in video games. Today, however, they are used everywhere where many calculations run in parallel – especially in AI.
The combination of CPU and GPU makes Grace Hopper a “super chip” in Nvidia’s eyes. By the way, it was named after a woman: Grace Hopper was an American Navy admiral and engineer who worked on the first computer, Mark I, in 1944 and invented the Cobol programming language, which is still used today.
Many companies today work with a combination of CPU and GPU. But the question is how the two parts are connected – i.e. how quickly data can flow back and forth between them. The key figure for this is 900 gigabytes per second, which is 700 times as fast as a cellular connection with 5G and would allow over a hundred movies to be sent back and forth in a single second.
The same applies here: the extent to which this will actually be achieved in the new Grace Hopper is still untested. At the moment, only the key figures from the marketing material are available, not independent analyses.
The combination of Grace CPU (left) and Hopper GPU (right) makes GH200 Grace Hopper a “super chip”. Photo of the GH200 Grace Hopper taken May 29, 2023, Taipei, Taiwan.
Imago
Power consumption: about as much power as a hair dryer
Anyone who calculates as much as Grace Hopper also needs a lot of energy. The manufacturer Nvidia left an inquiry from the NZZ about the exact power consumption of an individual chip unanswered. But the approximate power requirements can be calculated from information about data centers that are to be equipped with Grace Hopper. Accordingly, a single chip requires around 1230 watts – about as much as a hairdryer.
Now Grace Hopper is probably only used alone very rarely. The chip was developed as part of a larger system. Nvidia wants to connect 256 Grace Hopper chips together and use them to build a data center. Connected to each other, the chips function as supercomputers. If the project succeeds, the system should immediately make it onto the list of the fifty fastest computers next year.
If the individual chips require a lot of power, this applies even more to entire data centers. A large data center quickly consumes as much electricity as a small town. Now Nvidia repeatedly emphasizes in its marketing documents that Grace Hopper uses 20 times less power for AI applications than comparable systems. Nevertheless, Nvidia’s new supercomputer with the Grace Hopper chips is likely to consume more electricity than 500 average Swiss households combined.
Potential: AI will become commonplace
If Nvidia keeps its marketing promise, the new chip could create the technical prerequisites for various new AI applications to be economically worthwhile. Ultimately, countless industries are hoping for breakthroughs thanks to AI, from autonomous driving to the development of new medicines to more accurate prediction of natural disasters.
Regardless of whether you use AI to write your Christmas cards or develop new medicines: the hourglass is likely to turn for less and less time in the future.