More and more people are using artificial intelligence in everyday life. However, little is known about power consumption. The clarification of this question depends on whether voice AI is economically worthwhile
Google’s data center in Saint-Ghislain, Belgium: The cooling consumes energy and generates huge amounts of water vapor. (9/19/2017)
Google / Press
Be it a plane ride, heating your apartment or even just streaming your favorite series – anyone who is interested in their ecological footprint can now usually get information about their own energy consumption without much effort. Of all things, when it comes to artificial intelligence with its energy-hungry neural networks, it’s not that easy.
The end user can hardly estimate how much energy a reply from Chat-GPT costs. Not even AI developers usually know in detail about the energy consumption of their creations.
“As long as you don’t measure energy consumption, you can’t reduce it effectively,” says Stefan Naumann, professor of sustainability informatics at the Birkenfeld Environmental Campus at Trier University of Applied Sciences. Together with his colleagues, he has set himself the goal of quantifying and optimizing the energy requirements of air conditioning systems.
According to estimates, all information and communication technology is already responsible for 2 to 4 percent of global greenhouse gas emissions. This puts it roughly in the range of global air traffic. “AI has become the new driver in this area,” says Naumann. “The training of neural networks in particular requires a lot of energy.”
Complex training
During the training phase, the large language models behind applications such as Chat-GPT learn the structures of human language using huge amounts of data. To do this, they optimize the often billions of parameters of their artificial neural networks step by step in lengthy processes. And as the size of the models grows rapidly, the computational effort also explodes.
In extreme cases, according to Naumann, there are thousands of networked processors that run in huge data centers for several weeks and generate hundreds of thousands of euros in electricity costs until training is completed. An estimated 1287 megawatt hours were spent training the GPT-3 model. This corresponds roughly to the amount of electricity that a medium-sized nuclear power plant produces in one hour.
The tech companies don’t reveal any details
However, as clear as the increase in overall power consumption is, little is currently known about energy efficiency at the level of individual processors or even individual calculations. That’s why Naumann and his team are conducting experiments with smaller models on their own computers and trying to determine power consumption as precisely and in detail as possible.
And large language models are already being examined for research purposes. For example, the AI startup Hugging Face has published detailed data on the power consumption of its Bloom language model, which has 176 billion parameters.
In the future, the information should help to make optimal use of the available computing capacity. Ultimately, the energy efficiency of a data center is greatest when all processors are optimally utilized.
In reality, however, the utilization is significantly lower because individual processors have to temporarily wait for a new computing task to be assigned. Just like a switched-on PC that sits unattended on the desk, they also consume electricity during this time without providing any performance.
In the case of Bloom’s training, it turned out that only a little more than half of the energy actually went into executing the code. A third was wasted on idle computers, and the rest was spent on general data center infrastructure such as running the cooling system.
The boom since Chat-GPT increased energy consumption
Even when a model is fully trained, the energy requirement continues: with every application. According to data scientist Alex de Vries, a single chat request requires no more energy than a bedside lamp that stays on for a few hours. On the other hand, Chat-GPT alone found an impressive 100 million new users in just two months when it was launched and, according to estimates, used over 500 megawatt hours of electricity every day to keep the service running. This means that an economical electric car could circle the earth a hundred times.
Chat-GPT is now integrated into Microsoft’s search engine Bing. “Microsoft is still keeping a low profile about the energy consumption of the new system,” says Naumann. If all of the 9 billion daily search queries on Google were answered by a language model, according to de Vries’ most pessimistic estimates, that could increase energy consumption thirty times – and thus exceed the total electricity needs of a small country like Ireland.
“However, it is unclear whether combining Internet searches with language models will actually lead to increased consumption,” says Naumann. “Ultimately, better answers could also mean that fewer search queries have to be made overall.” In addition, for such a large-scale use, hundreds of thousands of new graphics cards would first have to be produced and installed, which would not only take a few years, but would probably also be economically unprofitable given the current status.
According to de Vries’ estimate, purchasing the additional hardware alone would cost around $100 billion. Even divided over three years, this, combined with the expected horrendous increase in electricity costs, would almost completely eat up the annual profit generated by Google.
The solution could be less accurate models
One proposed solution is to train more efficient AI models. This causes additional effort during training. But: “For services that are used particularly intensively, this effort would probably be worth it if the efficiency of the individual applications is improved,” says Naumann. “In any case, the goal must be to attach some kind of energy price tag to every service.”
Another key to taming the energy hunger of artificial intelligence is the accuracy of the calculations.
AI algorithms already run on special cards today, and chip manufacturers like Nvidia are adapting their newly developed chips ever better to the requirements of machine learning. Essentially, these processors are still designed for classical and mathematical calculations. “They are precise and accurate, just as we are used to from computers,” says Professor Ralf Herbrich, who heads the department of artificial intelligence and sustainability at the Hasso Plattner Institute (HPI) in Potsdam. “For AI algorithms that only estimate probabilities, they often work far too precisely and therefore use an unnecessarily large amount of energy.”
This level of accuracy is hard-wired into the processors, which is why every calculation, no matter how small, is usually carried out with an accuracy of 38 digits. “Even if the result in the end only predicts with one percent accuracy whether a picture shows a dog or a cat,” says Herbrich. Basic research is therefore currently investigating how the necessary calculation processes could be carried out with less precision.
It still has enough power, but maybe not for long
At the HPI in Potsdam, researchers are working on so-called one-bit networks, in which the parameters that describe the connections in an artificial neural network are only represented with a single bit instead of a continuous value.
“The only information available is whether there is a connection or not,” explains Herbrich. “In certain cases, this can reduce the computing effort to a tenth, while the accuracy of the result hardly decreases because the errors are averaged out across the many nodes of the network.” At the same time, flexible hardware is being developed in the research departments of major chip manufacturers, the accuracy of which can be adapted to the requirements of the algorithms.
“It is high time to look intensively at the energy costs of AI,” says Herbrich. The starting signal for the triumph of large language models was five years ago. “Today we still have enough electricity, but the path from basic research to application is long,” says the researcher warningly. “And if we don’t want to have an energy problem in five years, we have to address it now.”
An article from “NZZ am Sonntag”