The staggering numbers behind the big AI model: 5 questions to ask ChatGPT that consumes 500ml of water?

Author | Ling Min, Nuclear Coke

The explosion of ChatGPT has set off a boom in AI large models, but scientific and technological progress has always been a double-sided sword, and AI large models have brought many conveniences, but also caused people to worry about energy consumption and environmental pollution. Previously, studies have said that training GPT-3 emits carbon emissions equivalent to driving to and from the moon, and studies have said that training artificial intelligence models emits much more carbon than long-distance flights.

In terms of energy consumption, building large-language models requires analyzing patterns embedded in massive amounts of human texts, all of which consume a lot of electricity and generate considerable heat. To stay cool, data centers need pumped water to cool and store it in warehouse-sized cooling towers outside the equipment room.

Recently, foreign media reported that Microsoft's technical facilities to support OpenAI require a lot of water, which is pumped from the watershed of the Raccoon and Des Moines rivers in central Iowa to cool a powerful supercomputer. One study showed that large AI models like ChatGPT consume staggering amounts of water: for every 5-50 prompts or questions a user asks ChatGPT, ChatGPT consumes 500 milliliters of water.

Ask ChatGPT 5-50 questions, which consumes 500ml of water

Recently, Microsoft revealed in its latest environmental report that from 2021 to 2022, the company's global facility water consumption soared by 34% (to nearly 1.7 billion gallons, equivalent to more than 2,500 Olympic-level swimming pools). This number is much higher than outside researchers a few years ago, and the driving force behind it is naturally the need to build AI.

Shaolei Ren, a researcher at the University of California, Riverside, said that "it is reasonable to infer that most of the increase [in water use] is due to AI," including "a lot of investment in generative AI and partnerships with OpenAI." ”

Google reported a 20% year-over-year increase in water use, which Ren believes is also largely due to AI research needs. Of course, Google's water use growth has been uneven — water use for its Oregon infrastructure has remained stable, but the area around Las Vegas has doubled. Iowa is also a big water user, and Google's Conthell Bluffs data center here consumes more water than anywhere else.

In a paper to be published later this year, the Ren research team estimates that for every 5-50 prompts or questions users ask ChatGPT, ChatGPT consumes 500 milliliters of water (depending on the location of the infrastructure and the seasonal climate). This estimate does not include unmeasured indirect water use, such as power generation for data center cooling power.

Ren says, "Most people are unaware of ChatGPT's resource consumption. But if we don't understand resource usage, we can't help conserve resources. ”

It is understood that Microsoft allocated its first $1 billion investment to San Francisco-based OpenAI in 2019. Subsequently, OpenAI officially released ChatGPT. As part of the partnership agreement, Microsoft is responsible for providing OpenAI with the computing power needed to train AI models.

To deliver on their promises, the two companies are turning to West Des Moines, Iowa, a town of 68,000 that has been Microsoft's data center hub for more than a decade, powering its cloud computing services. Microsoft's fourth and fifth data centers will open here later this year.

It is understood that the local climate in Iowa is quite cool for most of the year, and Microsoft can directly use the outdoor air to keep the supercomputer running properly and discharge the generated heat directly. The company said in a disclosure report that they only need to switch to water-cooled mode when the temperature exceeds 29.3 degrees Celsius.

But even so, the water consumption of local facilities in the summer is still quite staggering. In July 2022, a month before OpenAI officially completed its GPT-4 training, Microsoft pumped about 11.5 million gallons of water to its Iowa data center cluster, or about 6% of the region's total water use, according to West Des Moines Waterworks.

In 2022, a document from the plant mentioned that the company and local governments will no longer "consider approving Microsoft's future data center projects" unless Microsoft can "demonstrate and implement technologies that can significantly reduce peak water use." Only then can they secure the water supply needs of local residential and other commercial operations.

Microsoft said it is working directly with the water plant to address the feedback from the other party. The water plant pointed out in a written statement that Microsoft has always been a good partner and has been working with local officials to explore how to meet demand while reducing water consumption.

What are the carbon emissions of the big model?

In addition to energy consumption, the carbon emissions of large AI models such as ChatGPT have also caused public concern. Previously, computer scientists said that the carbon emissions of GPT-3 throughout the training cycle are equivalent to driving to the moon and back to Earth; The GPT-3 consumes enough electricity in one round of training to support 126 average families in Denmark for an entire year.

The experts who made this guess came from the University of Copenhagen in Denmark, and they developed an open-source tool called Carbontracker to predict the carbon footprint of AI algorithms. Carbontracker estimates that the neural supernetwork built with NVIDIA GPUs in Microsoft's data centers runs at about 190,000 kilowatt-hours, which would produce 85,000 kilograms (85 tons) of carbon dioxide at average U.S. carbon emission levels, equivalent to the emissions of building a new car in 2017. Such emissions are equivalent to 800,000 kilometers of vehicle travel in Europe, which is roughly equivalent to the total distance driven to the moon and back to Earth.

Lasse Wolff Anthony, one of the creators of Carbontracker and co-author of the AI Power Consumption Research paper, believes that communities must take resource consumption seriously. The article mentions that between 2012 and 2018, the cost of energy for AI research increased by about 300,000 times.

Anthony said in the interview, "The CO2 estimate is calculated based on the average carbon emissions of local electricity generation during the model training period plus the total power consumption of the hardware running the model. "We track carbon intensity through multiple APIs. If no API is available in the region where the model is trained, we default to the European average, as there is currently no freely available global monitoring data. These APIs periodically query hardware energy consumption during training to accurately estimate the overall carbon footprint. ”

Of course, the premise of the above results is that the data center that trained GPT-3 is completely dependent on fossil fuels, which may be different from the actual situation.

Some analysts believe that the carbon emissions of the current large model may be seriously exaggerated. In fact, the global technology sector accounts for only 1.8%-3.9% of total warm gas emissions, and only a fraction of them are AI-related. At scale, AI's carbon emissions are nowhere near comparable to other major sources of carbon, such as aviation. Compared to cars and airplanes that are running at all times, the carbon emissions corresponding to training models such as GPT are definitely not the main contradiction.

Compared to cars and airplanes that are running at all times, the carbon emissions corresponding to training models such as GPT are definitely not the main contradiction.

Admittedly, we don't know exactly how many big AI models are being trained, but if you only consider GPT-3 or other larger models, the total number of such models is less than 1,000. Here we can do a simple calculation:

A recent assessment concluded that training GPT-3 emits 500 tons of carbon dioxide, while Meta's Llama model estimates 173 tons. If 1,000 such models are trained, the total CO2 emissions are about 500,000 tons. In 2019, the commercial aviation industry emitted about 920 million tons of CO2, almost 2,000 times more than large language models were trained. And it should be noted that this is one year of aviation industry operation compared to years of large language model training. While the environmental impact of the latter is a cause for concern, exaggeration is clearly contrary to objective fairness and requires more careful consideration.

Of course, this is only the model training stage. The operation and use of the model also consumes electricity and generates associated emissions. According to one analysis, ChatGPT may emit about 15,000 tons of carbon dioxide for a year of operation. But another analysis is much more optimistic, putting it at around 1,400 tons. But no matter which figure is taken, although it is not so low as to be negligible, there are still several orders of magnitude of difference compared to aviation.

It should be emphasized that the point of the problem is not to explore the carbon footprint of large models such as GPT-3, but to draw attention to the huge resources consumed to train advanced neural networks.

At present, many enterprises have begun to pay attention to energy consumption and environmental pollution, and are formulating corresponding solutions. In a statement, Microsoft said it is funding research to measure the energy consumption and carbon footprint of AI development, "while working to improve the efficiency of training and application of large language model systems." ”

"We will continue to monitor our own emissions, accelerate progress, and increase our use of clean energy to power data centers and source renewable energy to achieve our carbon negative, positive water cycle, and zero waste sustainability goals by 2030," Microsoft said. ”

OpenAI echoed the comments, saying it was "seriously considering" how to make better use of valuable computing power. "We realized that training large models could consume electricity and water," so we are working to improve efficiency. ”

A transparent emissions regime is needed

As AI systems continue to be developed and applied, we really need to be concerned about their impact on the environment. In addition to traditionally proven practices, we should explore ideas for reducing emissions specific to generative AI.

First, transparent emissions will be essential. With this assurance of transparency, we can monitor the carbon emissions associated with AI model training and usage, ensuring that model deployers and end users can develop AI usage strategies based on these numbers. In addition, AI-related emissions should be included in GHG inventories and net-zero targets as part of an overall transparency regime for AI.

France recently passed a law requiring telecommunications companies to submit transparency reports on their sustainable development. Similar laws may in the future require AI-powered products to report their carbon emissions to customers and require model providers to open up carbon emissions data through APIs.

Greater transparency will lead to stronger incentives to build increasingly energy-efficient generative AI systems and explore new avenues for efficiency. In a recent InfoQ article, Sara Bergman, Senior Software Engineer at Microsoft, called attention to the entire lifecycle of AI systems and suggested adopting tools and practices proposed by the Green Software Foundation to improve the energy efficiency of AI systems. Specific terms include careful consideration of server hardware and architecture choices, and attention to time/region differences in power generation emissions. What's more, generative AI itself is expected to make a unique contribution to improving energy efficiency.

Reference Links:

https://apnews.com/article/chatgpt-gpt4-iowa-ai-water-consumption-microsoft-f551fde98083d17a7e8d904f8be822c4

https://www.infoq.com/articles/carbon-emissions-generative-ai/

https://www.theregister.com/2020/11/04/gpt3_carbon_footprint_estimate/

This article is reproduced from:

https://www.infoq.cn/news/NuKxISZRb5sjg1lXgmeN