This article is from the WeChat public account: Snow Leopard Finance Club (ID: xuebaocaijingshe), author: Wei Linhua, title picture from: Visual China
At the end of March, Holtec Palisades, a shut down nuclear power plant on the shores of Lake Michigan, entered the restart phase with a $1.5 billion loan guarantee from the U.S. Department of Energy. If it passes the regulatory process, it will be the first nuclear power plant in U.S. history to be restarted.
The U.S. Department of Energy (DOE) restarted nuclear power generation out of concern about the imbalance in electricity demand. At a time when the demand for electricity consumption in manufacturing, electric vehicles and other industries is rising, the rapid development of the artificial intelligence industry has accelerated the arrival of the power crisis in the United States.
"AI itself is not a problem because it can help solve the problem. U.S. Energy Secretary Jennifer Granholm said in an interview with Axiro, but the growing demand for electricity in artificial intelligence and data centers is a real new problem.
A small app, an electric maniac
How power-hungry are AI applications?
In his paper, Dutch scientist Alex de Vries calculated this for the conversational bot ChatGPT:
Whenever ChatGPT tries to respond to a question, it consumes 2.9 watt-hours of power. What is this concept? 10 responses can support a 15W LED bulb for 2 hours, and 100 responses can charge a smartphone about 13 times.
In a single day, ChatGPT needs to process an average of about 200 million conversation requests from users, which means that it consumes more than 564 MWh of electricity in a single day (1 MWh = 1,000 kWh, 564 MWh is equivalent to 564,000 kWh). Based on the average daily electricity consumption of each household in the United States, ChatGPT needs to consume the electricity consumption of 17,000 American households every day.
Due to differences in model parameters, energy consumption and other factors, the power consumption of different AI models cannot be accurately estimated. Therefore, Alex used the A100 server launched by NVIDIA as the measurement object to estimate the possible power consumption of the entire artificial intelligence industry.
Based on his assumptions, Nvidia could launch 1.5 million A100 servers by 2027, 95% of which will be used in the AI industry. Each DGX A100 server has 8 built-in A100 chips, and with the power consumption of 11.4 million A100 chips, it is estimated that in 2027, the annual power consumption of the entire AI industry will be as high as 85~134 TWh (1 TWh = 1×106 kWh).
In other words, by 2027, AI electricity consumption could be close to the total annual electricity consumption of Sweden with a population of more than 10 million or the Netherlands with a population of 17 million, equivalent to 0.5% of the current global electricity consumption.
According to this estimate, the power consumption of AI may be comparable to that of Bitcoin mining. According to the University of Cambridge, Bitcoin mining consumes about 165.99 terawatt hours a year, which is close to the annual electricity consumption of Egypt, which has a population of 100 million.
Bitcoin mining is so power-hungry because of the way it works. In Satoshi Nakamoto, the father of Bitcoin, Bitcoin uses Proof-of-Work to allow miners to compete to calculate a hash (a string of numbers + letters) that is difficult enough to create a new block and earn a reward. This competitive computing process requires a lot of power and computing power.
The reason why AI is so power-hungry is that the training and inference processes of large models consume a lot of electricity.
The key to the quality of large models lies in data, computing power and top talents, and behind the high computing power is the support of tens of thousands of chips running around the clock.
GPUs (graphics processing units) have proven to be more suitable hardware for AI training than CPUs (central processing units) that are usually installed on laptops. If you think of the CPU as an element that handles a single task, then the advantage of the GPU is that it can handle multiple concurrent tasks at the same time. Although GPUs were not originally designed to handle AI needs, the nature of multitasking at the same time makes them a ticket to the training ground for large AI models.
Compared with the CPU, the GPU can handle multiple parallel tasks, source: Nvidia's official website
Fast comes at the cost of higher energy loss. It is estimated that the energy consumption of a GPU is 10~15 times higher than that of a CPU. In the process of large model training, multiple GPUs need to run one after another. The larger the number of parameters and data of a large model, the greater the power consumption of training.
Taking GPT-3 training as an example, the "2023 Artificial Intelligence Index Report" released by the Stanford Institute for Artificial Intelligence shows that GPT-3 with 175 billion parameters consumes up to 1,287 MWh of power during the training phase. Liu Jie, dean of the Institute of Artificial Intelligence of Harbin Institute of Technology, made an analogy, which is equivalent to driving from the earth to the moon and round trip.
After training, the AI consumes far more power for inference than it consumes for training.
Each time it responds to a request, the large model needs to complete the inference process to find the closest solution to the problem. According to the above data, the power consumed by GPT-3 during the training phase cannot even support ChatGPT to run for 3 days.
With the development of multimodal large models into the mainstream, the power consumption will further increase in the inference process of AI response to demand. According to research by artificial intelligence company Hugging Face, not only do multimodal large models consume much more power than ordinary models, but models involving image processing also consume more power than plain text processing.
Specific to different tasks, simple tasks such as text classification, marking, and Q&A are relatively low-cost, and only 0.002~0.007 kWh are required for 1,000 times of reasoning. In response to multimodal tasks, a text-to-image generation requires up to 2.9 kWh, which is equivalent to the power consumption of ChatGPT 100 times.
The giant's AI dream has torn the power gap even wider
From training GPT-2 with 1.5 billion parameters to GPT-3 with 175 billion parameters, OpenAI's leap from one billion parameters to 100 billion parameters took only one year.
As large models soar, more and more large technology companies have begun to put the integration of AI and the company's main business on the agenda.
Google has tried to incorporate AI capabilities into search, but its energy consumption is staggering. In February last year, John Hennessy, chairman of Google's parent company Alphabet, said that applying AI in search would cost 10 times more than regular search.
According to the 2023 Artificial Intelligence Index Report released by the Stanford Institute for Artificial Intelligence, the power consumption of each AI search is about 8.9 watt-hours, compared to the power consumption of 0.3 watt-hours in a single Google search, and the single power consumption of adding AI is almost 30 times that of ordinary searches.
Microsoft, which works closely with OpenAI, also plans to vigorously "cramm" AI into several of its main product lines, such as Office software, Windows operating system, Bing search engine, Azure cloud service, etc.
In order to provide more sufficient computing power to support the training and use of large AI models, the construction of data centers as the infrastructure of the foundation has been included in the next step of the planning of technology companies.
In 2023, Google will spend more than $2.5 billion to build data centers in Ohio, Iowa, and Mesa, Arizona. Amazon, which is optimistic about the development of AI, plans to invest $150 billion in data centers over the next 15 years.
When the inflated demand for electricity could not be met one by one, the power of some cities in the United States sounded the alarm of emergency.
The United States has the largest number of data centers in the world. As of 2022, the U.S. has more than 2,300 data centers, accounting for 1/3 of the world's data centers.
Among them, cloud computing giants, including Amazon, Microsoft, Google, etc., have a particularly large data center layout in the United States. According to Synergy Research Group, Amazon, Microsoft and Google together account for more than half of all major data centers among hyperscalers. Microsoft has 24 Availability Zones in the United States, with three or more data centers in one Availability Zone.
According to the International Energy Agency (IEA), the U.S. data center electricity consumption will grow rapidly in the coming years. The IEA warns that data centers in the U.S. will use more than 4% of total electricity in the U.S. in 2022 and will increase to 6% by 2026 and will continue to expand in the years to come.
But, contrary to the rapidly growing demand for AI-powered electricity, the U.S. shows no signs of significant growth.
According to the U.S. Energy Information Administration, in 2023, the U.S. will generate 4,178.171 billion kWh of full-caliber net electricity, down 1.2% from the previous year. In fact, for nearly a decade, the U.S. has been teetering on the edge of 4,000 billion kilowatt-hours of annual net electricity generation.
Changes in U.S. net electricity generation from 1950 to 2023 (in billion kWh), photo: Statista
One of the main culprits in the United States is its fragile power grid transmission facilities. Power grid infrastructure such as transformers and transmission lines in the United States was built in the 60s and 80s of the last century, and the problem of circuit aging is obvious. In a 2022 document, the White House noted that many transformers and transmission lines are approaching or exceeding their design lifespan, with 70 percent of the nation's transmission lines having been in use for more than 25 years.
With an aging grid infrastructure, the idea of the U.S. transmitting electricity from other parts of the world and connecting to clean energy sources to expand the grid and reserve energy is not possible. A report released by the U.S. Department of Energy (DOE) pointed out that in Texas, Alaska and other regions, the transmission system built in the United States is already facing full load.
To strengthen the resilience and reliability of U.S. state power grids, last year, the U.S. Department of Energy announced that it would invest $3.46 billion in 58 projects in 44 states.
The power crisis is imminent. In the near future, it may also become a key constraint on the development of AI.
In February 2024, at the World Economic Forum in Davos, OpenAI CEO Sam Altman mentioned the power crisis brought about by AI. In his opinion, AI will consume far more electricity than people expect. "We are not yet fully aware of the energy needs of AI. We can't get there without a major breakthrough. ”
At the Bosch Internet Forum, Tesla CEO Elon Musk also emphasized the development dilemma faced by artificial intelligence. "The next shortage will be electricity. He judged that the power deficit could occur as early as 2025. "You'll see next year that we don't have enough power to run all the chips. ”
Clamp and the way out
Overburdened power grids have begun to limit the expansion of tech companies.
On Social Media X, OpenPipe founder Kyle Corbitt shared a conversation he had with Microsoft engineers about the transfer dilemma that OpenAI faced with GPUs between different states during the training of GPT-6.
"It's impossible for us to put more than 100,000 H100 chips in one state without destroying the power grid. "The maximum power consumption of an H100 is 700 watts, and according to the calculations of Microsoft engineers, the power consumption of 100,000 H100s will be as high as 42 MWh based on an annual utilization rate of 61%.
To meet soaring electricity demand, the first thing to be sacrificed is the goal of reducing carbon emissions.
According to the Washington Post, electricity demand growth in many parts of the United States has exceeded expectations. In Georgia, for example, new electricity consumption over the next decade is expected to be 17 times more than the last 17 times. Coal-fired power plants in Kansas, Nebraska, Wisconsin and South Carolina have decided to delay retirement.
In the face of large power mining machines, different countries have introduced different degrees of regulatory policies. The U.S. Department of Energy estimates that cryptocurrency mining could account for between 0.6% and 2.3% of U.S. electricity consumption annually. To this end, the United States is considering imposing a consumption tax of up to 30% on digital asset mining energy for cryptocurrency mining operations. Three provinces in Canada have announced bans on cryptocurrency mining.
AI has also attracted the attention of regulators. Because it is difficult to quantify the energy consumption of each AI company, it is difficult to quantify and estimate it uniformly. Overseas regulators have begun to push for legislation that requires AI developers to disclose energy usage in order to reasonably estimate the impact of AI on energy consumption.
In March, the Artificial Intelligence Act, approved by the EU's 27 member states, requires "high-risk AI systems" to report on their energy consumption and resource use.
The helmsman of a tech company bet on new energy companies a few years ago, expecting to support huge electricity demand with clean, renewable energy.
In 2021, OpenAI CEO Altman invested $375 million in fusion startup Helion Energy. In May 2023, Microsoft signed a power purchase agreement with the company, which aims to purchase 50 megawatts of electricity from it starting in 2028. The bad news is that it's not even enough to support 1/25th of the power consumed by GPT-3 training.
Energy consumption can also be significantly reduced by optimizing performance through technology.
At this year's GTC conference, Nvidia CEO Jensen Huang brought a new GPU product, Blackwell. By using the new architecture, it reduces energy consumption by more than 70 percent: training a 1.8 trillion parameter GPT model, traditional methods can require 8,000 GPUs, 15 megawatts, and 90 days. Blackwell, on the other hand, only needs 2,000 GPUs and consumes 4 megawatts.
Huang is equally concerned about the supply of electricity in contrast to the cautionary tales of Musk and Altman, but he gives a more optimistic outlook: "Over the past decade, we've increased computing and artificial intelligence by a factor of 1 million...... And the cost, space, or energy it consumes has not increased by a factor of 1 million. ”
Write at the end
More than a century ago, the energy revolution changed the way people lived. From the thermal power of burning wheat straw to coal and oil, in the critical period of historical development, people's excavation of new energy has promoted the process of industrial revolution.
"There is power and civilization in every coal basket. Emerson, an American thinker and writer, once sighed.
The scarcity of one energy source often becomes the driving force for mining a new energy source. In The Love and Hate of Black Stone: The Story of Coal, author Barbara Frieze recounts the "timber crisis" that took place in 16th-century England.
"As the city expanded, the forests in the nearby counties were gradually deforested, and people had to bring in timber from farther and farther away....... Brewers in London alone burn 20,000 truckloads of wood every year. "When the price of wood rose faster than inflation and became a scarce resource, the amount of coal used in the UK increased dramatically.
The extraction and use of energy has become a key hand in determining industrial development. Abundant coal supported the development of the textile and steel industries in Britain, becoming the center of the first industrial revolution, and the extraction of oil led to the prosperity of automobiles, aircraft and other industries.
Under the crisis of fossil energy depletion, the use of new energy can not only alleviate the energy crisis of the artificial intelligence industry, but also carry the "power and civilization" of human science and technology to continue to move forward.
Resources
[1] Granholm eyes talks with Big Tech on AI power needs. Axios
[2] Amid explosive demand， America is running out of power. The Washington Post
[3] Nvidia CEO Jensen Huang at World Government Summit.
[4] The AI Act Explorer.
[5] Bitcoin: A Peer-to-Peer Electronic Cash System.Satoshi Nakamoto
[6] A.I. Could Soon Need as Much Electricity as an Entire Country.The New York Times
[7] Cambridge Blockchain Network Sustainability Index: CBECI. CCAF
[8] The Biden-Harris Administration Advances Transmission Buildout to Deliver Affordable， Clean Electricity.The White House
[9] Microsoft， Amazon and Google Account for Over Half of Today’s 600 Hyperscale Data Centers.Synergy Research Group
This article is from the WeChat public account: Snow Leopard Finance Club (ID: xuebaocaijingshe), author: Wei Linhua
This content is the author's independent view and does not represent the position of Tiger Sniff. May not be reproduced without permission, please contact [email protected] for authorization
People who are changing and want to change the world are all in the Tiger Sniff APP