Shin Ji Won reports
Edit: Peach
Microsoft, Google, Meta and other large technology companies chase me to seize market opportunities. This AI race has skyrocketed Nvidia's market value, which is about 5 Intels.
ChatGPT is in hand, there are all questions to answer.
You know, the computational cost of every conversation with it is tearful.
Earlier, analysts said that ChatGPT replied once, which cost 2 cents.
You know, the computing power required by artificial intelligence chatbots is behind the burning of the GPU.
This just makes chip companies like Nvidia make a lot of money.
On February 23, Nvidia's stock price soared, increasing its market value by more than $70 billion, with a total market value of more than $580 billion, about five times that of Intel.
Outside of NVIDIA, AMD can be called the second largest player in the graphics processor industry, with a market share of about 20%.
Intel holds less than 1% of the market.
ChatGPT is running, Nvidia is earning
As ChatGPT unlocks potential use cases, this could usher in another inflection point for AI adoption.
Why do you say that?
John Hennessy, chairman of Google's parent company Alphabet, said in an interview with Reuters that the conversation cost of artificial intelligence such as large language models can be more than 10 times that of traditional search engines.
Morgan Stanley analyzed that Google had a total of 3.3 trillion searches last year, and the cost per search was only 0.2 cents.
It is estimated that if Google's chatbot Bard were introduced to the search engine and used to handle half of Google's searches and questions, the company's cost could increase by $6 billion in 2024 based on 50 words per answer.
Similarly, SemiAnalysis, a consulting firm focused on chip technology, said that adding chatbots to search engines could cost Google an additional $3 billion, influenced by Google's in-house chip Tensor Processing Units.
He believes that Google must reduce the operating costs of such AI, but the process will not be easy, and the worst-case scenario will take several years.
As such, searching through AI language models requires more computing power than traditional search.
Analysts say the additional costs could run into the billions of dollars in the coming years.
Gartner expects that by 2026, the share of specialty chips such as GPUs used in data centers is expected to rise by more than 15% from less than 3% in 2020.
While it's hard to pinpoint exactly how much AI accounts for Nvidia's revenue today, AI has the potential to grow exponentially as big tech companies race to develop similar AI applications.
On Wednesday, Nvidia also announced an AI cloud service and is working on Oracle, Microsoft and Google to provide it with the ability to use Nvidia GTX for AI processing through simple browser access.
The new platform, which will be provided by other cloud providers, will help technology companies that don't have the infrastructure to build their own.
"People's enthusiasm for ChatGPT has made business leaders see the power of AI. But now, it is mostly a general-purpose software. The realization of its true value also needs to be tailored to the company's own needs, so that it can improve its own services and products."
NVIDIA dominates the GPU market
According to New Street Research, Nvidia accounts for 95% of the graphics processor market.
In the Philadelphia Stock Exchange semiconductor index, Nvidia stock has risen 42% this year, the best performer.
Investors flocked to Nvidia, betting that demand for AI systems such as ChatGPT would push up orders for the company's products, making it once again the world's most valuable chipmaker.
For a long time, whether it is ChatGPT, which has become the top, or models such as Bard and Stable Diffusion, there is a chip worth about $10,000 Nvidia A100 behind the computing power.
The NVIDIA A100 is capable of performing many simple calculations at the same time, which is important for training and using neural network models.
The technology behind the A100 was originally used to render complex 3D graphics in games. Now, the goal is to handle machine learning tasks and run them in data centers.
Investor Nathan Benaich said the A100 has now become a "workhorse" for AI professionals. His report also lists some of the companies that use the A100 supercomputer.
Machine learning tasks can consume the processing power of an entire computer, sometimes lasting hours or days.
This means that companies with a best-selling AI product often need to buy more GPUs to cope with peak access periods or improve their models.
In addition to a single A100 on the card that can be plugged into an existing server, many data centers use a system with eight A100 graphics processors.
This system, the Nvidia DGX A100, costs up to $200,000 for a single system.
Nvidia said Wednesday it will sell cloud access to DGX systems directly, which could lower the cost of entry for researchers.
So what does it cost to run a new version of Bing?
An evaluation by New Street Research found that OpenAI-based ChatGPT models in Bing Search can require 8 GPUs to give an answer to an answer in less than a second.
At this rate, Microsoft would need more than 20,000 8-GPU servers to deploy this model to everyone.
Microsoft could then spend $4 billion on infrastructure spending.
This is only Microsoft, if it wants to reach the scale of Google's daily queries, which is 8 billion to 9 billion queries per day, it will cost $80 billion.
The latest version of Stable Diffusion, for example, performs 200,000 hours of computation on 256 A100 GPUs or 32 DGX A100s.
Mostaque, CEO of Stability AI, said that training the model alone would cost $600,000 at market prices. This price is very cheap compared to competitors. This does not include the cost of inference or deploying the model.
Huang said in an interview,
In terms of the amount of computation required for such models, Stability AI's products are actually not expensive.
We took a data center that would have required $1 billion to run CPUs, and then shrunk it down to a $100 million data center. Now, that $100 million data center in the cloud is shared by 100 companies, which is nothing.
NVIDIA GPUs allow startups to train models at a lower cost. Now you can build a large language model, such as GPT, for about $10 million to $20 million. It's really, really affordable.
The 2022 State of Artificial Intelligence report says that as of December 2022, more than 21,000 open source AI papers used NVIDIA chips.
Most researchers in the State of AI Compute Index use the NVIDIA chip V100, launched in 2017, but the A100 grew rapidly in 2022 to become the third most frequently used chip.
The A100's toughest competition may be its successor, the H100, which was launched in 2022 and has already begun mass production. In fact, Nvidia said Wednesday that H100 revenue surpassed A100 in the quarter ending January.
At present, NVIDIA is riding the AI express train to "money".
Resources:
https://www.reuters.com/technology/tech-giants-ai-like-bing-bard-poses-billion-dollar-search-problem-2023-02-22/
https://www.businessinsider.com/ai-expensive-google-chatbot-bard-may-cost-company-billions-dollars-2023-2
https://www.bloomberg.com/news/articles/2023-02-22/nvidia-gives-strong-forecast-as-ai-helps-fuel-computing-demand?srnd=technology-vp
https://www.reuters.com/technology/nvidia-results-show-its-growing-lead-ai-chip-race-2023-02-23/
https://www.cnbc.com/2023/02/23/nvidias-a100-is-the-10000-chip-powering-the-race-for-ai-.html