Meta's high-profile Llama 3 model was officially unveiled this week, with a maximum parameter scale of over 400 billion and a training token scale of 15 trillion that shocked the industry. Compared with GPT-3.5, Llama 3 has a winning rate of more than 60% in human evaluations, and is known as "the strongest open-source model on the surface".
At present, the development of large models is at a critical turning point, and the Morgan Stanley report pointed out that we are ushering in a new era of rapid growth in large model capabilities driven by both software and hardware. However, the unprecedented computing power required for large model training and the sharp increase in development costs are also a huge challenge for tech giants, and it is even more difficult for start-ups or small and medium-sized enterprises to get involved.
In addition, the increase in chip power supply and AI technology barriers also pose barriers to entry in the field of large models, and it is difficult for small and medium-sized enterprises to compete with giants. As a result, Morgan Stanley is bullish on big tech companies such as Google, Meta, Amazon, and Microsoft, which are expected to dominate the development of large models. At the same time, start-ups may be able to find new opportunities by developing smaller models with lower costs.
In the era of large models, how can start-ups or small and medium-sized enterprises grasp the outlet?
In the future, the demand for computing power will grow rapidly, and NVIDIA Blackwell chips may become a key engine?
The Morgan Stanley report revealed that large model technology is developing rapidly, and the demand for computing power has jumped exponentially. Nvidia's Blackwell chips, with their superior performance, are the key engine leading this growth.
The computing power demand of GPT series models is increasing year by year, and GPT-4 has consumed a lot of time and resources, using tens of thousands of GPUs to process huge token data. With the debut of follow-up models such as GPT-5, the demand for computing power will soar further.
Supercomputers have become an important tool for large model training. Morgan Stanley predicts that in the next decade, supercomputers will provide 1,000 times more computing power for large model development than they do today. Blackwell chips help make this leap forward with their powerful performance.
Supercomputers using Blackwell chips can greatly shorten the training time of large models and improve computing efficiency. This change will accelerate the development of large model technology and promote the overall progress of the field of artificial intelligence.
With the continuous evolution of the GPT series model, its computing power demand will continue to grow, accounting for a significant share of NVIDIA's annual chip sales. NVIDIA will increase investment in chip R&D and innovation to meet future computing power needs.
In the future, the growth of computing power will provide strong support for the development of large model technology. As a key engine, NVIDIA's Blackwell chip will promote the improvement of large model training efficiency and accelerate the progress of artificial intelligence. We look forward to more innovative products launched by NVIDIA in the future to lead a new era of computing power growth.
In the era of large models, how can start-ups or small and medium-sized enterprises grasp the outlet?
Tech giants become the biggest winners in the era of big models?
The era of large models is booming, but the development and training behind it are full of challenges, such as capital investment, chip supply, power demand, and software development capabilities. These challenges create a high barrier to entry that puts tech giants in the driver's seat.
Morgan Stanley provides an in-depth look at data center capex for Google, Meta, Amazon, and Microsoft in 2024. The investment in supercomputers is enormous, with $30 billion for just one gigawatt of infrastructure and hundreds of billions on a larger scale.
Projections show that the four hyperscale computing companies will see explosive growth in capital spending over the next two years, which is expected to reach about $155 billion in 2024 and more than $175 billion in 2025. Small businesses can't afford it.
Therefore, Morgan Stanley is bullish on the prospects of these four companies, believing that they will directly benefit from the growth of computing power. With strong capital and technical strength, they can overcome the challenges of large model development and supercomputer training. Institutions have given them an overweight rating, indicating that they will continue to lead in the era of large models and become real winners.
Where are the opportunities for startups?
In the era of large models, although the technology giants are dominant, small companies also have opportunities. Morgan Stanley pointed out that the low cost of developing small models provides development opportunities for start-ups. Small models can play an important role in specific industry areas and achieve significant benefits.
Startups can focus on the needs of the industry, develop models that meet the characteristics, and provide customized solutions. Its flexibility and focus enable SMEs to respond quickly to market changes. In addition, with the popularization of large-scale model technology and the reduction of computing power costs, startups can use advanced technologies to improve their products and services, and cooperate with tech giants to conduct data analysis, prediction, and optimization to enhance their competitiveness.
For example, as an AI quasi-unicorn and a leading enterprise in hyper-automation, TARS, a vertical large model developed by Real Intelligence, ranks among the best in various lists of large models in China, with differentiated advantages such as "effect availability, cost control, customized training, and privatized deployment"
In practice, the agent is able to simulate and surpass manual operations, not only perform repetitive rule-based tasks, but also handle more complex decision-making processes, such as identifying potential risks in the bank loan approval process, interpreting unstructured data, and synchronizing data across systems and platforms.
In the era of large models, how can start-ups or small and medium-sized enterprises grasp the outlet?
As a result, SMEs don't have to be discouraged by the dominance of tech giants. Instead, we should actively look for development opportunities and take advantage of small models to achieve breakthroughs and innovations in specific industries. Only in this way can start-ups gain a foothold in this era full of challenges and opportunities.
With the support of software, what can the future large model do?
While chips and other hardware continue to innovate, the innovation of software architecture has also become the key to promote the leap in the capabilities of large models. Among them, the Tree of Thoughts architecture has attracted much attention.
In December 2023, Google DeepMind joined forces with researchers at Princeton University to unveil this innovative architecture, which is inspired by the workings of human consciousness, specifically "System 2" thinking. Unlike fast, unconscious "System 1" thinking, "System 2" is a deep, long-term cognitive process that requires more thought and planning.
With the introduction of the Tree of Thoughts architecture, large models will be able to more closely resemble the human way of thinking, showing greater creativity, strategic thinking, and the ability to deal with complex and multi-dimensional tasks. This means that the big models of the future will no longer be simple data processors, but will be able to think deeply, strategize, and even solve complex problems like humans have done before.
This change will not only greatly improve the capability boundaries of large models, but also accelerate the popularization and application of AI technology, bringing unprecedented opportunities and challenges to all walks of life. We look forward to the future of this innovative architecture that has the potential to drive the continued development of AI technology."
The cost of computing has dropped dramatically, and the computing power of large models has soared
Morgan Stanley's latest forecast shows that the growth of large model computing power is driving down the cost of computing significantly. The chip upgrade from Nvidia Hopper to Blackwell has reduced computing costs by about 50%. Sam Altman, CEO of OpenAI, emphasized that computing power may become a key resource in the future, and its importance is comparable to that of currency.
With the reduction of costs, large models will be more widely used, promoting the rapid development of AI technology. In addition, the report predicts the construction of a small number of supercomputers, possibly located near nuclear power plants to meet energy demand. In the United States, Morgan Stanley is bullish on Pennsylvania and Illinois due to their large number of nuclear power plants that can provide a stable supply of multiple gigawatts of electricity.
The decline in computing costs and the construction of supercomputers will provide strong support for the soaring computing power of large models and promote the innovative application of AI technology in various fields. We expect this trend to lead to more breakthroughs and advance the development of AI.