None from 2023
量子位 | 公众号 QbitAI
4 trillion degrees Celsius (345 MeV), created by the Brookhaven National Laboratory in New York, USA, in 2010 when using the relativistic heavy ion collider to conduct a gold ion collision experiment, the highest temperature that humans can produce so far is 260,000 times the temperature of the sun's core.
If there's one technology that can reach such unprecedented "heat" in 2023, it's undoubtedly generative large language models.
However, unlike the fleeting 4 trillion degrees of high temperature that is less than one billionth of a second, the far-reaching impact of large models on all walks of life will be like a "spring thunderstorm" and a "moisturizing thing silently" in 2023. Therefore, if you want to use two key words to describe the large model industry in 2023, in addition to "hot", there is also "volume".
In November 2022, ChatGPT was born, and after only two months, ChatGPT successfully exceeded 100 million monthly active users, and became the fastest consumer application in history to exceed 100 million monthly active users. The fledgling ChatGPT is like an all-round warrior, able to chat, write code, write papers... While people lamented the power of this conversation app, the large model behind it, GPT, was truly pushed into the eyes of all the public for the first time.
Modern AI technology mainly simulates the learning and Xi process of the human brain by building deep neural networks, and "precipitates" AI models by learning a large amount Xi of data to complete specific tasks such as image classification, object detection, machine translation, and language understanding. But this time, the large model is different, and the ultra-large-scale data volume, computing power, and neural network scale make the model produce "intelligent emergence".
At present, the most popular explanation for "intelligent emergence" may come from Baidu founder Robin Li, who believes that artificial intelligence in the past was to teach machines whatever skills they wanted. What has been taught, it is possible, and what has not been taught, it will not. After the emergence of "intelligence emergence" of large models, it is possible to use skills that have not been taught before.
In 2023, research and innovation in the field of large models will "fly together" at home and abroad. Global technology giants, many start-ups, and academic institutions have joined this wave of technology. According to rough statistics, hundreds of large models have been released around the world, which can be said to have "rolled" a general technology to the sky in one year.
However, in order to plant the "flower" of large models in thousands of industries, in addition to the model itself, it is also necessary to upgrade the cloud computing infrastructure for large models, support the corresponding platforms and engineering capabilities, and support new paradigm development tools for upper-level applications.
Volume model: Hundreds of basic large models have been released around the world, and 2024 will enter the large-scale elimination competition
As a global leader in large-scale model technology research, OpenAI is backed by Microsoft, the "golden father", and in March, September and November 2023, it has successively launched GPT-4, GPT-4V, and GPT-4 Turbo.
But in November, OpenAI staged a co-founder and CEO, Sam Altman, who was lightning fired by the board of directors, and after joining Microsoft and finally returning to the "Gong Dou" farce, many people also had some concerns about OpenAI's future prospects.
As OpenAI's strongest competitor, Anthropic was founded by former OpenAI executives.
In March and July '23, Anthropic successively released its large-scale model products Claude and Claude 2, and launched a conversational robot application that directly competes with ChatGPT, emphasizing the creation of "safe and responsible AI". It is worth mentioning that Claude 2 supported 100k context windows when it was first launched, and upgraded to version 2.1 in November to support 200k "super large cup" context windows, which crushed GPT-4 and GPT-4 Turbo respectively. Anthropic's strong performance also attracted $4 billion in new investments from Amazon and $2 billion from Google in the second half of '23.
If OpenAI gave GPT a soul, then this "shell" can be said to have been given by Google in the early years.
As the initiator of the Transformer architecture, Google is not far behind in 2023, launching phenomenal large models such as PaLM 2 and Gemini, while AWS, a cloud computing giant that has been silent in the field of large models for a long time, was not revealed to be training a new internal codename "Olympus" until the end of the year after releasing the Titian large model in April In addition, the UAE's Institute of Technology Innovation (TII) and Meta are making efforts to open source, and TII's newly released Falcon 180B surpasses Meta's Llama 2 and becomes the strongest open source model to date.
Returning to the domestic market, the first company to develop a large model is Baidu.
In March 2023, Baidu took the lead in launching a generative large language model, Wenxin Yiyan, filling the gap in this field in China, and within four months of its release, it iterated to version 3.5 at a high speed, which increased the training speed by 2 times, the inference speed by 30 times, and the cumulative effect of the model increased by more than 50% compared with version 3.0. In the first truly authoritative evaluation (IDC's "AI Large Model Technical Capability Evaluation Report, 2023"), the Wenxin large model surpassed GPT-3.5 and won the first place in the performance of large models in China. In October, Robin Li announced the official release of Wenxin Model 4.0 and made a bold statement that "the comprehensive ability of Wenxin Model 4.0 is not inferior to GPT-4".
In addition, Alibaba and Tencent released their own large models in the first and second half of the year: Tongyi Qianwen and Mixed Yuan. With a number of startups such as Baichuan Intelligence, Zhipu AI, and Zero One Everything joining the basic large-scale model melee, the domestic large-scale model market has completely entered the "Warring States" era.
Volume computing power: large models take the lead in reconstructing cloud computing, and intelligent computing will begin to fight for "cost performance" in 2024
Large models require a huge amount of computing resources to support huge systems, training, and inference tasks.
It is not difficult to see from the composition of the world's mainstream large model players, such as AWS, Microsoft, Google, Baidu, Alibaba, etc., are themselves cloud computing vendors with sufficient computing power reserves. Although startups such as OpenAI, Anthropic, Zhipu AI, and Baichuan Intelligence do not have cloud service capabilities, they also need to rely on cloud computing vendors to achieve iterative upgrades of models.
Revenue of $18.12 billion, up 34% sequentially and 206% year-over-year, and data center revenue of $14.51 billion, up 41% sequentially and 279% year-over-year, are Nvidia's third-quarter earnings report. "Emerging from the company's strong growth, we can see that industries are undergoing a platform transformation from general-purpose computing to accelerated computing and generative AI," said founder and CEO Jensen Huang. ”
Cloud computing vendors have a huge advantage that startups cannot match in the wave of large-scale model development. Recently, Nvidia's GPU shipment calculation chart released by Omdia Research has become popular on the Internet, which to a certain extent reflects the anxiety of cloud computing manufacturers about AI computing power. But is it enough to buy and buy the computing power of a large model?
In fact, cloud computing vendors generally choose to walk on multiple legs, in addition to hoarding GPUs, based on their own understanding of large model technology, to create exclusive DSA (Domain Specific Architecture) architecture chips for large model training and inference scenarios, which can not only dilute the cost after large-scale use, but also avoid being tied to a single GPU vendor in the future. For example, the Trainium and Inferentia series chips built by AWS, Microsoft's Maia, Huawei's Ascend, Baidu's Kunlun chip, etc.
For example, in addition to purchasing NVIDIA GPUs in large quantities, Microsoft will also vigorously introduce heterogeneous computing power such as AMD Instinct MI300X, and vigorously optimize the intelligent computing cluster based on the self-developed chip Maia.
In China, cloud computing manufacturers led by Baidu have also invested a lot of energy in the field of intelligent computing. For example, at the end of the year, Baidu released two AI computing instances based on its self-developed Kunlun core and Huawei's Ascend, upgraded the AI heterogeneous computing platform Baige 3.0, and the effective training time of 10,000 card clusters accounted for 98%, and it was compatible with a number of mainstream AI chips at home and abroad.
Volume tool: large model from "rough room" to "fine decoration", 2024 continue to roll "platform supporting"
In addition to the "hard knots" of big data and large computing power, the breakthrough of large model technology is often overlooked behind the accumulation of platform and engineering, which are also the key elements for customers to use large models well.
Startups generally focus on developing large models themselves, and many have chosen the open source route. Although open source has better flexibility, in the era of large models, in addition to the expensive cost of computing power, the lack of supporting tools will also generate extremely high hidden costs, and put forward extremely high requirements for customers' AI technology reserves and secondary development capabilities. For users, the large model should not be a "rough house", nor can it be without a "property".
Different from the development paradigm of small models in the traditional deep learning Xi era, large models need to first have a new and complete tool chain to support the whole process from data management to model retraining, fine-tuning, and evaluation. On a global scale, the first to launch such platforms is neither OpenAI, nor overseas giants such as Microsoft, AWS, and Google, but Baidu.
The large model itself and the supporting tools must go hand in hand, otherwise the large model will land in thousands of industries, and the model manufacturers will be able to do customization door-to-door?
In March 2023, when Baidu launched the Wenxin Yiyan model, Robin Li told that the bigger story of Wenxin Yiyan was in cloud computing. Just 10 days later, Baidu Intelligent Cloud revealed the answer and launched the world's first enterprise-level one-stop large model platform Qianfan, announcing that Baidu can not only make the best large model in China, but also help others make large models. At an event in May, Baidu engineers also demonstrated for the first time in China how to fine-tune the whole process of the industry's exclusive large model based on a one-stop platform.
Subsequently, Zhou Jingren, CTO of Alibaba Cloud, released the Alibaba Cloud Bailian large model service platform at the Apsara Conference at the end of October, Microsoft released its own large model service platform Azure AI Studio in mid-November, and AWS updated the Amazon Bedrock service functions at the end of November, adding new functions such as model fine-tuning and pre-training based on Amazon Titan large models.
Volume applications: The application development paradigm has been completely overturned, and AI-native applications will emerge in 2024
On top of the model, how to help users develop large-scale model applications is also a problem that must be solved. In the final analysis, the large model is a new technology, which does not bring value in itself, and what really creates great value is the application of the upper layer of the model. In this regard, the thinking of major manufacturers is different.
OpenAI and Microsoft have chosen a relatively closed strategy to strengthen their own application ecosystems. For example, at the first developer conference held in November 23, OpenAI continuously released GPTs and AssistantAPI, two killer features for GPT native application development, which also sounded the alarm for AI Agent startups and large model middleware manufacturers that have been tending to their own; Microsoft launched Copilot Studio to help users build intelligent assistants based on Microsoft 365 family buckets.
Other manufacturers have chosen a more developed approach, but few have made efforts to help users develop large-scale model applications, except for Baidu. In the second half of 2023, Baidu released AppBuilder, an AI native application development workbench, for large-scale model application development, which encapsulates and templates frameworks such as atomic components, RAG (Retrieval and Generative Enhancement), and Agent commonly used in the development of large-scale model applications, and opens up two product forms, low-code and code-state, to meet the needs of different developers. It is worth mentioning that Baidu does not tightly couple application development with its own Wenxin model, but presets dozens of mainstream open source models that have been enhanced in addition to the Wenxin model, giving developers more choices.
Different from model communities such as HuggingFace and Moda, which are more developer-oriented, Baidu pays more attention to the construction of applications and industrial ecology, and the models adjusted from the large model platform directly lead to the application workbench, and the applications developed from the application workbench can be put on the AI native application market, forming a closed loop from technology to market and then to business.
In terms of large-scale model landing in the industry, Huawei shouted that the Pangu model "does not write poetry, only does things", and then Baidu Intelligent Cloud reconstructs the four major industry solutions of government affairs, finance, industry, and transportation based on the large model, providing a full-link support system for industry ISVs. We have reason to expect that in terms of large-scale model technology landing industry, China's speed will once again shock the world.
Looking back at 2023, the large model, the "chosen son" of the science and technology industry, has broken through the circle and become a hot spot in the whole society, but it is still a "child" after all, and it will inevitably make people feel the gap under the eager expectation. For ordinary people and people, the "dawn of intelligence" led by large models is shining into reality, but for technology companies that are joining the wave of large models, there is still a lot of work to be done.
Sam Altman wrote down OpenAI's 2024 development list on Christmas Eve. Who will be next?
— END —
QbitAI · Headline number signed
Follow us and be the first to know about cutting-edge technology trends