The average annual salary per capita is 1 million, and the mobile phone manufacturers have tens of billions of open-book models
The device-side model is becoming a new increment of innovation in the mobile phone industry.
Text: "Chinese Entrepreneur" reporter Zhao Dongshan
Editor|Li Wei
Header image source: Visual China
With an average annual salary of 1 million per capita, what kind of industry has such a favorable treatment?
The answer is the AI model. This is the answer given by the vice president of vivo in an interview with the media such as "Chinese Entrepreneur", he said, "The vivo model now has an annual input cost of 2 billion ~ 3 billion yuan, and the total investment cost has exceeded 20 billion yuan, with half of the talent and data computing power, and the average talent cost is 1 million yuan per person after tax." ”
In the past year, AI large models have swept the entire Internet technology industry, and when the large models have completed the infrastructure construction from 0 to 1, the application of large models based on different scenarios is becoming a new round of competition, and smartphone manufacturers with a huge user base have become the first batch of end-side large models to grab the beach.
From August 2023, China's leading smartphone manufacturers Huawei, Xiaomi, vivo, OPPO, and Honor have successively announced their large-scale model R&D and landing plans.
In August this year, Huawei Hongmeng OS 4 was the first to announce access to the Pangu large model, followed by Xiaomi, which has trained a large language model with a scale of 1.3 billion and 6 billion parameters, and has been applied in some scenarios of Xiaomi's surging OS system and artificial intelligence assistant Xiao Ai.
By November, the pace of large-scale model releases by mobile phone manufacturers became more and more intensive. At the beginning of November, vivo released five large model matrices with three parameter levels of billion, 10 billion, and 100 billion in one go; Subsequently, OPPO officially launched the self-trained Andean large model (AndesGPT), and connected to the latest operating system ColorOS 14, AndesGPT supports models with different parameter scales from one billion to hundreds of billions.
The mentality of mobile phone manufacturers entering the market is not just "focusing on participation", but large-scale investment of real money. In April this year, when Xiaomi set up a large model team, Lei Jun expressed his attitude: "Full support, there is no upper limit on investment." At present, Xiaomi has more than 3,000 AI-related R&D personnel, and Shen Wei, the founder, president and CEO of vivo, who has always been low-key, also told the team, "In the future, AI will become the underlying technology of technological innovation." We must seize this historical opportunity and invest with high standards to create industry-leading technologies and products, and make our historical contribution to the arrival of the intelligent era. ”
There is no doubt that at a time when smartphone shipments are declining, the user replacement cycle is prolonging, and homogenization competition is serious, the end-side large model is becoming a new increment of innovation in the mobile phone industry. But at the same time, how to build a large model with parameters of one billion or ten billion into a palm-sized mobile phone, and what substantial changes it can bring to mobile phone companies, are also facing many challenges.
challenge
If mobile phone companies want to really build a large model into a palm-sized mobile phone, there is a paradox: if the model parameters are too large, the mobile phone terminal will not be able to put in or run at all, but if the model is too small, it may not be able to truly realize intelligent emergence.
In the implementation of the device-side large model, a reality that cannot be ignored is that the data of 1 billion parameters will occupy 1G of memory on the mobile phone, the data of 7 billion parameters will occupy 4G memory, and when the data volume reaches 13 billion parameters, the memory occupation will reach 7G. However, the vast majority of high-end mobile phones on the market now have 12G or 16G running memory, and if the data volume of the large model reaches 13 billion parameters, it means that it will occupy 7G of memory, which will seriously affect the running smoothness of the mobile phone.
Source: Visual China
However, compared with the large model on the cloud, the advantages of the device-side large model are obvious, such as it can fully protect the privacy of the user, and the interaction data between the user and the large model does not need to be uploaded to the cloud during the user's use. At the same time, the response speed of the large model on the device side will be faster, and one of the most extreme cases is that the large model on the device can still be used even when there is no network, while the large model on the cloud cannot be used without the network.
In addition, the invocation of large models on the cloud is costly. "The lowest cost of a large model cloud computing is 1.2~1.5 yuan, if 300 million users use it ten times a day, it means that mobile phone manufacturers will spend more than 10 billion yuan a year. An AI model practitioner told "Chinese Entrepreneur".
Therefore, on the basis of the existing model training level, the mobile phone manufacturers are facing the challenge of talent and technology accumulation, and only by gathering the core technical talents can we better solve this dilemma.
Surrounding told "Chinese Entrepreneurs" that since the establishment of the AI Global Research Institute in 2017, vivo has set up a team of more than 1,000 AI experts, and the annual investment in artificial intelligence is conservatively estimated at 2 billion ~ 3 billion yuan, and it has now exceeded 20 billion yuan, of which the cost of data and computing power accounts for half of the investment in self-developed large models, and the cost of personnel accounts for half.
Xiaomi's layout in artificial intelligence is also relatively early, after AlphaGo came out in 2016, Lei Jun began to invest heavily in AI, starting with the visual team, and then gradually expanded to various fields of AI.
"Xiaomi has more than 3,000 people doing AI-related R&D work, and Xiaomi's AI lab has accumulated technology in vision, acoustic speech, NLP, knowledge graph, machine learning and other directions, and has strong closed-loop capabilities from algorithm pre-research to engineering implementation. We had a human-machine dialogue team before, and we did a dialogue model with 2.8 billion parameters. Wang Bin, director of Xiaomi Group's AI Lab and chief scientist of natural language processing (NLP), said.
In addition to mobile phone manufacturers working hard to lay out large models, upstream chip manufacturers of smartphones are also joining the war.
On October 25, 2023, Qualcomm released the Snapdragon 8 Gen3, a next-generation mobile processing platform. Compared with the previous generation Snapdragon 8 Gen2, the Snapdragon 8 Gen3 not only has a significant increase in GPU and NPU performance, but more importantly, it can run a model with 10 billion parameters on the device side.
Qualcomm focused on improving the AI computing power of its chips on the Snapdragon 8 Gen3, with NPU performance soaring by more than 98%, and in addition to supporting models running up to 10 billion parameters, it can also run 7 billion parameter large language models that generate up to 20 tokens per second (the smallest unit of Chinese for large models). This means that all kinds of virtual assistants and GPT chatbots will be able to run on mobile phones and other terminals in the future.
Falling Ground
China Entrepreneur observes that almost all smartphone manufacturers have adopted a gradual development approach when laying out large models, that is, first developing and training models with small parameter scales, and then developing models with larger parameter scales after wading through all the holes on the road.
Regarding the performance of different levels of models, he told "China Entrepreneur": "According to internal tests, the large model with 7 billion parameters is enough to do a good job of simple document summarization and dismantling functions, but in order to truly achieve 'intelligent emergence', there is still room for improvement in the dismantling ability of the large model task with 7 billion parameters, and 13 billion parameters may be a better choice." ”
In addition to progressive development, up to now, smartphone manufacturers have also divided their layout into two paths: first, the use of lightweight and localized deployment of mobile phone side large models, typical manufacturers such as Xiaomi and Honor, and second, the use of cloud, device collaboration architecture design, the launch of a large model matrix, the deployment of tens of billions of large model training models in the cloud, and the deployment of billion-level large models on the mobile phone side, typical manufacturers such as Huawei, vivo and OPPO.
In April 2023, Wang Bin took the lead, led by Luan Jian, the head of the AI laboratory large model team of the Xiaomi Technical Committee, and a self-developed large model team was officially established within Xiaomi, Lei Jun personally promoted the establishment of the self-developed large model team, and highly participated in the self-development of the Xiaomi large model throughout the process, he will personally check the team's weekly, monthly and even daily reports, and pay attention to the progress of the large model.
Xiaomi's self-developed device-side large model emphasizes the combination with the product and the driving of the scene. "The main breakthrough direction of Xiaomi's large model technology is lightweight and local deployment, and we will not consider it purely from a technical point of view, nor will we aim at the competition. We don't engage in an arms race, and Xiaomi's starting point for making a big model is not to become China's OpenAI. From the very beginning, we considered how to combine the large model with the company's scenario. Wang Bin told "Chinese Entrepreneur".
Unlike Xiaomi's main layout of the device-side model, Huawei, vivo, and OPPO's self-developed large models adopt the path of "cloud and device collaboration", and Huawei's Pangu model, vivo's blue heart model, and OPPO's Andean model all cover more than one billion, 10 billion, and 100 billion parameters.
"Why use matrices to solve this problem? Because today's large models are called language models, and only language and text have real large models, and sound and video do not have large models. Before there is a big breakthrough in the algorithm I can imagine, matrix is the better solution. Surrounding said in an interview.
In addition, it is also said that the matrix model is also a comprehensive result of the impact of user demand and computing power cost. First, the matrix mode allows users to use large models on the cloud while localizing key data on the device side to meet privacy and security requirements, and second, the matrix mode can effectively balance the problem of excessive computing power costs in the cloud.
Have to
Although the mobile phone side of the large model is difficult, but participation is becoming a must for mobile phone manufacturers.
In 2017, the smartphone market ushered in the first decline in history, with a 5% decline in shipments in the Chinese market as a node, announcing that the smartphone industry has entered the stock era. In the third quarter of 2023, global smartphone shipments fell by 8% year-on-year, showing nine consecutive quarters of recession, and also setting a record for the worst shipments in the same period in the past decade.
In the past few years, in the face of increasingly homogeneous competition, mobile phone manufacturers have successively carried out innovation and involution in the fields of fast charging, battery life, full screen, photography, and high refresh rate. At present, the end-to-end model is becoming the entrance to innovation that they are eager to seize. After all, no one wants to miss out on a certain outlet and become the next Nokia.
"AIGC is one of the technological innovations that I am most excited about since I made mobile phones, and it can even be said to be a revolution. It will revolutionize the experience of mobile life. The knowledge of the large model goes beyond the knowledge of the individual, it can understand your language like a person, observe you, learn from you, understand your habits every day, give you the best help, and complete your tasks. When talking about the large model, Liu Zuohu, chief product officer of OPPO, told "Chinese Entrepreneur". In his opinion, the future mobile phone is a super assistant, which is similar to ChatGPT.
Source: Visual China
The large models of mobile phone manufacturers are also gradually landing in their artificial intelligence applications, for example, the Xiaoyi smart assistant of Huawei P60 and Mate60 series mobile phones has been connected to the Pangu large model. At the same time, OPPO's Xiaobu and Xiaomi's Xiaoai have all been connected to their respective large models. In addition, in vivo's latest X100 model, users are clearly provided with the download option function of device-side large models, and users can voluntarily choose whether to download or not, or choose to run on the cloud or locally.
The large model application on the mobile phone is also really improving the user experience. For example, in the past, to find a certain photo in the gallery, you needed to search for keywords very accurately, but in the era of large models, you only need to tell the AI voice assistant "The photo of the Great Wall taken when I traveled to xx before" and other generalized blurred instructions to find the corresponding photo.
At present, the large model can perform simple operations such as language understanding, text creation, and image generation on the device side, but when it comes to ticket booking, APP interaction, equipment control and other operations, it is still necessary to mobilize the capabilities of the cloud large model. But even so, it is a huge possibility for mobile phone manufacturers.
"We are now holding the mentality of 'first is invincible' in "The Art of War" and are investing in high saturation, that is, we don't want to be distanced by excellent business partners, rather than necessarily involution or confrontation and competition on the large model. When talking about the mentality of making a big model, the surroundings told "Chinese Entrepreneurs".
In his view, the importance of the device-side large model is at the same level as communication and chips, and it will become a key way to obtain high-end users in the future. "If you look at the huge improvement of production efficiency in science and technology, I think the large model is also a historical and event-level, and it is inevitable that after doing a good job in the large model, it will be recognized by more high-end users, so we are also investing in high specifications. Surroundings added.
At the moment when domestic mobile phones collectively shouted to hit the high-end, all new innovation opportunities have become new variables of competition. What's more, the battle of the end-side large model has begun.
News Hotline & Submission Email: [email protected]