What are the barriers to stuffing a large model into a mobile phone?
Author|Tian Siqi
In the wave of intelligent technology, generative artificial intelligence (AI), which has risen rapidly in the past two years, has become a key force in promoting the development of the mobile phone industry. At the MediaTek Dimensity Developer Conference (MDDC 2024) on May 7, Guanzhou Chen, Director, General Manager and Chief Operating Officer of MediaTek, said: "Generative AI has revolutionized the value of end applications, and smart devices are the key carrier for the popularization of generative AI. ”
With the full promotion of chip manufacturers, large model manufacturers, terminal manufacturers and application developers, a picture of tomorrow composed of personalization, immersive experience and seamless interaction is slowly unfolding.
Apple CEO Tim Cook said on May 2 that he expects "the vast majority of devices" to have generative AI capabilities and that he is very optimistic about Apple's opportunities in generative AI. As expected, at the press conference on May 8, Apple characterized the new iPad Pro as an "AI device", and also said in a high-profile manner that the latest M4 is the perfect chip for performing AI tasks;
Samsung also launched the Galaxy S24 series of phones that include generative AI technology in January this year. According to media reports, a quarter of Galaxy S24 buyers said that the AI feature was a key reason why they chose the phone. According to research firm Counterpoint, 2024 will be the first year of generative AI phones.
However, this emerging area of the mobile phone industry still faces many challenges. How will the large model fit smoothly with the phone? How will users directly perceive the new features of generative AI phones? How can industry vendors work with developers and partners to build an intelligent ecosystem and truly improve users' mobile phone experience?
One fundamental question has not been properly addressed: what exactly counts as a generative AI phone?
Based on these challenges, Counterpoint reveals a hint of caution in its optimistic outlook: MediaTek, Counterpoint and ecosystem partners jointly released the "Generative AI Mobile Phone Industry White Paper" on May 7, pointing out that less than 1% of the 1.17 billion mobile phones shipped in 2023 meet Counterpoint's definition of generative AI mobile phones.
But the report also notes that by 2027, the penetration rate of generative AI phones will reach 43%, and the stock size will grow from one million in 2023 to 1.23 billion units in 2027.
Source: Generative AI Mobile Phone Industry White Paper
Obviously, both upstream and downstream of the smartphone industry chain need to actively embrace change. In the process of making the leap towards the goal of 1 billion units, MediaTek, as a mobile phone chip giant, took the lead in proposing a set of systematic Dimensity AI ecological strategies composed of chips, models, and applications, allowing the industry to see the future paradigm of the end-side generative AI ecosystem and the generative AI mobile phone industry.
1. Why do you need generative AI on the device side?
Since the outbreak of ChatGPT, generative AI has been in full swing on the cloud side, with various novel AI applications emerging one after another, and many amazing cases have a wide impact on the industry. However, while professional users are already innovating with generative AI, most users are yet to truly experience the creativity and convenience of generative AI. How to achieve AI inclusiveness has become a key topic for mobile ecosystem pioneers such as MediaTek.
Mobile terminals such as smartphones not only have the largest user base, but also have unique advantages in convenience, interactivity, privacy protection, multi-modal input and output, and situational awareness. As the most popular smart terminal, smartphones will be the primary breakthrough for end-to-end generative AI.
From being able to install apps on your own, to changing a physical keyboard to an oversized touchscreen, mobile phones have changed dramatically in just over two decades. Although generative AI brings new opportunities, it also needs to follow technology trends for a long time and boldly follow up in order to seize market opportunities.
MediaTek saw the inevitable impact of the development of the AI industry on mobile phones earlier. As early as 2015-2016, when AI was widely discussed for the last time, MediaTek began to think about whether the end side also had the opportunity to catch the technological torrent of this era. After the Transformer architecture was born in 2017, MediaTek decided to deploy new algorithms in advance to match new applications that had not yet appeared.
Li Yanji, deputy general manager of MediaTek's wireless communication division, said in a recent exchange with "Jiazi Lightyear" and other institutions that this is MediaTek's determination to invest in technology, and it will look at the future development of AI with a relatively open mind.
At the same time, end-to-end generative AI does have several advantages. Li Junnan, director of technical planning of MediaTek's wireless communication division, believes that this is reflected in "privacy protection" and "multimodality" respectively: "Many things of users do not want to go to the cloud, but even if there is no network on the device side, the large model can quickly answer and interact" "Users bring their mobile phones with them every day, it has natural multimodal advantages, and has the most natural interaction methods such as sound and image".
As the convergence deepens, Counterpoint believes that generative AI technology will give birth to one or more AI agents on smartphones, becoming the application portal for each user. Therefore, generative AI phones can be defined as the use of large-scale, pre-trained generative AI models to achieve multimodal content generation, situational awareness, and increasingly human-like capabilities.
Obviously, the generative AI mobile phone led by MediaTek is not only a technological revolution, but also a profound reshaping of the human-computer interaction model, indicating that smartphones will move from passive response to a new wave of active service.
However, is it better to get into the larger model of the phone?
Larger models usually mean higher hardware costs, which also affects the phone's battery life. Considering the limitations of computing power, memory, and heat generation, the device side needs to adopt a different development strategy from the cloud side, rather than blindly pursuing a larger parameter scale.
Chen Yiqiang, deputy general manager of MediaTek's wireless communication business unit, said: "The 2 billion parameter model a few months ago is much smarter than the 7 billion parameter model half a year ago. If we move such a model to a mobile phone, it may be another level higher than the entire AI capability half a year ago. He said that at the beginning of the new era, the criteria for evaluating generative AI phones in terms of computing power or specific functions may not be very clear. The standard was set, but half a year later, it turned out that the original standard was so low because the progress of generative AI was too fast.
Therefore, MediaTek focuses more on qualitative analysis, and is willing to cooperate with all ecological manufacturers to make the concept of generative AI mobile phones truly land, through a series of advanced hardware, software and tools, to explore a more suitable technical route for the end side, and turn the experience concept brought by generative AI mobile phones into reality, so that AI can benefit users.
2. Build an end-to-end AI ecosystem with the Dimensity AI strategy of chips, models, and applications
MediaTek's preparations have been unveiled at the MediaTek Dimensity Developer Conference (MDDC 2024) on May 7. At the conference, MediaTek unveiled the latest Dimensity 9300+ flagship 5G generative AI mobile chip. The chip adopts an all-large-core CPU architecture, and is the first to support AI speculative decoding acceleration technology on the device side, and at the same time supports Dimensity AI LoRA Fusion 2.0 technology to provide a more efficient and personalized generative AI experience.
In addition, the Dimensity 9300+ supports cutting-edge and mainstream generative AI models, providing users with end-to-end generative AI multi-modal innovation experiences such as text, images, and music. It also supports the AI framework ExecuTorch, which accelerates the development of generative AI applications on the device side.
It is worth mentioning that during MDDC 2024, in order to allow developers to develop generative AI applications on devices more efficiently and conveniently, and accelerate the construction of a generative AI application ecosystem covering all scenarios, MediaTek also cooperated with Alibaba Cloud, Baichuan Intelligence, Transsion, Zero One Everything, OPPO, Honor, vivo, and Xiaomi to launch the "Dimensity AI Pioneer Program" to help developers create innovative user experiences on terminal devices equipped with Dimensity chips.
For the specific implementation plan, Li Yanji said that the pioneer plan consists of two parts, the first link is to cooperate with some models and OEMs; The second is to explore with all app developers to develop new applications on the basis of hardware.
Why did you choose these large-scale model manufacturers mentioned above as partners? In response to the questions raised by Jiazi Lightyear at MDDC 2024, Zhang Li, Senior Director of Ecosystem Development of MediaTek's Wireless Communications Division, responded that on the basis of ecological openness, there will indeed be some measurements and considerations for large models:
First, we will look at the main business of the large model or whether its scene is to B or to C. We still believe that generative AI will soon enter the consumer market, and a very important aspect of our business is also the mobile phone business, so for large model manufacturers, we will pay more attention to large models that do consumer business or have their own apps or their own C-end operation capabilities.
Second, we will consider the impact of open source. Because open source can embrace the entire ecosystem more openly, for us, we will be able to cooperate more smoothly with open source models.
The most important criterion, the selection of large models, is essentially determined by whether these models can exert their capabilities on the device side to create an innovative user experience.
At the same time, there are two main types of problems faced by large model manufacturers and developers for deploying large models on the device side: first, whether the operation is efficient, how much power is consumed, or whether the speed is fast enough; Second, the memory usage may be too high. However, MediaTek provides a series of solutions that allow large model manufacturers and application developers to enjoy a convenient and fast development method that can be implemented in practice.
For example, MediaTek's "Dimensity AI Development Kit" includes four modules: fast and efficient GenAI best practices, GenAI Model Hub covering the world's mainstream large models, GenAI optimization technology that efficiently improves performance, and Neuron Studio one-stop visual development environment, providing developers with a "fast, complete, strong, and easy" professional development experience and empowering the entire process of terminal generative AI application development.
At present, the Dimensity AI development kit has covered smart terminal devices such as smartphones, smart cars, Internet of Things, and personal computers, empowering the development of generative AI applications in all scenarios.
"It is a big challenge to install such a large model, such as 7B and 13B models, into such a small device as a mobile phone, so it is necessary to compress it through Neuron Studio, a set of tools, to make the best and smallest network structure, and provide developers with better development efficiency." ”
At the conference, Neuron Studio has created a set of highly integrated, one-stop, and visual AI application development tools for developers, in which developers can complete development work such as model modification, compilation, platform trial operation, and one-click terminal deployment, while the visual interface can also quickly complete bug locating, running performance analysis, etc., making it easier and more efficient to develop end-side generative AI applications.
At the same time, the GenAI best practices in the Dimensity AI Development Kit have accelerated the deployment of large models from weeks to one day through model quantization, model compilation, and model inference technologies.
In addition, GenAI Model Hub can adapt to the industry's cutting-edge mainstream large models, providing developers with rich large model resources for efficiently building generative AI applications. The Dimensity AI development kit also supports advanced GenAI optimization technologies such as speculative decoding acceleration and LoRA Fusion.
Zhang Li said that respecting the needs of developers, making all MediaTek services closer to the needs of developers, and ultimately facing the user experience, is an important way to win.
Zhang Li also emphasized that generative AI and chips will be strongly correlated in the future. Wang Xiaochuan said that he never saw a chip company when he was doing Sogou, and he had to see it after making a large model. When developers achieve deep technological innovation in generative AI, they cannot do without the support of chip companies. Everyone is at a different level to ensure the user experience of future innovation.
In the context of generative AI accelerating the transformation of the mobile ecosystem, MediaTek has in-depth cooperation with large model manufacturers, and provides developers with complete and efficient solutions, and constantly makes improvements according to the needs of developers, showing MediaTek's attention to and embrace the entire developer ecosystem, MediaTek's determination to lead the mobile ecosystem to seize the opportunities of generative AI, build a new application ecosystem and create new value for users is obvious.
3. Be one step ahead and revolutionize the mobile app ecosystem with AI and games
In the extremely involuted mobile phone industry, App developers once fell into a situation where they were at a loss. In the established rules, the competition relationship and traffic have formed a stable business format, and there are not many opportunities for innovation to follow.
Generative AI has brought new opportunities, and the entire ecosystem is accelerating its linkage. Chen Yiqiang told "Jiazi Lightyear" and other institutions that it is a long link from the bottom layer of the chip to the system, model and application. Therefore, MediaTek attaches great importance to cooperation with games and some leading manufacturers:
The game is the best implementation scenario for AI, because it is an independent and complete scene, which can provide more room for trial and error, and it is also an entertainment content with rich imagination, and now AI can fully meet this need.
At MDDC 2024, MediaTek also announced a comprehensive upgrade of Dimensity gaming technology, namely the Star Speed Engine. With the help of the Adaptive Technology Software Development Kit and hardware ray tracing technology, Starspeed Engine can help game developers create an all-inclusive experience for users with more realistic and smooth graphics, faster touch and network response speeds, and longer battery life through key technologies such as accurate performance, power consumption management, ray tracing effect optimization, and network quality monitoring.
"Previously, we were thinking about the Dimensity gaming experience in the direction of continuous improvement of chip capabilities. And now the most fundamental change is that we are starting to embrace more ecological partners. We found that game developers didn't know that the original chip could provide so many capabilities," Chen said. Therefore, MediaTek is carefully listening to the needs of developers and important game manufacturers for chip capabilities, facing the technology to the entire development ecosystem, and working with developers to develop better games.
Of course, in other subdivisions, MediaTek will also give nutrients and soil, so that the majority of developers can find their own innovative fields through generative AI, and gradually cultivate future popular apps.
At present, MediaTek has worked with dozens of ecosystem partners such as Alibaba Cloud Tongyi Qianwen, Cocos, Honor of Kings, Huya Live, Kugou Music, Meitu, WeSing, RWKV, Soul, Tencent AI Lab, Xiaohongshu and other dozens of ecosystem partners to create many innovative end-side generative AI applications.
"Jiazi Lightyear" saw at MDDC 2024 that a variety of video GAI special effects have been implemented through the Dimensity mobile platform. End-to-end generative AI applications such as AI singing, AI composition, and lyrics are also becoming more and more mature, and these applications will allow users to deeply feel the new interactive experience brought by generative AI.
Zhang Li said: "If a technology itself can bring an innovative user experience, there is no need to worry about whether there will be explosive products, and the result will definitely appear." ”
Indeed, MediaTek uses the Dimensity mobile AI ecosystem to provide soil for end-side AI technology and application innovation, laying the cornerstone for the emergence of explosive applications. It's reasonable to expect generative AI phones to revolutionize the user experience.
As for how to benefit more smartphone users, Li Yanji pointed out that the Dimensity 8300 at the end of last year was the first mobile platform that can support generative AI with a light flagship, which has also landed on Redmi's mobile phones. Not only the flagship, MediaTek will also continue to provide better AIGC capabilities in the light flagship grade:
"We want to empower developers through the improvement of flagship and light flagship capabilities, as well as the tools we provide, and I believe that one day in the future, it will be popularized to more terminal applications." Li Yanji said.
Thanks to the empowerment of AI models and the comprehensive Dimensity AI ecosystem strategy, the mobile ecosystem of smart devices and smartphones has entered a new stage of development. This means that MediaTek not only promotes the acceleration of generative AI technology on the device side, but also provides strong support for the innovation process of the entire smartphone industry. At the same time, MediaTek and ecosystem partners jointly define the "generative AI phone" for the market to plan a clear innovation route.
It is particularly noteworthy that as the leader and dominant player of the mobile ecosystem, MediaTek Dimensity's system layout for developer needs and user experience indicates that generative AI phones will be more intelligent and personalized in the future. It is believed that the world's 4 billion smartphone users will soon be able to open a new era of intelligent life.
Resources:
"Generative AI Mobile Phone Industry White Paper," Counterpoint
《Consumers embracing AI on their smartphones》，《Mobile Europe》
(Cover image and unexplained image from MediaTek Dimensity Developer Conference (MDDC 2024))