No tension, vivo "leaked" the ability of the big model in advance

Ming Min Xiao Xiao is from the Cave Fei Temple

Qubits | Official account QbitAI

The highest-grossing Android phone is actually going to be implemented?!

Or release can be used that - the new version of the mobile phone system is directly carried on, not the whole virtual.

You know, although domestic large models are blooming, the "hundred model war" on the mobile phone side can be said to have just entered the warm-up stage.

There are many self-developed large model mobile phone manufacturers, but there are almost no mobile phone systems that are really installed in the mobile phone system.

What made this big news was the blue factory vivo, which has been the first in domestic market for two consecutive years.

Some time ago, the news of vivo quietly self-developed mobile phone AI large model spread wildly, and now the boots have landed, and the official announcement is scheduled for the vivo developer conference on November 1, and the new version of the system OriginOS 4 is on the machine.

From the news of the spoiler of the head of vivo, three main points can be glimpsed:

Release 5 large models in one go

Large models are embedded in mobile phones, and assistants can also draw

7 billion version of the large model, open to the industry

Around the vice president of vivo and the president of the Global AI Research Institute, he also "leaked" the capabilities of the big model on Weibo two weeks in advance:

After a set of coherent operations, netizens were excited first.

There are also netizens who are already fantasizing about the scene of a sharp decrease in workload.

So, what signals did this vivo official announcement of the big model release? Specific to the mobile phone, what functional changes will it bring?

Vivo big model, which signals have spoilers?

Let's take a look at the exposure information of the vivo large model.

vivo's new version of self-developed large model is named vivoLM.

There are 5 versions of vivoLM to be released this time, and the parameters are divided into three levels: billion (1B/7B), 10 billion (66B) and 100 billion (130B/175B).

Whether it is a 7 billion model version or a version with a larger number of parameters, vivoLM has already "brushed" a wave in advance on the two major Chinese model evaluation lists C-Eval and CMMLU.

On the C-Eval list, vivo LM achieved an average score of 82.3 points, especially in STEM and humanities;

On the CMMLU list, whether it is Five-shot (only 5 sample examples) or Zero-shot (0 sample examples), the vivo LM-7B version occupies the TOP 1, and is also "riding the dust" in the humanities.

It is worth mentioning that the vivoLM-7B, that is, the 7 billion version of the large model that topped the CMMLU list, is the version that vivoLM will open to the public.

Of course, no matter which version, the purpose of vivoLM's release is very clear -

Put into the latest version of vivo's mobile phone system OriginOS 4, as a mobile assistant to sense user needs, identify intentions and assist decision-making.

But as we all know, compared with other application fields, the mobile phone end is actually one of the most difficult scenarios for large models to land.

On the one hand, limited by volume and power consumption, the computing power on the mobile phone side is "pitiful" compared with the cloud computing power.

The large model known as "large", if directly deployed on the end side, is often difficult to achieve better use results, even if it can run, the inference speed is not as expected, according to the surrounding words:

If you do the input method word recommendation, 2 seconds to come up with a word.

But shrinking the size of the model will definitely reduce the effect.

On the other hand, if large models are directly uploaded to the cloud for online use, the advantages of end-side deployment will be lost.

For example, the large model could have personalized the mobile assistant on the mobile phone based on user information, and ensured that the information was not uploaded to the cloud; However, if the large model is loaded in the cloud, personal information will have to be uploaded through the network, and privacy security cannot be guaranteed.

In addition to this, the cost of running large models in the cloud is very high. The surrounding introduction said:

Vivo has 300 million Chinese mainland users, if you use it 10 times a day, the cost of computing a day is about 30 million yuan, and it costs about 9-10 billion yuan a year.

How exactly does vivo deploy large models to mobile phones?

From the parameter settings of vivoLM's large model that have been spoiled, as well as some existing large model operation studies, it may be possible to speculate some of the "techniques" used.

One of the skills lies in the parameter design of large models, and different parameter magnitudes are used to deal with different tasks.

Among them, the smallest large model, including models with 1 billion and 7 billion parameters, can be directly deployed on the end side to ensure that power consumption is not high.

When it comes to personalized tasks such as user information, you can use such large models to complete them, such as adding memo content to the calendar with one click and setting alarm reminders.

Larger models, such as those with 66 billion, 130 billion, and 175 billion parameters, decide the call situation according to the difficulty of the task.

After all, from the current accepted statement in the academic circles, the magnitude of parameters required for the "emergence of intelligence" of large models is tens of billions of dollars.

For example, using oversized models such as GPT-4 to summarize the behavior of e-mail has been ridiculed as "opening a Lamborghini to deliver pizza". (Manual dog head)

Therefore, when encountering a task with a "low difficulty coefficient", you can switch to a smaller model to carry it out, and more complex tasks such as higher requirements for context length and output effect, and then call a large model with hundreds of billions of parameters to complete.

The second trick lies in the operation mode of the large model, which does not rely on a single computing power, but takes into account the speed and experience of cloud collaboration.

At present, hundreds of billions of large models are still difficult to deploy to mobile phones, and even if they can be deployed, the running speed and power consumption are unacceptable.

At least from the current research in the industry, it is not impossible to run a large model by means of device-cloud collaboration.

Taking the simultaneous launch of speculative sampling/decoding by Google and DeepMind as an example, this technology can improve the inference speed of large models while ensuring the generation effect.

Specifically, it combines a large model and a smaller large model (draft model) to solve the "memory limitation" problem when inference about large models.

However, this smaller model is not "casual to find", it must be "matched" with the larger model, such as the interface should be unified, the probability distribution should be close, etc.

If the five large models of vivoLM are "matched" with each other, similar technologies can be used to achieve the effect of device-cloud collaboration: the large model is calculated in the cloud, and the smaller model is run on the end side, which can save considerable inference costs.

This set of large and small combinations fist, equipped with a large model of vivo mobile phone new system will have what new features, seems to be able to take a peek at the leopard.

In terms of usage, vivoLM Visual Inspection will use the form of voice assistant as the entrance, as the "intelligent assistant" of the whole machine on call;

In terms of specific functions, it may be mainly divided into three categories:

Generate tasks, such as intelligent email composing and AI avatar generation.

Complex task scheduling, such as one-click summary of call content and setting specific usage scenarios.

Intent understanding, such as tailoring travel to ambiguous needs.

As large models continue to "evolve", richer features will emerge. Around think:

The application of large models on mobile phones has two application stages: auxiliary intelligence and active intelligence. In the active intelligence stage, there is also a relatively high-level presentation.

So, is Vivo's new system equipped with a large model worth looking forward to?

Why is it worth looking forward to?

The most direct reason is, of course, that playing with mobile phones can be cooler.

And this also pushes more people into the latest wave of science and technology, embraces big models, and truly feels the transformation of human-computer interaction under the influence of AI.

The deeper reason is that vivo as a head mobile phone manufacturer self-developed large model, itself is expected.

First, vivo's self-developed AI model must be user-oriented.

It has a clear application landing scenario - mobile phone; Clear goal – to reinvent the operating system with AI.

This makes vivo's self-developed large model will be more compatible with the mobile phone operating system, thus stimulating more potential of large models in the end-side application.

Second, vivo itself has a solid foundation in self-research, and has been deeply engaged in the field of AI for many years.

Now the launch of the self-developed large model is not only another self-proof of R&D strength, but also from this achievement, it can peek into Vivo's understanding of the latest technology trends.

Vivo's AI strategy first began in 2017, when Shen Wei, founder and president of vivo, proposed at the time:

Artificial intelligence is the lowest level of all technological innovations, and we should all be "oversaturated" for a long time to come.

The Vivo Artificial Intelligence Global Research Institute also began preparations this year and was officially established in 2018.

As a result, vivo has also become one of the first Chinese mobile phone companies to set up a research institute specializing in artificial intelligence.

The data shows that since its establishment, vivo AI Research Institute has maintained a research and development team of thousands of people. By 2021, vivo has more than 600 people focused on developing large models.

Since 2018, vivo has published more than 70 high-level papers in top journals, and continues to transform top academic achievements into algorithms and engineering applications.

At the same time, the Atlas Research Institute was also established, and through the professional team building of the knowledge graph, 2800TB of data has been accumulated.

△Vice President of vivo and President of Global AI Research Institute

In addition to AI, vivo's self-research strength in chips, system software, and image algorithms is also good.

Since 2021, vivo self-developed image chips have been launched for three generations.

The latest generation V3 uses the 6nm process to achieve the first 4K cinematic portrait video and 4K post-editing on Android phones.

Moreover, it can also achieve the effect of turning on a large number of self-developed image algorithms at the same time.

For example, AI noise reduction, MEMC interpolation algorithm, HDR technology, and image zoom algorithm, V3 can handle it simultaneously and keep the delay within the user's acceptable range.

In terms of system software, vivo put forward its own understanding on the Android system: OriginOS, which makes the system interaction and design more humane.

For example, atomic notifications and atomic components, which have many likes, are a good example of OriginOS's more intuitive interaction logic, and the component functions can be directly used through clicks, touches, and swipes.

The sidebar in OriginOS 3 has scene recognition capabilities that match the desired application in the sidebar according to the interface the user is browsing.

In addition, vivo also injected many innovative technologies into OriginOS, bringing a smoother and smoother operation experience from the bottom of the system.

The intelligent computing hub of OriginOS is based on the innovation of the upper, middle and lower layers, and the bottom layer realizes breakthroughs in basic capabilities, optimizes resource scheduling in the middle layer, and upgrades the visual and control perception of the upper layer.

It also self-developed "unfair scheduling", which can realize the priority scheduling of system resources to the foreground, suspend the background demand, and improve the smoothness of operation.

In terms of more cutting-edge technology exploration, vivo's layout is also quite advanced.

For example, vivo has launched 6G projects since 2019, released a series of 6G white papers in October 2020, and displayed a series of 6G core technology prototypes in 2022. Including wireless respiratory monitoring and target ranging speed measurement integrated with communication perception, extremely low-power communication based on backscatter, and four prototypes of AI communication.

Moreover, these R&D investments in 6G are also claimed to be "no budget cap, as much as you want." ”

△Communication perception integration - target ranging speed measurement prototype

In summary, vivo's self-developed large model is indeed worth looking forward to.

Not only because it is another head mobile phone manufacturer that quickly follows the trend of large models, but also because vivo has a deep understanding of mobile phone scenarios, and also has considerable technical reserves and self-research capabilities.

Its large model answer sheet can directly affect ordinary users, but also bring a new understanding and reference to the industry, and even contribute to promoting the development of large model trends to the end side.

What does vivo's self-developed AI big model mean?

First of all, for ordinary people, a mobile phone operating system with a large model base is almost the most convenient entrance to embrace a large model.

With the latest wave of technology trends sweeping in, who wouldn't want to try it fast, keep up with trends, or even change their lives with technology?

But there is still a certain threshold for the use of large models. And for many people, how and where to use the latest technology is still a question.

The mobile phone can just become a platform to connect large models and ordinary people, and directly provides use scenarios, and the operation threshold is not high.

As a result, the underlying logic of a new generation of human-computer interaction has unconsciously penetrated into the public group.

Secondly, for the industry, vivo's entry into the big model is not unexpected, but an inevitable mission.

After ChatGPT brought up the industry trend, people's imagination of super AI assistants became more concrete, and Iron Man's Jarvis has been repeatedly discussed in the past period.

Back to reality, isn't the closest to this role the voice assistant in the mobile phone? Therefore, for the mobile phone industry, injecting large model capabilities into voice assistants is almost a natural thing.

But that doesn't mean it's simple.

How to make ordinary users deeply feel the ability of large models? How to clearly distinguish it from the previous generation of voice assistants? How to achieve real convenience and efficiency? Behind these vague requirements, a clear action goal and specific technology are needed to support it.

Who has the capacity to solve these problems?

Of course, it is a manufacturer that understands both AI and mobile phones, and can reach the majority of ordinary consumers, vivo is one of the names that will come to mind.

Canalys reported that in the second quarter of this year, vivo topped the domestic smartphone market with an 18% market share.

As a leading manufacturer, vivo has more thorough and comprehensive access to large model capabilities, which is not only a major upgrade of its own products, but also a positive promotion of the development of the industry.

The trend of large models has brought new growth points to the current mature mobile phone industry, how to grasp the trend and quickly follow up, but also depends on the specific actions of each manufacturer.

It is not difficult to see from the latest trends of vivo:

First, vivo affirmed the trend of big models and decided to enter the game;

Second, vivo believes that this will be an industry-wide action, so while releasing large models, they must also contribute the capabilities of 7 billion parameter versions of the model, and integrate and contribute the ability of training, tuning, and hot plugging of data in various industries.

Going further, vivo brings its own understanding to the industry.

Surrounding previously revealed in interviews:

Vivo looks at the conflicts and problems that may occur in the industry from the perspective of 3-5 years: if the large model application wants to solve the cost problem, it must be at the upper end, and if it is to be on the upper end, it must solve the problem of system minimum privilege and computing power and the most important memory resources.

In addition, to solve the problem of privatization and self-protection of core competitiveness, vivo will come up with solutions in the future and contribute to the industry.

And through such understanding and planning, it is not difficult to draw a conclusion:

Vivo's entry into the big model trend is to create new growth points for itself.

Combined with the recent situation of the industry, it is not difficult to find that the end-side application competition of large models has emerged. Major mobile phone manufacturers have bet on large models, self-research or cooperation, in short, to seize the advantage in the track first.

But the entry is just a start.

After all, the application of large models is landed, how to continue to iterate, continuously optimize, and improve the experience is more critical, and the slightest mistake may be overtaken by the curve. This undoubtedly puts forward higher requirements for the team's technical research and development capabilities, product understanding capabilities and trend insight capabilities.

Therefore, the arrival of the application trend of large models not only brings a new growth cycle to the mobile phone industry, but also brings a technology upgrade cycle covering the entire industry.

No wonder some people have suggested that this year's big model reshaped the search engine, and next year may reshape the end side. This also makes people look forward to the large model developed by the mobile phone manufacturer, what new changes will it bring?

On November 1, vivo's self-developed large model will officially meet with everyone.

What upgrades and changes are you looking forward to seeing? Welcome to discuss in the comment area~