Dialogue with Xu Li, chairman of SenseTime: General artificial intelligence is either far away or passed all at once

Just like standing on the high-speed rail platform, the high-speed rail is driving quickly towards humans, and humans define this site as AGI, and the high-speed rail does not stop and drives past all at once. So AGI is either far away from humanity or passed all at once.

The big model is an expressor, and today, by mining people's intentions, the model ability can be further improved, and then continue to make the model bigger. The development of China's large models will definitely force the development of multimodal large models with scenes.

Xu Li, Chairman and CEO of SenseTime.

In 1964, science fiction writer Arthur Clark predicted a future in which humans would become a stepping stone to advanced life. In 2019, Musk said in Shanghai that carbon-based biology is the bootstrapping program for silicon-based biology. Humans are always committed to AGI (Artificial General Intelligence), but what moment is the real AGI? Will future superintelligence pose a threat to humanity?

On April 9, on the eve of SenseTime's release of the large model "Family Bucket", Xu Li, chairman and CEO of SenseTime, told The Paper (www.thepaper.cn) that human beings have been developing towards more powerful intelligence, and when intelligence reaches a certain level, it is indeed necessary to sit down and discuss whether to limit intelligence. At present, there is still no path to superintelligence in technology, artificial intelligence has not developed to the point of panic, and the problem is how to make the model more versatile.

AGI is either far away from humanity or passed all at once

In this round of artificial intelligence boom, OpenAI came out of the circle with ChatGPT, and the development of large models and large computing power pointed to the road of AGI. Humanity has always strived to lead to AGI, but what moment is the real AGI? Xu Li made an analogy, "It's like standing on the high-speed rail platform, the high-speed rail is driving quickly towards humans, and humans define this station as AGI, and the high-speed rail does not stop, and it drives over all of a sudden." So AGI is either far away from humans, or it passes all at once, and the moment close to humans cannot be measured at all. ”

"Some people talk about AGI, but not about Artificial General Intelligence, but about Digital Super Intelligence, and at this point in AGI, is it still artificial? It's hard to say. If it really reaches the level of human intelligence, who goes to press the button and tells it to stop here, intelligence can no longer be higher? Xu Li said that the definition of AGI itself is vague, the so-called AGI lies in how humans define it, if it is divided into several tasks in the industry to meet the requirements of the industry, it can be considered that the AGI of the industry has been completed.

Xu Li believes that the "Emergent Ability" brought about by the parameter explosion of natural language models wins in this wave of artificial intelligence. In large language models (LLM), emergence refers to sudden performance gains in the effect of such tasks when the model size crosses the threshold. We have seen the ability of thought chains emerge from large models.

Google invented the Transformer model, RLHF (Reinforcement Learning from Human Feedback), and CoT (Chain-of-Thought). Finally, OpenAI kept trying, and ChatGPT suddenly became critical, as if all probabilities increased by 0.1%, and the error of the final multiplication was small. ”

"The explanation for the ability to emerge is from quantitative change to qualitative change, it is not a sudden emergence." Xu Li hypothesized that if a problem is broken down into multiple subproblems, if the accuracy of each subproblem is 80%, the probability of success in multiplication is very low. If the accuracy rate of each question is increased to 85%, the ability to emerge after multiplication occurs. "What we are currently seeing is the improvement in the accuracy rate accumulated by large models and the ability to form a chain of thought, which is mathematically explainable."

Although it is difficult to say whether the emergence of the future will exist in the end, he said that there is still no path to superintelligence in the technology, and he has not seen what the real intelligent emergence is. When asked if superintelligence would threaten humans in the future, he argued that when intelligence reaches a certain level, it is indeed necessary to sit down and discuss whether to limit intelligence. The three elements of artificial intelligence in the new era are that the algorithm (parameter) multiplied by the data equals the computing power, and the scale of the computing power determines the model capability. As long as the scale of computing power is controlled, intelligence is controlled. But AI hasn't advanced to the point of panic, and the problem now is how to make models more versatile. A revolutionary cognitive change that OpenAI has brought to the industry is that it has brought a new paradigm for large model development, just like deep learning in the past. The big model is an expressor, and today, by mining people's intentions, the model ability can be further improved, and then continue to make the model bigger.

Natural language models are bridges that activate other business models

Domestic science and technology enterprises are still in the catch-up stage. "To go your own way, you have to have some differentiation. The so-called differentiation is to make good use of the industry's own endowments. Xu Li believes that the development of China's large model will definitely use scenarios to force the development of multimodal large models, which have industry differences.

In order to truly reach the "inflection point" of large model development, it is necessary to set an expected task set or task goal, complete a series of tasks in real scenarios, and pass the Turing test in such vertical scenes. "At present, everyone is moving forward, and as for what it is, it actually depends on the industry in which everyone is located. There is no one-size-fits-all model that solves all problems. ”

Since 2019, SenseTime has laid out a visual model with a scale of 1 billion parameters, and on April 10, 2023, it released the "Ridayxin" large model system, covering the 180 billion parameter Chinese large language model application platform "Consultation", the self-developed text and student diagram generation model "Second Painting" with more than 1 billion parameters, the AI digital human video generation platform "Ruying", the 3D content generation platform "Qiongyu" (scene generation), "Lice Object" (object generation), etc.

As a bridge, natural language models can activate models and applications of other formats. Xu Li said that SenseTime's use of hundreds of billions of parametric natural language models to string together other models to form a complete task set is the first step. "In the process, we will see more multimodal data fed into the network, presenting completely new capabilities to develop the next stage of more hybrid multimodal models."

The purpose of the "Daily New" big model system is aimed at B-end users, which is to fully connect with users' usage habits, mine the large capabilities of the model with more B-side scenarios, and provide a new multimodal training framework to train the next stage of multimodal large models, and help the development of multimodal large models through end-to-end iteration and application of subdivided scenarios. "The future model trend is that there may be 1-2 large models in an industry, and there are many small models and industry models left. In fact, natural language models or multimodal models, the future is this trend, large models will be developed, in each layer can do a lot of model superposition, for the final reasoning will also have a great performance improvement. ”

In the field of artificial intelligence, "model as a service" is constantly mentioned, Xu Li also talked about the commercialization of large models, model training and deployment, model downstream application and model incremental training, the whole set of services superimposed on artificial intelligence infrastructure SenseTime AI device, "Our benefits come from training models, deploying models, inference models, and providing some incremental services on the model." "In addition, the revenue also comes from vertical segments, such as finance, healthcare, commercial live broadcasting, etc." In the B-side application, there will definitely be many productivity tools that will be developed first, and then gradually the C-side application will bloom. ”