Discover, catch up, hug, APUS Li Tao's AI "distillation method"

Written by | Wang Pan  

Edit | Wu Xianzhi

After the advent of ChatGPT, Li Tao, founder of APUS, a global mobile Internet company, found that "there are three types of people in the market, one type of people are constantly telling stories, one type of people are extremely unwilling and lonely, and another type of people have a good mentality and start to do things desperately." ”

Li Tao may be one of the first Chinese entrepreneurs to come into contact with ChatGPT, and 4 years ago, OpenAI CEO Sam Altman showed him and Zhang Yiming, Shen Nanpeng and others through a video that using GPT-2 to operate Dota 2.

When Li Tao came into contact with ChatGPT last December, American elementary and middle school students were already using this new tool to complete their homework, and he began to feel the oppression brought by competition. "We may be 5 minutes behind others, but these 5 minutes are important, and you will find that we are at least 3 to 4 months behind in consciousness, technology and many places."

What is clear is that ChatGPT has revolutionized the means of production.

Since the traditional IT industry is a handicraft age, every programmer has to write code by hand, and now artificial intelligence is allowing the IT industry to enter the era of automation - GPT4 has been able to solve many programming problems.

In order to allow APUS employees to adapt to AI as soon as possible, the company has embarked on a series of business reforms based on large models. Since the beginning of this year, APUS programmers, designers and employees on the production line have begun to use artificial intelligence to improve their production efficiency, and began to use artificial intelligence to transform existing products 2 months in advance.

Photon Planet learned that in order to adapt to the era of artificial intelligence opened by ChatGPT, the standards for APUS salary increases this year are linked to the use of artificial intelligence tools, and a set of operable systems has been formed: if artificial intelligence can improve production efficiency, it corresponds to the salary increase ratio, such as increasing efficiency by 30% and raising salary by 10%; Increase by more than 50% and raise salary by 20%-30%.

Perhaps with a personal feeling, Li Tao has a new thinking about the future of artificial intelligence.

Growth and entrepreneurship

Li Tao's attention to ChatGPT, or APUS, embracing artificial intelligence is not an accidental thing.

APUS quickly started a new round of strategic change from the inside out, and there were two motivations. As a business manager, Li Tao enjoys the process of continuous entrepreneurship, and the development process of APUS is always sensitive to external feedback. Another driver is the previous shift to globalization and a greater awareness of the changing external environment.

When it comes to entrepreneurship, Li Tao believes that first of all, we must maintain a peaceful attitude, followed by following the trend, and finally enjoying the entrepreneurial process.

"You can find that most of China's digital enterprises represented by the Internet have a serial entrepreneur behind them, and they are still moving forward unremittingly after listing the company, why? Because going public is only a node for him, taking the listing as an opportunity, there are still many things to do. ”

In 2014, APUS was one of the earliest overseas enterprises in China. On July 2 of that year, the first product, APUS Launcher, was launched, and today, the total number of users of APUS system and its industrial product group has exceeded 2.4 billion worldwide, covering more than 200 countries and regions.

While going overseas has put APUS into the fast lane of growth, it is also facing blockades and suppression from all over the world, like most overseas enterprises today. So in 2019, APUS adjusted its development strategy, and this former pioneer began to transform itself into a "global" enterprise.

"[We] don't define ourselves as a Chinese company going overseas, but as an international company. China and the United States are just one of our chosen markets, and facing the global market and new technologies with this mentality makes APUS international in perception and mental modeling. ”

The emergence of ChatGPT has allowed APUS to see that artificial intelligence will change the format in all aspects. Li Tao once demonstrated the high-definition design on the spot, and it took at least two days to produce according to APUS's past AI model, while the AIGC capability based on ChatGPT only took one second.

In fact, even though AIGC has made great strides today, it is undeniable that most AIGC products are not large models, but self-built medium models. The so-called large model refers to the large amount of computing power and data, and the model that runs out of it is extremely inclusive, adaptable, and covers a wide range.

Large models are equivalent to collections of human knowledge.

As we all know, deduction and induction are the two major methodologies that promote the progress of human intelligence, and the underlying logic of the development of artificial intelligence is actually an abbreviated version of the history of human intellectual development.

Traditional search engines are actually the product of induction. This includes keyword entry, crawling data, and feedback on search needs. However, the inductive method is essentially collating and feedback existing knowledge, and does not create new value. It is equivalent to a person who can skillfully recite all Tang poems, but cannot compose a seven-law poem at all.

The reason why the industry believes that ChatGPT will be the next search engine is because ChatGPT helps humans reuse existing knowledge to produce a result that has never been seen in history. On the other hand, the results it produces are unique. In the past, these two things were essentially done by human intelligence, but now machines are also possible.

Li Tao believes that based on artificial intelligence, ChatGPT has shown signs of the next generation of operating systems. "If the first era of operation engines is Windows and Linux, the second era of operation engines is Android and iOS, and the next era of operation engines is artificial intelligence."

Because of this, APUS will fully introduce artificial intelligence into the production process.

Artificial intelligence in four steps

Previously, data, computing power, and algorithms were regarded as the "three major pieces" of artificial intelligence, and with the emergence of large models, "scenes" jumped to the most important part.

In the more than 20 years of Internet Galloping, China has not lacked scenarios and data, but there are many shortcomings in computing power and algorithms, especially the relative lag of the domestic open source environment, resulting in scattered algorithms and insufficient computing power.

Domestic science and technology companies regard competition as a top priority, and most of them have become "princes" with "algorithm" self-respect, objectively causing domestic algorithms to be in an extremely decentralized state, so they cannot form a joint force. Only the disclosure of code allows everyone to iterate, and the success of OpenAI has proved that the open platform can bring together more people to maintain and upgrade code, thereby improving the ability of data and models.

On the other hand, the computing power industry is still facing fragmented demand. For example, the domestic basic software ecology and supporting chips are not strong, the transmission capacity is insufficient, the ineffective computing power is increasing, the shortage of talents, and it is difficult to communicate and collaborate, and the domestic computing power is in a state of inadequacy.

There are two operational paths in front of domestic enterprises, one is a medium model with AIGC as the target, which can support 1 to 10 different products and software, which can not only ensure the operation of commercialization, but also maintain a large enough number of users, and train their own models.

Li Tao told Photon Planet that to solve the above problems, APUS found a method it calls "distillation" to train APUS models, and on this basis encapsulated into countless refining models to provide services, artificial intelligence represented by ChatGPT is the starting point of artificial intelligence. Simply put, use a large model such as GPT in reverse and use distillation to train your own model.

In addition, for the industry to focus on large models, APUS focuses on small and medium-sized models, such as AI painting, AI composition, AI programming, and so on. Compared with large models, small and medium-sized models are easier to find landing scenarios and can complete targeted training more efficiently, not to mention, they are encapsulated from ChatGPT.

Since large models and small and medium-sized models have clear boundaries, the large model is encapsulated as a small and medium-sized model, and the small and medium-sized model cannot be reversed to a large model, so the path of APUS to develop and integrate small and medium-sized models based on the large model of ChatGPT is of practical significance.

In fact, the boundary between small and medium-sized models is mainly reflected in parameters and scenarios. In contrast, the most complex model is generally a self-built independent model, such as Guangshi and SenseTime, most of which emphasize the inductive method and are lighter than the deductive method. In the future, there is limited space left for independent small models, and they are generally packaged into small models with large models or medium models.

In Li Tao's view, the primary goal of the big model is not commercialization, but to set industry standards, similar to the dispute between Android and iOS. For most enterprises, only need to care about small models and medium models. "After all, there are two most valuable things in the future world, one is the operating system, the other is the ecology is the scene, and the other things are passing eyes."

All in all, ChatGPT woke everyone up and opened a new starting point, now I should think about how to catch up quickly, so I need to borrow other people's data for my use, which is the core reason why APUS uses "distillation" to train its own model.

The singularity comes

Interestingly, while APUS embraces ChatGPT, Li Tao is not a radical in AI, and he believes that some human intervention should be introduced for now. In the long run, the efficiency of using artificial intelligence models to output results is higher than that of humans, and the efficiency of human intervention is far lower than that of AI model training, and human intervention is mainly on key algorithms.

APUS has set itself two "KPIs", in addition to the previously mentioned comprehensive transformation of artificial intelligence this year, the other is a performance growth of more than 200%. Li Tao's expectations for performance growth are not true, "Artificial intelligence means that every second, anywhere in the world, spending a dollar, what kind of revenue can be generated, and now we have begun to improve the productivity of internal systems." ”

At present, the reconstruction of artificial intelligence to the means of production is triggering a new round of industrial revolution, in Li Tao's view, the industry is in the singularity of change, will be carried out in four steps, namely internal efficiency improvement tools, existing product evolution, AI+ scene reshaping, and building a new industrial engine.

In 2023, the subtle changes in the live broadcast room seem to respond to the above views.

Wei Ya and Li Jiaqi retreated, and the anchors who succeeded them seemed to be unable to fully take over, and during this time, digital people began to fill the vacuum little by little. Although the current performance of digital humans is not satisfactory, with the development of artificial intelligence, especially the accelerated integration of large, medium and small models, not to mention the complete replacement of people, at least digital humans can replace most real people, and even the live broadcast room will be replaced by the meta-universe, thus completely changing the existing live broadcast e-commerce logic.

At present, there is a view that artificial intelligence will completely "replace" human capabilities, which is actually a kind of static thinking, ignoring that people, like AI, need continuous learning to continuously improve.

"Standing on the track of artificial intelligence, the competition is who uses artificial intelligence more thoroughly, who has the highest production efficiency, who can stand at the head of the tide, and those passive recipients will slowly be eliminated."