China already has 79 large models with 1 billion parameters, and the industry calls for the establishment of independent innovation "moats" as soon as possible

"According to incomplete statistics, 79 large models with more than 1 billion parameters in China have been released, with Beijing and Guangdong being the most numerous in terms of region, and natural language processing is the most active field for large model research and development." At the Zhongguancun Forum, Zhao Zhigeng, director of the China Institute of Science and Technology Information and director of the New Generation Artificial Intelligence Development Research Center of the Ministry of Science and Technology, disclosed the situation of the Chinese intelligent big model.

Today, competition in the field of artificial intelligence is fiercer than ever. Some people call the recent big model entrepreneurship "100-model war", from Baidu Wenxin Yiyan, Ali Tongyi Qianwen to SenseTime, Kunlun Wanwei and other large models, to Wang Xiaochuan's Baichuan Intelligence, Wang Huiwen's light-years away, Li Zhifei's sequence monkey, etc., the "Chinese version of ChatGPT" has almost ushered in a boom in recent months. At the application layer of large models, progress in the field of large models has been announced in many subdivided vertical fields, including online office, financial technology, and online education.

Kai-Fu Lee, chairman and CEO of Innovation Factory, said in his speech that the AI big model is a historical opportunity that cannot be missed, and the AI big model has slowly moved towards the real economy, which will rewrite every application, reconstruct human work, and replace many repetitive labors.

After the forum, when asked by reporters how they thought of the wave of large-model entrepreneurship in full swing, some participants commented that "this shows that Chinese companies are very enthusiastic", and added: "It's like refining pills." ”

Beijing and Guangdong have the largest number of large models

Zhao Zhigeng said that from the perspective of the global development trend of large models, the United States Google, OpenAI and other institutions continue to lead the cutting-edge technology direction of large models, while more and more R&D teams in Europe, Russia, Israel and other countries are also joining the research and development of large models.

From the perspective of the distribution of large models that have been released in the world, China and the United States account for more than 80% of the global total, the United States ranks first in the world in the number of large models, and China has entered a period of rapid development of large models from 2020, and is currently maintaining a synchronous growth trend with the United States.

According to the "Chinese Intelligent Big Model Map Research Report" released at the Zhongguancun Forum, the geographical distribution of the Chinese intelligent large model has obvious characteristics, and 14 provinces and cities have carried out large model research, of which Beijing and Guangdong have the most, Beijing has 38 large models, and Guangdong has 20 large models.

In terms of model field distribution, natural language processing is still the most active focus area for large model development, followed by multimodal field. There are still further breakthroughs in the fields of computer vision and intelligent speech, and there are currently few models.

If the birth process of generative AI of large models is compared to "alchemy", then the computing power as a GPU is like a fire burning under an alchemy furnace. By investigating the distribution of computing power infrastructure nationwide, the report found that Beijing, Guangdong, Zhejiang, Shanghai and other places have the largest number of large models, and these four places are also the regions with the highest number of artificial intelligence server purchases in the past three years, showing a very obvious strong correlation and providing important support for the development and application of large models.

In terms of publishing related papers, the China Grand Model has formed a certain academic influence through academic paper publication. Among them, Beijing, Guangdong and Shanghai are in the forefront of China in terms of both the number of papers published and the number of citations, reflecting obvious talent reserve advantages, and Jiangsu, Guangdong and Shanghai are also regions with relatively many large model talents.

In terms of open source innovation ecosystem, more than half of the major models have implemented open source. Beijing, Guangdong and Shanghai rank among the top three in China in terms of the number of open source and open source influence, which is mainly promoted by universities and institutions, such as ChatGLM-6B of Tsinghua University, MOSS of Fudan University and Baidu's Wenxin series of large models open source.

Large model talents are scarce and original innovation is insufficient

Talents provide key intellectual element support for the development of large models. However, from the perspective of quantity, the total number of large model talents in various places is still scarce, and the number is insufficient.

When talking about the challenges facing current AI big models, Kai-Fu Lee mentioned the need for higher quality data and the need for more AI engineers and AI scientists.

According to the "AIGC Talent Supply and Demand Report for the First Quarter of 2023" released by Lagou Recruitment, in the first quarter of 2023, AIGC talent recruitment demand climbed for three consecutive months, and the demand for AIGC talent positions increased by 42% in March this year. On the recruitment platform, many companies even offer millions of annual salaries to grab AI technology talents.

And China's big models themselves also need to be constantly polished. While promoting open source and openness, many industry insiders mentioned that China also needs to strengthen basic research, "independent innovation is the only way to develop a big model." ”

Kai-Fu Lee mentioned that it is necessary to support open source, but China's large model companies cannot rely too much on open source models. "We need to establish our own IP (intellectual property) and technological advantages as soon as possible to form a moat." Because the open source model cannot reach the performance of the self-developed model of foreign large manufacturers, its ability will become a "ceiling"; At the same time, the open source technology of overseas manufacturers is also at risk of being shut down. Moreover, due to the different cultures, user habits, and laws and regulations at home and abroad, it is risky to bring models trained abroad to China for fine-tuning.

Dai Qionghai, an academician of the Chinese Academy of Engineering, also said that at present, the application of artificial intelligence in the mainland is strong, but the original innovation is insufficient, and it is weak compared with the United States in terms of basic technology and talents. Dai Qionghai suggested that the mainland should deepen the talent training and basic research of artificial intelligence from the policy, mechanism and investment, and strengthen original innovation.

In addition, although different innovation subjects such as domestic universities, scientific research institutions, and enterprises are actively participating in the research and development of large models, there are relatively few joint development between academia and industry. Zhao Zhigeng mentioned, "We have observed a trend of cooperation contraction, which is the next thing to pay attention to. ”

She suggested that it is necessary to strengthen the coordination of resources and R&D forces to promote the orderly development of large models, such as strengthening the coordination of computing resources such as intelligent computing centers, supercomputing centers, and cloud computing centers. At the same time, accelerate basic research and technological innovation, and enhance the influence of academia and open source.

She also stressed the importance of strengthening international cooperation and actively participating in global AI governance. The importance attached to AIGC compliance in various countries is driving the introduction of corresponding regulatory measures. In April this year, the Cyberspace Administration of China issued the Measures for the Administration of Generative Artificial Intelligence Services (Draft for Comments). Zhao Zhigeng expressed the hope that these governance principles and ethical norms can take root in the whole chain of the big model. On the basis of enhancing consensus, strengthen global cooperation on artificial governance and create China's wisdom and governance solutions. Some practitioners pointed out that in order to participate in the formulation of rules, China's big model must first be on the card table before it can have the right to speak and have a ticket to the global competition.