Shin Ji Won reports
Edit: Peach
【New Zhiyuan Guide】2023 Artificial Intelligence Index Report Released! The report shows that China leads the world in AI top papers, but has lower citations than the United States. In addition, among the world's top ten institutions in terms of AI paper publications, China accounts for 9 seats, and has caught up with MIT.
Today, Stanford released its 2023 AI Index report.
It is worth noting that the Stanford AI Index report lists the world's top ten institutions in terms of "AI paper publications", 9 of which are all from China, and have caught up with MIT.
They are: Chinese Academy of Sciences, Tsinghua University, University of Chinese Academy of Sciences, Shanghai Jiao Tong University, Zhejiang University, Harbin Institute of Technology, Beihang University, University of Electronic Science and Technology of China, Peking University, and MIT.
This year's report is divided into eight main sections: research and development, technology performance, AI technology ethics, economics, education, policy and governance, diversity, and public perspective.
The following summarizes several key points of the report.
Sino-US thesis cooperation ranks first in the world
From 2010 to 2021, although the pace of cross-border collaboration on AI papers has slowed, the number of AI research collaborations between the United States and China has increased by about 4 times since 2010, 2.5 times more than the total number of cooperation between China and the United Kingdom.
However, from 2020-2021, the total amount of U.S.-China cooperation grew by only 2.1%, the smallest year-on-year growth rate since 2010.
In addition, the total number of AI papers has more than doubled since 2010. From 200,000 in 2010 to nearly 500,000 in 2021 (49,601).
In terms of the type of AI papers published, in 2021, 60% of all published AI papers were journal articles, 17% were conference papers, and 13% were from repositories.
While journal and repository papers have grown 3x and 26.6x, respectively, over the past 12 years, the number of conference papers has declined since 2019.
Pattern recognition, machine learning and computer vision are still hot topics in the field of artificial intelligence.
China continues to lead in the total number of journals, conferences and repository papers.
The U.S. still leads in AI conferences and repository citations, but those leads are slowly being eroded. Nevertheless, most of the world's large language models and multimodal models (54% in 2022) are produced by US institutions.
China dominates the AI rankings, but the number of citations is lower than that of the United States
China has always maintained its leading position in the publication of AI journal papers, with 39.8% in 2021, followed by the European Union and the United Kingdom (15.1%), and then the United States (10.0%).
Since 2010, the proportion of citations of Chinese and engineering intelligence journal papers has gradually increased, and the European Union, the United Kingdom, and the United States have all declined. China, the European Union, the United Kingdom and the United States accounted for 65.7% of the total global citations.
So, what about the publication of the world's top papers?
In 2021, China accounted for the largest share of the number of papers published at the Global AI Summit with 26.15%, followed by the European Union and the United Kingdom with 20.29%, and the United States ranked third with 17.23%.
From the perspective of citations of top papers, although China is prolific, the citation volume is lower than that of the United States. The citations of top papers in the United States were 23.9%, and in China it was 22.02%.
From the side, it can be seen that China has the largest number of published papers, but the quality is not as high as that of the United States.
In terms of AI paper repository submissions, the United States leads the world, with 23.48%. China is the lowest, at 11.87%.
In 9 institutions in China, AI papers have been published to catch up with MIT
In 2021, China accounted for 9 of the world's top ten institutions in terms of total number of published papers, and the total number of papers published by different institutions is shown in the figure below, with MIT ranking tenth with 1745 published papers.
In terms of computer vision (CV), China's ten institutions rank among the top ten in the world: Chinese Academy of Sciences, Shanghai Jiao Tong University, University of Chinese Academy of Sciences, Tsinghua University, Zhejiang University, Beihang University, Wuhan University, Beijing Institute of Technology, Harbin Institute of Technology, and Tianjin University.
In the field of natural language processing (NLP), it's different.
The top ten institutions/companies in the world are: Chinese Academy of Sciences, Carnegie Mellon University, Microsoft, Tsinghua University, Carnegie Mellon University-Australia, Google, Peking University, University of Chinese Academy of Sciences, Alibaba, Amazon.
The speech recognition field is ranked as follows:
Industry leads academia
Among the important AI machine learning systems released in 2022, language systems account for the most, with 23, which is 6 times the number of multimodal systems.
In terms of paper output, industry is ahead of academia.
Until 2014, most important models were released by academia. Since then, industry has turned around. By 2022, 32 important machine learning models will be born in industry, compared to only 3 in academia.
It can be seen that building state-of-the-art AI systems increasingly requires a lot of data, computer power and financial resources than nonprofits and academia, and industry players certainly have more financial resources to do this.
In 2022, the United States produced the largest number of significant machine learning systems with 16, followed by the United Kingdom (8) and China (3).
In addition, since 2002, the United States has surpassed the United Kingdom and the European Union and China in terms of the total number of significant machine learning systems created
Looking at the distribution of researchers behind these important AI systems, the United States has the most researchers, with 285 people, more than twice that of the United Kingdom and nearly 6 times that of China.
LLM is getting bigger and bigger, and the more expensive the hash rate
Large language and multimodal models, sometimes referred to as base models, are an emerging and increasingly popular type of AI model that is trained on large amounts of data and suitable for a variety of downstream applications.
Large languages and multimodal models such as ChatGPT, DALL-E 2, and MakeA-Video have demonstrated impressive capabilities and are beginning to be widely deployed in the real world.
By analysing the country affiliations of the authors of these models, most of these researchers were from U.S. institutions (54.2%).
The Stanford AI Index report also lists a timeline for the release of large language and multimodal models.
Large language models are getting bigger and more expensive.
The first large-scale language model, GPT-2, was released in 2019 with 1.5 billion parameters and a training cost of about $50,000. Google PaLM is one of the large language models to be launched in 2022, with 540 billion parameters and a cost of up to $8 million.
In terms of parameters and training costs, PalM is 360 times larger and 160 times more expensive than GPT-2.
Not just PalM, large languages and multimodal models as a whole are getting bigger and more expensive.
For example, Chinchilla, a large-scale language model launched by DeepMind in May 2022, is estimated to cost $2.1 million, while training for BLOOM costs about $2.3 million.
Over time, GAN has progressed in face generation, with the last image generated by Diffusion-GAN, a model that made the latest SOTA on STL-10.
Last year, with the release of models such as OpenAI's DALL-E 2, Stability AI's Stable Diffusion, Midjourney, Meta's Make-AScene, and Google's Imagen, text-to-image generation models gradually came into view.
As below, enter the same prompt, "A Panda Plays Piano on a Warm Parisian Night", images generated by three publicly accessible AI text-to-image systems, DALL-E 2, Stable Diffusion, and Midjourney, respectively.
Of all the recently released text-to-image generation models, Google's Imagen performed best in the COCO benchmark.
This year, Google researchers who created Imagen also released a harder text-to-image benchmark, DrawBench, designed to challenge increasingly powerful text-to-image models.
In addition, the report describes some biases in current generative AI models, such as when giving DELLE-2 a hint to the CEO, everyone seems to adopt a confident posture of crossing their arms.
In Midjourney, when prompted to generate "influencers", it generates 4 images of older looking white males.
For the full report, see:
https://aiindex.stanford.edu/wp-content/uploads/2023/04/HAI_AI-Index_Report_2023.pdf