Record time: In 2021, why has China's artificial intelligence not yet reached the "golden age"?

Author | Chen Caixian

Edited by 丨 Cen Feng

What has AI Tech Review done in the past year?

As the year draws to a close, when I think about touching fish, I recall 365 days of uninterrupted article updates, and the most common thing that comes to mind is that at the end of May 2021, our contribution to the ICML paper of Ma Yi, a well-known Chinese scholar at UC Berkeley University, was accepted by 4 judges but was still rejected by AC for reporting. At that time, as soon as the article was issued, it immediately attracted the attention of AI researchers, and even Professor Ma himself popopo to Weibo:

However, in this matter, the most noticeable thing for the author is not Ma Yi's own forwarding, but a comment under the professor's Weibo (there is an illusion that the teacher is calling to the office to explain the motivation of the behavior during recess):

It should be known that the slogan of AI Technology Review is "focus on the frontier research of AI and pay attention to the growth of AI youth", and to achieve these two goals, it is essential to pay attention to and interpret the research papers of scholars. What is the motivation? In addition to paying close attention to the dynamics of each AI scholar and the growth of young students, what motivation can there be?

(As for how to make a profit, as long as the boss does not mention, we dare not say, we do not dare to ask... After all, the people who do research are intellectuals, and talking about money is a bit tacky... ）

Unconvinced, as a technology number, especially the current information explosion, media channels, modern people's attention is easy to be dispersed, and the group of people concerned about artificial intelligence technology has shown more and more complex multi-level characteristics, we are also a bit of an "identity crisis". Taking the opportunity to report to party A's father at the end of the year, the "hit worker" editorial team of AI Technology Review took stock of the excellent articles of the past year and found that in the past year, we were really too idealistic (dog head)!

Next, I would like to share with you several important roles in the past year's "AI Technology Review" for the progress of Internet memory artificial intelligence research professional gossip postures:

I. Academic Discipline Inspection Commission

In 2021, the field of artificial intelligence research is still a year of turmoil. During the year, AI Technology Review tracked and "exposed" more than 30 incidents of academic research corruption or unfairness. Among them, in January, Peking University professor Rao Yi reported that Academician Pei Gang was the most sensational, and "low emotional intelligence academic fraud, high emotional intelligence picture misuse" also became the most "flavorful" conversation of the editorial team after dinner:

• "Rao Yi reported Academician Pei Gang's academic misconduct late at night!" Netizen comments: low emotional intelligence academic fraud, high emotional intelligence picture misuse"
• "Pei Gang student Ling Kun sent a letter asking Rao Yi to cooperate, Rao Yi: Don't help"
• The dust of the "Rao Yi Report" incident has settled, and Pei Gang said that pei gang has not been found to be counterfeit. Netizen: I am confident to publish 20 SCI articles a year
• Rao Yi indignantly asks whether "image misuse" can be removed from china? China's "No Scientific Research Fraud" First Year: "Unprecedented""

The prevalence of paper fraud is affecting China's academic credibility. In the March 23, 2021 nature article "The fight against fake-paper factories that churn out sham science," Nature said that "since January last year, Nature magazine has retracted 370 papers, and the number is still increasing, all of which are from China." Moreover, 197 of these papers were medical papers, and these papers were flagged by fake fighters because of picture problems.

Professor Rao Yi has been called the "backbone" of Chinese intellectuals, and his demands and efforts to establish a fair academic environment have always been praised. Another well-known academic research fraud incident related to Professor Rao Yi was that Rao Yi reported in November 2019 that Geng Meiyu, director of the Shanghai Institute of Materia Medica of the Chinese Academy of Sciences, had committed academic fraud, and was later dealt with as "a large number of pictures misused", and Geng Meiyu countersued Rao Yi in the case of reputation infringement. Fortunately, in mid-December 2021, rao Yisheng was sentenced in the first instance of the lawsuit.

The biggest feeling that this incident gave the editorial team was that academic counterfeiting requires technical strength. Ordinary people or ordinary people with ordinary knowledge accumulation cannot find technical principle errors and "fraud" suspicions under unfathomable scientific research papers in massive research work. In other words, in the field of scientific research, to crack down on counterfeiting, it is necessary for peers to "grasp" their peers, and the devil is one foot high and the road is one foot high.

In addition to Rao Yi Pei Gang, the academic misconduct incidents exposed by AI Technology Review in the past year include:

• "Chinese professors at the University of Minnesota deliberately submitted vulnerability code to Linux for research!" Entire universities are blacklisted by Linux..."
• "Spend 3W to buy the core of the paper hair ghost lecturer was pickpocketed, Peking University: has started the investigation! 》
• "Forcing contributors to cite their own papers, IEEE senior members were banned for life, netizens: this is very common in China..."
• ……

The "peer review" system, which is expected to safeguard academic fairness and healthy development, has also been repeatedly overturned in 2021:

On May 27, ACM Fellow and Professor of Computer Science at Brown University in the United States, Michael L. Littman, posted on the ACM newsletter, denouncing the use of peer review mechanisms to conspire to "go through the back door" in the submission and review of papers at the top meeting, and the peer review mechanism was seriously exposed and criticized for the first time on the platform of authoritative academic journals.

Then, on May 28, Nature revealed that the peer review mechanism could not effectively identify and "reject" spam articles automatically generated by computer software, and that 64% of these articles came from China.

On August 3, IEEE Fellow Li Tao was publicly named by ACM and revoked his ACM membership, once again pushing the "peer review" mechanism to the cusp of academic misconduct, because li Tao was accused of violating scientific research integrity, and the first of the reasons for Li Tao's condemnation of violating scientific research integrity was to take the loophole of peer review: "There are dozens of pieces of evidence that Li Tao violated the peer review process of papers in various ways. For example, sharing information from reviewers, asking others to write information and publish it in the conference review system, thereby facilitating the review of a manuscript."

• "Academic Circles Want to Fry: Paper Authors and Reviewers Collude to Deceive Blind Reviewers, ACM Fellow Denounced!" The top will be "in the middle of the thought" (headline 1.9W+, in the look 36, comments 14, likes 34, reprinted 7) -5.27
• Peer review is a joke! Nature's latest revelation that computer-generated junk articles are acceptable, 64% from China (headline 2.1W+, looking at 42, comments 16, likes 35, reprints 9) -5.28
• "Expelling IEEE Fellow Li Tao's Membership!" ACM rare public roll call: Don't forget the tragedy, warn the future (headline 4.4W+, in the look 132, comments 28, likes 142, reprints 9) -8.3

How to promote the construction of a healthy scientific research environment in colleges and universities will also continue to attract attention in 2021. Among them, on June 7, Jiang Wenhua, a teacher in the Department of Mathematics of Fudan University, killed Wang Mou, the secretary of the department's party committee, which was the most sensational and the most lamentable, and the "Tenure Track" system of "either rising or leaving" once again aroused heated discussion.

As the AI technology review concluded at the time:

• Compared with the foreign Tenure Track, China's "non-rising or go" elimination rate is indeed high;

• "Non-rising or walking" lacks clear measurable indicators, and the mechanism of "winner take all" is easy to cause the intuitive perception of "raising clams";

• AI Technology Review has also reported on Wei Dongyi, a mathematical genius at Peking University who is "one food and one drink". For most people who have chosen the path of scientific research, they do not actually have particularly high requirements for material needs, but still have to struggle in a one-size-fits-all "non-ascend and go" mechanism, and survive in a disempowered environment.

At the time of the avalanche, not a single snowflake was innocent. After two months, X Hu broke out that the suspected relationship of Dr. Shuangfei was appointed as an assistant professor of 985 College, and was rated by netizens as "this is 'soaring and speeding away'"... Is the academic environment fragmenting?

This series of events warns us: when talking about scientific research achievements and research breakthroughs, perhaps our eyes should not only focus on "scientific research", but also pay attention to the soil and environment that promote the development of "scientific research", and think about how to promote the healthy development of science from the establishment of a good organizational structure. And is peer review the most effective mechanism at present? This is worth pondering. If so, how should we improve? If not, how should we break the old and build the new?

Back to the field of artificial intelligence. In January, UC Berkeley, Harvard and other universities jointly submitted ICLR slammed ICLR review bias, saying that AC decision-making preferences for well-known institutions and authors; in June, CVPR issued a "bizarre" policy prohibiting social media discussion of papers under review, which was criticized by Turing Award winner Yann LeCun as "limiting scientific progress"; in July, ICML 2021 announced the list of outstanding paper winners on the 19th. On the 20th, the original outstanding paper was changed to an honorary nomination paper, which made a big oolong; in November, there was a plagiarism paper in ICLR...

• Matryoshka? UC Berkeley, Harvard and other co-contributors ICLR slammed THE ICLR review bias, program chair: good manuscript, reject it"
• "Prohibit social media discussion of "under review" papers, CVPR New Deal hotly discussed! LeCun denounced: This is limiting scientific progress"
• "AI top will be really strange! ICML's outstanding paper said that change is change, and Tachibuchi almost won the award? Exclusive reveal behind the big melon》
• "ICLR 2022 Appears Plagiarism Paper | Reddit Netizen Hot Discussion: Confusing Behavior? 》

We don't dare to say, we don't dare to ask.

All in all, in the new year, AI Technology Review will continue to pay attention to the academic research environment and style issues at home and abroad, and be a good "Discipline Inspection Commission" melon eating professional household.

II. Ai Frontier Research "Recorder"

After talking about the major academic key concerns, let's talk about the specific scientific and technological achievements record. As an "AI Hobby", we have documented hundreds of research efforts that are influential, large and small, over the past year.

At the beginning of 2021, on January 6, a blockbuster achievement in the field of language and vision appeared: OpenAI released a report entitled "DALL · E" neural network model, can be directly generated according to the natural language text description like magic,

Transformer, released in 2017, still has a lot of related research work launched in 2021. Among them, Google's MLP-Mixer launched in May has aroused discussion among well-known scholars at home and abroad, and the controversy is continuous...

• Transformer is doing things again! Megapixel HD image easy to synthesize, the effect is charming》
• Transformer surpasses ResNet: YITU open source "can be big or small" T2T-ViT, lightweight version is better than MobileNet
• "Google's latest proposal does not require convolution, attention, pure MLP composition of the visual architecture! MLP is All You Need ? 》
• "CV Circle Killing Crazy! Following Google, Tsinghua, Oxford and other scholars have published three MLP-related papers, and LeCun is also speaking out"
• "AI Circle Is Magical! Google's latest research shows that convolution is better than Transformer in NLP pretraining. LeCun Ambiguous Statement
• "ICCV 2021 List! Discover a treasure paper – how to train 100,000+ Vision Transformers at once? 》

Transformer has pioneered the idea of sequence modeling and RNN equivalence, and the language models such as GPT and BERT that have achieved success in various NLP businesses are based on Transformer. And Google, OpenAI, Microsoft, META (Facebook) and other enterprises set off a wave of AI large-scale model expansion of the armament war, just as Percy Liang, Li Feifei and other more than 100 scholars jointly released a research report to point out the opportunities and challenges of the "basic model", the era of large models has arrived, and the industry will also play a greater role in breaking the ceiling of artificial intelligence.

This is not to say that academia is doing nothing. Whether it's Hinton's new paper: How to Represent a "Part-Whole Hierarchy" in Neural Networks? Or Yann LeCun's latest article: A Unified Framework for Self-Supervised Learning, Human Infant-Like Learning, and The Bold Conjecture They Propose: GWT (Deep Learning) → a New Paradigm for General Artificial Intelligence, and Academia and Industry Have Chosen Their Own Approaches to Advancing Artificial Intelligence.

Let's take a look at the differences between research in academia and the corporate world:

academia:

• UC Berkeley Discovers Enhanced Version of the "No Free Lunch Theorem": Every Neural Network Is a High-Dimensional Vector
• Deep Stable Learning: Recent Advances in Causal Learning | Tsinghua University Team CVPR Research》
• Princeton Studies "Minimums": Breaking the Sum of Squares, Limits of Quadratic and Cubic Optimization Problems
• Still meeting the "small pond" simulation? This graphics paper conquered the ocean! Dr. UBC: "The Whole World" Together"

Business:

• "Super Fire AI Face Changing Effect Is Coming!" Ma Yun, Musk and Cai Xukun together "ant ya hey", Lee Kai-fu immediately "only you" ~》
• "DeepMind training AI to play soccer, the wind and turmoil is comparable to a human! Passing, stealing, scoring, and matching are better than Chinese men's soccer teams (dog head)"
• "In order to make the AI constantly fight monsters and upgrade, DeepMind created a "meta-universe""
• "Breaking the "monopoly" of GANs|OpenAI New Research: Diffusion Models Graphic Conversion Effect Surpasses DAL-E"

Similarly, there is no shortage of academic and industry collaborations:

• The ResNets Kings Return! Google and Berkeley jointly posted: It was not an architectural problem to lose to EfficientNets.
• Train ai to play CS Counter-Strike! The study by Zhu Jun of Tsinghua University and the Cambridge postdocs was too incendiary | Childhood Memories
• "With "big data", but also "multitasking", Google AI bull Quoc V. Le found the key to the zero-sample learning ability of large models"
• 10x GPT-3! The world's largest pre-trained model "Enlightenment 2.0" is launched: 9 world firsts, multiple tasks approaching the Turing test

If the trend in previous years was for scholars to go from academia to industry, today, this group of scientists who have gone to industry has begun to differentiate. While enterprises are gradually taking over the discourse power of artificial intelligence research, artificial intelligence is also gradually landing, and productization has become the focus of a new round of industry-academia integration. Whether it is from the AI factory "exodus" back to the corporate world, or continue to stay in the industry to "start again", the collision of theory and engineering practice will further promote the deepening of cutting-edge research.

In the new year, AI technology reviews will continue to report on the frontier research of AI around the switch between academia and industry.

AI for Science "Plumber"

In December 2021, Science announced the top ten "annual breakthroughs" in 2021, ranking first in the application of artificial intelligence in the field of life sciences - "predicting protein structure with AI", where scientists used AI-based software to accurately predict the three-dimensional (3D) structure of a large number of proteins based on amino acid sequences. In 2021, a number of research institutions have carried out relevant research with AlphaFold as a benchmark, and AI for Science (USing AI as a tool for scientific research) has become a hot topic in 2021.

As early as 2017, Ng Puta proposed in a speech at the Stanford Graduate School of Business that "artificial intelligence is the new 'electricity'", a view that he has mentioned on many occasions and speeches since then. From this perspective, the outbreak of AI for Science in 2021 is no accident - from 2017 to 2021, it is exactly a 4-year innovation cycle, and 4 years of accumulation is enough to push an "AI+X" from the concept to the eve of the outbreak.

Rapid advances in data acquisition, regulation and digitization methods, computing infrastructure, and algorithm training methods indicate that ARTIFICIAL intelligence will be applied in almost all medical fields and diseases. In a sense, 2021 can be counted as the first year of the outbreak of AI+ biomedicine.

• AlphaFold 2 is no longer a monopoly! DeepMind and the University of Washington team rushed to nature and science on the same day
• "Sloan Award Winner Li Jingyi: AI+X is not always effective, the amount of biological data is small, the noise is large, and interpretability is the key"
• "CMU Ma Jian's team uses machine learning algorithms to present the genome folding process and reach the top of Nature!"
• "Focusing on AI drug discovery, Demis Hassabis announces the establishment of an "isomorphic laboratory" to compete with DeepMind! (Recruiting))
• "Explainable AI pioneer, Professor Pan Yi of Deep Science and Technology: AI pharmaceuticals, we must do more reverse engineering of "using structure to find small molecules""
• "For the sake of medical AI, they made a decision to "go against the grandfather""
• "In three and a half months, the "Tsinghua" startup used AI to create a highly effective new crown antibody, and the founder won the highest award in "computational biology""

In the fields of basic research such as mathematics, physics, and chemistry, AI has also become a weapon to help research:

• "Reaching the Top of Nature | DeepMind uses AI to achieve major advances in mathematics for the first time, helping scientists confirm two major conjectures.
• "These 5 mathematical conjectures were first proposed 30 years ago, and now AI proves them all wrong"
• "2021, stop obsessing over GANs and Transformers, GNN outbreaks have spread from CV to physical chemistry"
• "Golden Bell Award Winner Zhang Linfeng: When AI meets a physical model, what kind of qualitative change will occur? 》
• "DeepMind and Google are making a big move again!" Solving NP-hard MIP Problems with Neural Networks
• "Re-emerge nature!" From the water flow of the glass box to the rain forecast of 2 million square kilometers, DeepMind's AI physics simulation is in the sky.
• P vs. NP Fifty Years: AI Is Solving The Unsolvable

In the view of AI Technology Review, AI is not only the power of the new era, it is also like the water that we are familiar with and everywhere, together with electricity, it has become the infrastructure that drives social progress. In 2022, AI Technology Review will be a qualified hydropower (porter) worker, so that more people can appreciate the promotion of AI to scientific research.

Fourth, technical thinking "leverage"

Why can monkeys play with their minds? Oil pipe network red blogger to create "100 shots and 100 hits" automatic aiming bow and arrow can also be shot with closed eyes, what is the reason? Why is China's largest AI model with 11.3 billion parameters so keen on writing poetry and telling you if your boyfriend should break up? How to appreciate the scene in "Three-Body" where the two-way foil devours the earth comes true? Turning a stone into a CPU solves the global chip famine or a sky-high scam? ......

Behind all this, is the progress of technology, the hype of moral degeneration?

As a public account that pays attention to technological progress, in 2021, AI Technology Review also reported a number of hot topics in technology. From the young geeks single-handedly DIY to the "group combat" of dozens of authors of a paper by a large company team, as well as the "breakthrough progress" in the fields of brain-computer interface and automatic driving, they can often harvest tens of thousands of traffic; but when the AI technology review editorial department is doing the processing of topic selection, the biggest problem is how to do a good job in reporting the technology itself and "catching the audience's attention" Balance.

• Too Cyberpunk! Huawei genius teenager self-made B station top 100 Up trophy, netizens: technical difficulty is not high, but the insult is extremely strong》
• "True Bike! Huawei genius teenager has just "released" a driverless bicycle, netizens: This TM is not hotter than Tesla? 》
• "Because he couldn't buy RTX 3090, he spent 190,000 yuan to build a professional-grade machine learning workstation"
• "Mind typing" is close to the speed of ordinary people's mobile phone chat, experts: this is much more difficult than Musk's "monkey play game" | Nature Cover》
• 《》
• "Black Hole Renovation after Two Years: Mankind Obtains High-Definition Photos of the Edge of Black Holes for the First Time"

According to Wikipedia's definition, technology refers to the use of machines, hardware, or artificial tools by humans, as well as broader architectures, such as systems, organizational methods, and techniques. Specific to the specific business we're concerned about, a technology is initially for a business, but with the development of the technology, the technology itself can also be transformed into a new business (a typical example is cloud computing) to provide valuable solutions for other people and organizations.

According to the life cycle theory of technology, the growth of a technology is usually divided into four stages: the introduction period (theoretical and laboratory stage), the growth period (technology prototype stage), the maturity period (industrial landing stage) and the decline period. At present, the difficulties encountered by the landing of artificial intelligence are mainly due to the huge gap in the original innovation of universities with maturity levels 1-3 and the product applications of maturity levels of 7-10, which is also the most likely place to "overturn" in science and technology news reports.

2021 is also the year of the turnaround of many big vs of science and technology. AI technology reviews deeply feel that it is not easy to do a good job in the popularization of technology topics, for a hot event that bursts the circle of friends, in order to make our report really "out of the circle", not only to understand the essence of technology, but also to be happy about the progress made by new technologies, but also to maintain a truth-seeking and questioning, as a "bar" role, Stay hungury, Stay foolish, in order to do a good job in the science and reporting of technical topics, rather than using headlines to explode eyeballs.

In 2022, what else do you expect from AI technology reviews? Welcome to leave a message.