(The author of this article is Boss Dai)
In 2019, two things have plagued Son: SoftBank's failed investment, and Japan's technological lag.
Speaking to the media, Son sadly said that the tech industry has all but disappeared from Japan, and we are becoming a forgotten country. The coming AI revolution is Japan's last chance to return to the table[1].
With the advent of ChatGPT, Son's call has finally become a consensus. However, just as the whole country of Japan was mobilizing and preparing to break into the track with one foot on the accelerator, an embarrassing fact was discovered:
Japan's current AI research relies heavily on foreigners, led by next-door neighbors.
In this regard, Shoshi Sugiyama, head of RIKEN's Innovation and Intelligence Comprehensive Research Center, once made a statistic.
RIKEN (RIKEN Research Institute) is the only national research institute in Japan and a gathering place for top minds. Japanese Nobel laureates such as Hideki Yukawa have conducted research here. However, nearly half of the authors of the RIKEN papers included in the AI summit are affiliated with foreign universities, and about half of them are from China [2].
There is no one available in the local area, which makes Japan always in a state of aphasia in the wave of generative AI.
However, if you look back in history, you will find that Japan was also an "AI superpower".
In the eighties and nineties of the last century, Japan was the center of deep learning. Yang Likun, Yu Kai, Lin Yuanqing, Jia Yangqing and other heroes who have entered the history of science and technology have all spent a period of time in the AI laboratory in Japan.
Why did Japan, which once held the king bomb in one hand, move towards the end of old age and helplessness?
Standing on the shoulders of the Japanese
In the 80s of the last century, Yang Likun, who was studying in college, was attracted by a group of "crazy people".
At that time, deep learning was a "falsified" technical route. However, there is still a small group of people who are struggling, including a group of Japanese scientists. Yang Likun found that most of the deep learning papers at that time were written in English by Japanese researchers.
Among them, the one who had the greatest influence on him was a Japanese named Kunihiko Fukushima.
In 1980, Kunihiko Fukushima designed a multi-layer network model called "Neocognitron" based on the visual structure of cats.
In the primary visual cortex of living things, there are multiple neurons, each of which "controls" only a small part of the visual field. Subsequently, the information collected by the neurons is transmitted to the visual cortex and combined into a complete visual image.
Inspired by this, Kunihiko Fukushima designed two neurons for the neurocognitive machine, "perceptual lighting" and "motor information", which were used to "extract graphic information" and "compose graphic information", respectively. However, there is a fatal problem with Kunihiko Fukushima's neurocognitive mechanism: it is too advanced.
At that time, the mainstream neural network had only one layer, but the neural cognitive mechanism had five layers.
Faced with the problems caused by the multi-layered design, Kunihiko Fukushima could not find a solution for a while, resulting in the neurocognitive machine being able to handle some extremely simple tasks.
It wasn't until 1986, when Hinton proposed the "backpropagation algorithm", that there was a standard answer to this question.
However, if we trace the backpropagation algorithm back, we will find that its source is still Japanese. In the 60s of the last century, the Japanese mathematician Shunichi Ganli proposed the "stochastic gradient descent method", which provided technical inspiration for him [6]. However, due to the limitations of a specific era, Ganli Shunichi did not have the conditions to conduct simulation verification on the computer.
In 1988, Yang Likun combined neurocognitive machines with backpropagation to create the famous convolutional neural network. To this day, convolutional neural networks are one of the most important algorithms in the field of image recognition.
It can be seen that these AI research results, which define an era, are realized by standing on the shoulders of Japanese scientists.
Chinese in the lab
At the same time, Japan's industry began a more radical chapter in its history.
At that time, Japan entered an era of extremely prosperous bubble economy. Large enterprises have built central laboratories out of their own pockets to build basic science. NEC (Japan Electron Co., Ltd.) was the most radical of all: it cut directly into the heart of the American technology industry, opening laboratories in Princeton and Silicon Valley.
NEC Lab, which has a lot of money, quickly gathered a large number of names that will become famous in the future.
Gong Yihong, Dean of the School of Software at Xi'an Jiaotong University and National Distinguished Professor, is the first Chinese scientist to join the NEC Lab in Silicon Valley. During his tenure as the director of the laboratory, he recruited a number of young talents.
Among them, there are not only Yu Kai, who triggered the tech giant's bid for Hinton, but also Lin Yuanqing, Xu Wei and other technology giants who are active in the front line of China's AI industry.
At that time, China's computer industry was just starting out, and it could not absorb so many talents. Silicon Valley NEC Lab has seamlessly undertaken this demand and recruited a large number of Chinese scientists who are determined to engage in AI research.
After Yu Kai took over the position of director of NEC Lab in Silicon Valley, he recruited Huang Chang. At that time, Huang Chang had an intern named Jia Yangqing. During his time at the NEC Lab, Jia Yangqing demonstrated his superb mathematical and engineering code skills, which convinced everyone in the lab that he would be able to achieve something in the future.
This kind of incubation line that has been passed down from generation to generation has continued after Lin Yuanqing took over the Silicon Valley NEC Lab.
他引进的实习生谢赛宁，后来与麻省理工教授何恺明共同提出了著名的ResNeXt模型。 2022年，谢赛宁还和OpenAI研究员Bill Peebles合著了论文《Scalable diffusion models with transformers》。
Based on this paper, OpenAI created the video generation model Sora.
The NEC Lab, located in Princeton, also recruited Yang Likun and Vladimir Vapnik, the inventor of support vector machines.
It can be said that there is no other institution in history that has such a team of experts as NEC Lab.
Yu Kai once described the influence of NEC Lab at its peak in a media interview: If you search for NEC Lab on Google, a sentence will immediately pop up on the page, do you want to work at Google [7].
However, NEC Lab, which had no other glory in its heyday, has long been buried in the gray line of decay.
Mayflower Trek
In 2002, Yang had just been working at Princeton for a year when the NEC began to press.
The management unceremoniously told Yang that NEC had no interest in deep learning, and dismissed the director of the lab at the time. This experience made Yang Likun completely disillusioned with the industry and ran back to New York University to become a teacher.
There are two practical reasons why the NEC suddenly self-destroyed the Great Wall cannot be ignored:
One is that people have lost faith in AI. At that time, neither the computing power of the chip nor the richness of the data were far from enough to allow deep learning to realize its potential. At the same time, the failure of the "fifth-generation computer" project was even worse.
The "5th Generation Computer" project began in the 80s with the goal of building an AI-powered supercomputer.
In Japan, fifth-generation computers are envisioned to be capable of answering questions, knowledge base management, image recognition, code generation, and so on [8]. This scientific research project, which was "40 years ahead of the times", once frightened the United States a lot, and immediately took out subsidies to compete with Japan.
With such a project that slaps the head, the outcome can be imagined.
In 1992, the fifth-generation aircraft project was officially declared bankrupt. Japan has not only wasted hundreds of millions of dollars in vain, but also fooled other countries that have followed suit. In a fit of rage, people blame the AI. For a long time to come, AI research will be like a rat crossing the street, and everyone will shout and beat it.
Secondly, Japan's central laboratory model is also problematic at this time.
Japanese companies position central laboratories such as NEC Lab as pure basic research institutions. This model is not in line with the market and industry, but blindly pursues to win a few more Nobel Prizes. This makes scientists very distressed, and they often joke internally that "what is made can't be used in the product anyway".
Therefore, when the economic bubble disappeared and Japan entered an era of loss, it was only natural that the useless central laboratory would be the first to be "stabbed".
From 2009 to 2020, NEC carried out several layoffs of 10,000 employees and significantly reduced R&D spending.
At this stage, Chinese and American scientists have chosen to start their own businesses or choose another good tree to live in.
In 2012, Yu Kai was invited by Robin Li to lead Baidu's AI business. Under his call, Xu Wei, Huang Chang and other colleagues of NEC Lab have also joined Baidu. Later, they followed Yu Kai to found Horizon.
The AI spark ignited by Japan's great efforts eventually created the skyrocketing spark of AI in China today. After the fourth major layoff in 2018, the technical backbone of its U.S. laboratories has almost been lost.
Lone Hero
The rapid pace of Japan's AI triumphant progress came to an abrupt end with the fall of NEC's American laboratory.
The history of global AI continues, as if the fifth-generation aircraft project never existed; And the NEC Lab, which once formed an all-star lineup, is gradually forgotten. In the lost three decades, Japan has left almost no trace in the field of deep learning.
Not only that, but deep learning has left a deep-rooted bias in Japan.
In 2016, Google's AlphaGo beat Lee Sedol to the front page of global tech news. This year, 528 AI companies were born in China, and 371 AI investment and financing were born. From scientists to VCs, there is a lot of talk about the potential of deep learning. Next door, however, Japan is a different story.
In the same year, Japan's Ministry of Industry and Economic Affairs also held a national artificial intelligence conference. Some scholars were about to put forward two projects on deep learning, but they were reminded by fellow researchers in the academic circle, "If deep learning is added to the name, it is estimated that no one will listen to it."
This hesitation is an important reason why there is no one in Japan today.
The only one who realizes the problem is none other than Masayoshi Son of SoftBank Group.
In 2017, when Son launched the Vision Fund, the world's largest private equity technology investment fund, he was adamant that the fund would only invest in one strategy, and that was AI.
In the next few years, SoftBank's investment in AI can be described as aggressive.
In his quarterly and annual reports alone, Son mentions "AI" more than 500 times and has poured more than $140 billion into more than 400 AI startups. He even confidently stated in 2020 that the unprecedented investment frenzy would make SoftBank the investment company leading the AI revolution.
However, it is difficult to speak alone. What's even more embarrassing is that SoftBank also bet on the wrong treasure.
According to data released by the venture capital database PItchBook in 2023, SoftBank has only invested in one of the 26 AI startups valued at more than $1 billion.
In addition, although Son spent $4 billion to invest in Nvidia, he sold it all before its stock price soared, missing out on the nearly 10-fold gain. If it weren't for the bet on ARM, Son's AI investment might have been in vain.
In 2023, when ChatGPT sparked the generative AI boom, Son said bitterly at the shareholders' meeting that he had been reflecting since the end of 2022, "ashamed of the many mistakes he made" and "couldn't stop crying for days" [13].
Lost times
Son's tears are not only remorse for SoftBank's frequent betting mistakes, but also hatred for Japan's AI industry.
In 2019, Son openly criticized that Japan has become a "backward country" in the most important scientific and technological revolution at present, and the essential reason for its continuous loss of competitiveness is its lack of greed for progress[15].
This remark was mixed with a lot of emotional factors. In fact, these are not the root causes of the lack of support for Japanese AI.
Deep learning has never been an isolated revolution.
In 2012, deep learning was able to explode, in fact, there are two prerequisites: one is the evolution of computing power, and the GPU developed by NVIDIA at that time has initially been able to support the computing power required for deep learning. Second, the comprehensive rollout of the Internet has made up for the problem of insufficient data.
Integrated circuits, the Internet, cloud computing, with the maturity of these pre-industries, deep learning has been able to enter the historical stage. However, there are almost none of these industries in Japan.
At that time, Andrew Ng, who was teaching at Stanford, wanted to conduct a large-scale image recognition experiment, and Google used the computing power of the entire data center to realize his project The Cat Neurons (i.e., "Google Cat").
However, Kunihiko Fukushima and Shunichi Ganli were not so lucky. Even in Japan today, there is no private company that has the enormous computing power needed to train large AI models. Only in the government-led RIKEN research institute can a supercomputer such as Fugaku be found.
There is no successor to Japanese AI, and the foreshadowing has been laid from the very beginning.
Today, Japan's AI-powered technology industry seems to be confirming Son's prediction five years ago: Japan has lost its past, but may be losing its future.
Resources:
[1] Masayoshi Son, worried about Japan, "If things continue as they are, it will be a forgotten country", Nikkei Business
[2] Japan's domestic AI development relies on foreigners, Nikkei Chinese Net
[3] Why Japan is lagging behind in generative AI, CNBC
[4] The Road to Science, Yang Likun
[5] Algorithm Development in the Intelligent Era, Zhangjiang Science and Technology Review
[6] Gan Li Shunichi | Information Geometry: An Important Tool for Understanding the Learning Mechanism of Deep Neural Networks, AI Technology Review
[7] Dialogue with Yu Kai, founder and CEO of Horizon: The poetic six years in Germany have deeply nourished me, Che Yun
[8] The Fifth Generation: Artificial Intelligence and Japanese Computers' Challenges to the World, by Edward Figgebaum, Pamela McCordek
[9] How the ambitious fifth-generation Japanese computer failed step by step, CSD
[10] A Brief History of Chinese Artificial Intelligence, Lin Jun, Cen Feng
[11] The Current State of Artificial Intelligence in Japan and the Topic of "Deep Learning", Nippon
[12] The development and status quo of artificial intelligence in Japan, Dinglian Intellectual Property
[13] Masayoshi Son voted for AI, cast a lonely one, and Wall Street saw it
[14] Masayoshi Son: A great revolution is coming, and SoftBank will eventually rule the world, Wall Street has seen it
[15] Masayoshi Son criticizes Japan's competitiveness for reflecting on the backwardness of artificial intelligence, Asia Weekly
[16] Silicon Valley NEC Lab Past: The Man Who Dragged Chinese Enterprises into the AI Era, Lei Feng
[17] The Rise and Fall of Japan's Electronics Industry, Yoshio Nishimura
(The author of this article is Boss Dai)
The views expressed in this article are solely those of the author.