Pictured: Qian Xuesen translates VR as "spirit realm" in his letter
"It is a good thing that concepts such as virtual reality and metaversics are hotly discussed at home and abroad, and it is also a catalyst to promote the progress of related disciplines, but as far as scientific research is concerned, I think the most important thing may not lie in the progress of a technology or algorithm itself, but the breakthrough of related technologies to the paradigm of traditional scientific research thinking, and the change of previous human life, movement, and health management methods." Sheng Bin, a professor in the Department of Computer Science and Engineering at Shanghai Jiao Tong University, said in an exclusive interview with The Paper (www.thepapewr.cn).
Sheng Bin is a young top talent winner of the National "Ten Thousand Talents Program", the head of the Key Laboratory of Artificial Intelligence of the Ministry of Education and the Shanghai Artificial Intelligence Research Institute of Intelligent Medical Technology, and has researched in the fields of artificial intelligence and virtual reality (VR). Based on the world's largest community diabetic fundus imaging database led by Academician Jia Weiping of the Sixth People's Hospital affiliated to Shanghai Jiao Tong University, Sheng Bin's research group has built the deepDR system of the whole course of diabetic retinopathy, which realizes the automatic diagnosis of the whole course of diabetic retinopathy from mild to proliferative lesions, and can provide real-time feedback on the quality of fundus images and the identification and segmentation of fundus lesions. The paper was selected as an Editors' Highlights by Nature Communications, a sub-journal of Nature, and is considered to be one of the most important technological innovations and research achievements in the field of translational and clinical research in the international academic community recently.
https://www.nature.com/articles/s41467-021-23458-5
In the field of virtual reality, from 2019, sheng bin team deeply participated in the "Science and Technology Winter Olympics" project of the Ministry of Science and Technology led by the Shanghai University of Sport, and built a virtual reality-based snowmobile simulation training platform for athletes preparing for the 2022 Beijing Winter Olympics. Due to the high cost of snowmobile slides and extremely high technical requirements, there are currently 17 snowmobile slides distributed in a very small number of countries in the world, and the extreme demand for related venue resources has greatly troubled Chinese athletes to prepare for competitions. For example, due to the lack of suitable track support, the first China Snowmobile Championship in 2017 was held in Calgary, Canada.
"With the strong support and help of Professor Wang Ran and Professor Gao Binghong of Shanghai Institute of Sport, we have repeatedly solicited the opinions and feedback of Chinese national team leaders, coaches and athletes, the purpose of which is to face the challenge of the card neck that plagues the large-scale popularization of Ice and Snow Sports in China, and then fundamentally promote the improvement of China's ice and snow sports level through scientific and technological forces." With the support of the Technology Winter Olympics project, we transform snowmobile sports training in a virtual environment into a virtual simulation-assisted teaching and training system with a high level of competitive experience through realistic graphic modeling and rendering, physics simulation in ice and snow skiing, and highly immersive virtual reality interaction. Sheng Bin said.
"Ice and snow sports should not belong only to Gu Ailing," Sheng Bin said with a smile, "virtual reality technology can allow more Chinese teenagers to experience the joy of ice and snow sports." Since 2019, the virtual simulation system of ice and snow sports and its teaching application led by Sheng Bin has been selected by the Shanghai Municipal Education Commission as the Shanghai Virtual Simulation Teaching Project and the first-class curriculum of Shanghai Jiaotong University.
In January 2022, on the eve of the Beijing Winter Olympics, led by Vice President Wang Jihong of Shanghai University of Sport, shanghai jiao tong university, Baoshan District Education Bureau and Shanghai Artificial Intelligence Research Institute and other industry-university-research units, with the support of the Ministry of Education, the "Intelligent Sports Education Department Application Practice Community" was established, one of the tasks was to rely on the virtual simulation system of ice and snow sports developed by Sheng Bin's team to promote it to the physical education teaching and scientific and technological experience of primary and secondary schools in Baoshan District, Shanghai. Help more Shanghai teenagers understand and love ice and snow sports. Sheng Bin said, "It is equivalent to leaving a little flame in his heart." ”
"Our country's long-term competitive performance in the Winter Olympics has not been outstanding, the reason is, first of all, there are not enough ice and snow geographical resources, and second, there are shortcomings in our knowledge popularization of ice and snow sports among young people." Sheng Bin said. Not only in China, in fact, few countries except those that have hosted the Winter Olympics will build such high-cost and low-output professional slopes – large ski resorts can also be converted into civilian leisure facilities after the competition, and purely professional tracks are really difficult to use in addition to training competitions. Moreover, the speed of snowmobile movement is generally up to 130 km / h, the world record has exceeded 200 km / h, known as "snow F1", for safety reasons also has high professional requirements for participants.
In this regard, Sheng Bin believes that "whether it is steel frame snowmobiles, skiing, or speed skating, the reason for the success of these athletes is of course their physical conditions and hobbies and other factors." However, there are many Chinese, with rich physical fitness suitable for ice and snow sports talent resources, but due to geographical and climatic constraints, not all of these people can easily be in the enlightenment stage can be in the field to contact ice and snow sports, a large number of ice and snow geniuses may be buried, then, Shanghai Jiao Tong University and Shanghai University of Sport jointly developed in the virtual reality environment ice and snow sports simulation system, may be able to help China's ice and snow sports future talent reserves to the greatest extent, help them take the first step in ice and snow life. ”
Sheng Bin also combined the current hot concept of "metacosmology" to further explain the technical background behind the virtual simulation system of ice and snow sports, "Our virtual simulation system of ice and snow sports can be seen as a metacosmic application of ice and snow sports teaching, and the metacosm can be seen as an expression of human network life in a virtual reality environment." The so-called virtual in virtual reality refers to the use of computer technology as the core, combined with related scientific fields, to generate a virtual digital environment, to provide people with vision, hearing, touch, spatial information and other perceptions; and the so-called reality, is to refer to such a digital virtual environment and all kinds of perceptual information provided to people, can make people immersed in it, produce a sense of reality. ”
VR is just beginning to mature: vertigo is not yet fully solved
In the 5G and even post-5G era, the basic hardware that supports the large-scale development of VR, such as chips, sensors, optics, and displays, has mature development conditions. Roblox, a meta-universe concept stock company that uses VR and other devices as interfaces, had a market value of $50 billion after its listing in March 2021, and ByteDance's acquisition of Pico, the first VR manufacturer in China, once again attracted widespread attention. In addition, Goertek Group began to lay out the VR/AR industry in 2012, and currently occupies 70% of the global market share of high-end virtual reality headsets, and takes the lead in introducing the world's leading 12-inch nano-embossing production line to create the world's leading VR/AR high-end optical components industry base.
According to a report released by IDC in November 2021, shipments of VR headsets will increase from about 5 million units in 2020 to more than 28 million units in 2025. Meta (former facebook) CEO Zuckerberg has said that about 10 million people need to use and buy VR content on a platform to enable developers to continue to develop and monetize; once this threshold is exceeded, content and ecosystems will leapfrog.
In the domestic academic ecology construction of virtual reality, Virtual Reality & Intelligent Hardware (VRIH), an international journal in the field of virtual reality independently founded by Chinese academics, was founded less than three years ago, that is, at the end of January this year, it was included in the world's most extensive and complete engineering literature database EI Compendex. And in October this year, with the world's oldest and most renowned international academic conference in the field of virtual reality and graphics, Computer Graphics International (CGI2022), co-organized the "Meta-Universe" album, as a member of the editorial board of VRIH journal, Sheng Bin believes that the above events have proved that China's virtual reality academic and industrial achievements have an increasingly important influence in the world.
In the process of designing the virtual simulation system for ice and snow sports, Sheng Bin realized that the most subtle part lies in the realistic modeling and simulation of a sport that has no prior experience at all, and is committed to using virtual reality and artificial intelligence technology to restore and reproduce the real experience and feelings of others on this activity to the greatest extent. "This is actually one of the charms of the metaverse, to really do this well, in addition to the modeling, rendering or interaction technology of a difficult problem, it is also very important to integrate other disciplines, such as motion physiology, some equipment and equipment on the ice motion law, etc., into the system development process."
According to the "White Paper on the Development of virtual reality industry 2021" released by CCID Research Institute, there is no breakthrough in the key core technologies such as vertigo and interaction in the current virtual reality industry, and there is still a gap between the wearing comfort of VR, AR, MR and other terminal equipment and visual comfort and comfort experience.
As for the feeling of vertigo, Sheng Bin explains, "Essentially, the dizziness actually comes from the discomfort caused by the brain processing 'not real enough information'. For example, the unreality brought about by visual perception, the flickering of the picture, the low resolution, or the abnormal light and shadow effect will bring discomfort. There are a lot of techniques for dealing with vertigo, but at the moment I don't think this problem has been completely overcome. This is also why many people now talk about metaversms, but the actual experience will still be stuck in the device. ”
"Ultimate Display": The Virtual Reality Window of the Future Human Digital Society - Metacosm
As early as 1965, The Father of Computer Graphics, Ivan Sutherland, published a short article entitled "The Ultimate Display", proposing that the ultimate goal of display technology is to immerse people directly in the virtual environment controlled by computers, just as daily life is in the real world. "At that time, the expectation of the ultimate state of this display technology was the window of the Windows system, where people could walk into Windows, just like Alice in Wonderland, they could move, they could hear, they could see, they could touch and smell." Sheng Bin said.
This "ultimate display" idea continues to this day with the saying of "the next generation of universal computing platform". In 2016, Goldman Sachs released a report, "VR and AR: Augmented Reality" (Augmented Reality). In February 2022, Guotai Junan released the report "MR: Mixed Reality for the Next Generation of Computing To the Metacosm".
Sheng Bin revealed that he is currently working closely with the team of Academician Jia Weiping, a chair professor at Shanghai Jiao Tong University, to carry out scientific research with a multidisciplinary team of experts from sports science, clinical medicine, psychology and brain science, and to develop a "spiritual realm" system for the intelligent and refined management of people's health in future life, hoping to answer the problems that have plagued the scientific community for many years through artificial intelligence and virtual reality: what is the impact of real-world physical exercise and virtual world (meta-universe) VR sports on human physical and mental health.
Sheng Bin believes that behind these efforts lies his insistence on scientific research beliefs, "If we finally communicate with machines through AR, VR or MR, then through the comprehensive use of multidisciplinary interdisciplinary knowledge, based on big data and high-performance universal computing platforms, virtual simulation and artificial intelligence can fully realize the simulation and prediction of the real world and human body state changes." ”
Pictured: Qian Xuesen expresses his expectations for the future of VR in his letters