Since the concept of metaversics received widespread attention last year, relevant industrial entities have actively laid out, and local governments in mainland China have also introduced relevant policies to support the development of metacosmities. Movies such as "Ready Player One" and "Out of Control Player" outline the use of the metaverse for users: put on a helmet, wear gloves, put on a special costume, and the user can enter a virtual space, in this space, the user can run, jump, communicate with people as in reality, and can see the expressions of other users as in reality.
From the current product form on the market, the user's dialogue in the virtual space has been realized, and some game and hardware manufacturers' solutions can also support the transmission of the sense of hitting in the game to the human body, Baidu, NetEase and other companies have also released meta-universe products such as Xilang and Yaotai, which can be used by users to participate in and interact in the virtual space. However, on the whole, there is still a great distance between the metaverse experience scheme currently available on the market and the metaverse immersion experience outlined in the movie.
Can users hug and shake hands in the metaverse as depicted in the movie? How long will it take for a fully immersive experience to be realized?
The metacosm of "man"
"The so-called metaverse, the core is people." Yuan Yu, senior analyst of CCID Consulting, said this in an interview with the reporter of China Electronic News.
The most core problem to be solved in the metacosm is to incorporate people into the virtual world, so that the human body identifies with the perception of the virtual world, so that the human body is immersed in the experience of the virtual world, and people can interact and interact as in real life. "People" is the core of building a metaverse, and user "interaction" is an important feature that distinguishes the metaverse from previous stand-alone games. To allow the human body to have the same experience in the virtual world as in the real world, facial expressions, physical interactions, and tactile perception are necessary for the realization of an immersive human experience. Correspondingly, facial recognition, motion capture, haptic recognition and haptic feedback technologies are key technologies that enable the human body to achieve a more immersive experience beyond the audiovisual senses.
"With a facial recognizer, users can see each other's expressions as they would in real life." Wang Congqing, president of HTC China, said this in an interview with China Electronics News. HTC Vive's face tracker can be mounted on VR glasses to capture a person's expression into a virtual person's expression through the camera.
VIVE Face Tracker
In terms of motion capture, various equipment manufacturers have roughly introduced two technical routes. It is more common to carry a radar module and an infrared perception module in the handle, and with the help of optical vision schemes, the position of the hand is transmitted to the host or headset through the handle. Whether it is Inside-out or outside-in, it is through the optical scheme to complete the capture of the position of the opponent. Another solution is for wearable devices, such as the HTC Vive tracker tracker, which uses the technical path of lidar and IMU inertial sensors, mounts the action tracker on the user, and uses the lidar to complete position capture. To avoid laser blocking caused by motion occlusion, the IMU (Inertial Measurement Unit) provides signal complement.
In order to improve the tactile experience effect of users in the virtual world, major manufacturers are also launching their own technical solutions. Last November, Meta introduced tactile gloves, which allow users to experience the effect of grasping objects and touching the tactile sensation of different material surfaces with the help of ridge-like inflatable plastic sheets. In June last year, haptic feedback company bHaptics launched the tactile vest bHaptics TactSuit, which allows users to play supported games and experience the sense of hitting characters in the game. Last December, bHaptics unveiled its first consumer-grade haptic glove, Tactglove, which allows users to experience the feeling of a sphere rolling in the palm of their hand.
Overall, products that support users to achieve stereoscopic sensing in the metaverse are emerging in an endless stream, but specifically, it will be found that such products face many problems such as insufficient market application, slow technology iteration speed, difficult technology breakthrough, and lack of motivation to solve technical problems. "The product iteration speed is not as fast as expected", I am afraid that it is the most accurate expression of the three-dimensional sensing product facing the metacosm.
Three major obstacles to stereoscopic sensing: content, price, and comfort
The three-dimensional sensing vision is beautiful, but there are few landing products and limited market promotion products, is this a problem of technical bottlenecks or market demand?
According to statistics, in 2021, the global SHIPMENT OF VR all-in-one machines will be 8 million to 10 million units, of which Oculus Quest products account for more than 80%. One of the most popular, the Oculus Quest2, comes with two Touch controllers. The domestic VR hardware equipment brand Pico Neo3 is also equipped with a handle. The functions of such a gamepad include several aspects: gesture recognition - detecting the position of the user's hand and reflecting it in the virtual space; haptic control - the user completes the manipulation of the experience content through the handle; haptic feedback - conveying the user's tactile experience in the virtual space. The handle is the most common way the current market provides a user experience with the touch of the virtual world. For gamers, increasing the haptic experience is significantly more three-dimensional than the sensory experience brought about by pure audiovisual experience. For example, Sony's PS5 controller is designed with haptic feedback function, when the player uses the controller to play the "Astro's Playroom" game, they can experience the character walking in the grass, hitting objects, force ejection and other very rich and diverse, different levels of vibration. This is very different from the tactile experience that a traditional grip can provide with just "shaking". The Joy-Con handle of the Nintendo Switch is also widely praised for its ability to provide rich, delicate, layered tactile feedback.
XR Stop Motion Animation "Sick Rose"
However, these hardware products that have won market praise to enhance users' tactile experience in the virtual world have one condition: the content is complete. Last year's popular Oculus Quest2 supported more than 4,000 games, including beat Saber, a popular musical rhythm game. Sony and Nintendo are more because of their strong content production capabilities, providing a very large space for hardware product haptic feedback function and content support.
"Content comes first." In an interview with China Electronics News, Li Jie, vice president of Xiaopai Technology, emphasized the importance of content to the development of meta-universe stereoscopic sensing, "If there is no good content, users will not buy for hardware products." ”
Yuan Yu also believes that users buy experience rather than technology. Quality content is an important part of the user experience. If the content provides enough appeal to the user, and the use of hardware haptic functions will raise the user experience to more heights, there will be users who will pay for it. The supporting content is not amazing enough, and the effect of the content and hardware cooperation is not amazing enough, which is an important reason why the sales effect of tactile experience, motion capture, and facial recognition products on the market is less than expected.
The high price is also an important reason for the low willingness of users to buy accessories. At present, the common action tracking devices on the market need 3 to 5 to achieve the simulation of human body movements, 1 hanging at the waist, 2 binding to the legs, you can basically achieve the basic simulation of human body movements, and if you match 2 trackers in both hands, you can achieve a perfect simulation of human movements. However, the price of buying only 3 tracking devices has approached the price of VR headset hosts. This is still difficult for the vast majority of consumers to accept.
VR concerts
From the perspective of the motion capture technology solutions available on the market, visual motion capture, mechanical motion capture, acoustic motion capture, electromagnetic motion capture, and inertial navigation motion capture are several more common technical solutions. Among them, the IMU sensor used in inertial motion capture, infrared sensor for optical motion capture and other accessories, the price of a single piece is not high, resulting in the high price of motion capture equipment, the primary reason is still the high cost of the overall equipment program. "The price per sensor is not high, but the use of end devices is not enough for one sensor. To bring all the sensors together and make the entire hardware available, the process of integrating its technology requires technical reserves, which makes the cost of the entire system not cheap enough. Wang Congqing said.
Experiencing stereoscopic sensing not only brings consumers possible consumer cost pressures, but also brings pressure to manufacturers to weigh product comfort and improve product accuracy. Sun Kai, general manager of products and solutions of Goertek, said that the use of micro-actuators is a cutting-edge solution to convey the force feedback generated by the virtual world on the user's hand, including the use of pneumatic actuators, electroactive actuators, microfluidic actuators and so on. According to public information, the tactile gloves released by Meta use the scheme of pneumatic actuators. Meta claims that there are about 15 ridge-like inflatable plastic sheets on each finger, and internal sensors can capture how the wearer's fingers are bent, and will also give the user the feeling of fingertips touching objects. However, the use of this product not only requires users to wear bulky gloves, increasing the weight of the product to the user, but also has wire connections, thereby limiting the user's range of activities. The ultrasonic sensor proposed by ultrasonic sensor manufacturer Emerge gives users the possibility of feeling the shape of objects in the virtual world without wearing additional equipment, but the product also inevitably has the problem that users need to be in a specific location to use the product.
How long does it take to hold hands with netizens in the metaverse?
"90% of the technology used in Ready Player One is available now, but 90% of the current technology is immature." Li Jie said, "Many product technology ideas and prototypes were launched a long time ago. For example, the treadmill in Ready Player One launched a prototype product in 2016. But without exception, these products have many problems in terms of comfort, safety, adaptability and so on. ”
From a technical point of view, it is already possible for one user to achieve haptic interaction with another user in a virtual space. However, to achieve the application for a wide range of market groups, so that it becomes a widely used and accessible technology, it still needs to be matured by the entire industry.
Yuan Yu said that the application and promotion of the meta-universe needs to be gradually promoted, and it is currently in the initial stage. The so-called primary stage refers to the fact that the industry's application exploration of the metaverse is still in the experimental stage, and the understanding of the metaverse by the whole society is still in the conceptual stage. Judging from the domestic industrial attempts in the name of the metacosm, the current industry's mining of the industrial value of the metacosm is still in the testing stage, and the exploration of the metacosm form is still relatively young. Baidu Xilang and NetEase Yaotai have built their own meta-universe platforms and hosted many activities, but users walking in it will still have a feeling of visiting the exhibition hall online, and the sense of interaction with other users is still not strong enough. Compared with the more immersive three-dimensional sensing interaction experience brought by users who can recognize each other's facial expressions and actions, and feel each other's actions such as shaking hands and clapping, at present, the focus of metaverse-related industry participants, including the meta-universe platform, is still looking for the industrial landing direction of the meta-universe and implementing application scenarios. As for optimizing stereoscopic sensing, it is a higher-level goal to consider on this basis.
In terms of technological improvement, Li Jie believes that the accuracy of sensing adaptation is not high enough and the lack of corresponding parameter standards are important reasons why the somatosensory capture technology of meta-universe products is less mature than expected. As an industry that is still in its infancy, technical standards are in the state of each company standing on its own. The establishment of industry standards requires the discussion and cooperation of many enterprises, which is obviously not a problem that can be solved at this stage. From the perspective of market promotion, the meta-universe supporting hardware represented by VR headset devices is still in the niche market stage, the VR headset market is small, and the market size of its accessories is naturally difficult to achieve huge growth.
VIVE envisions an office scenario using VR equipment
Solving the problem of market demand for VR equipment and "cultivating scenes" is the first problem to be solved in the process of the development of the metaverse to industrialization, and it is also the first problem to be responded to in the development of metaverse-oriented stereoscopic sensing. What is the demand for improving the stereoscopic sensing effect?
Judging from the relevant policies of the metaverse introduced by various places since the beginning of this year, the planning of the metacosm in various places lies in promoting the integration of the metacosm and the existing local resources and driving the development of local existing industries. For example, the "Taihu Bay Science and Technology Innovation Belt Leading District Metacosmic Ecological Industry Development Plan" released by Wuxi Binhu District mentions that it is necessary to accelerate the integration and innovative development of metacosmities with integrated circuits, blockchains, artificial intelligence, cloud computing and other technologies. The "Meta-universe Industry Development Action Plan" released by Shanghai Hongkou District proposes to focus on building a demonstration area for the development and application of the meta-universe in the North Bund, cultivating and introducing a number of high-quality enterprises for the application of meta-universe scenes, and building a number of meta-universe industrial economic spaces.
Cooperate with the current existing resources to promote the development of existing industries, which is the common answer given by various places. Implemented into subdivision scenarios, where is the growth space for stereoscopic sensing?
Zhao Siquan, an IDC China analyst, said in an interview with China Electronics News that the game and office fields are areas where it is easier to highlight the advantages of facial and motion tracking technology. In her view, in both areas, products equipped with face tracking and motion recognition technology will easily highlight their technical superiority, allowing users to finally choose the application.
Yuan Yu also believes that facial recognition technology will be the earliest in a variety of stereoscopic sensing technologies to achieve market expansion and technology optimization. In his view, image recognition in the security system, medical field there is a broad application space, in contrast, the scope of market demand for action recognition is narrower, will only be in the design, games, movies, cultural tourism and other specific areas have needs, its demand group is more niche.
For hardware manufacturers, improving the sense of product experience and optimizing feedback programs is the direction of their long-term efforts. Sun Kai said in an interview with China Electronics News: "High precision, high reliability and low latency are the direction of product sustainable development, expanding the new application of mechanical vibration and touch full chain direction, organically integrating virtual and real, and providing end users with a more realistic and rich tactile experience." "In the case of product form remaining unchanged, optimizing existing products and improving tactile accuracy is also a way to optimize users' three-dimensional perception of the virtual world."
Optimizing the stereoscopic sensing performance that products can provide is also the direction of efforts of many manufacturers. On the one hand, manufacturers are exploring the possibility of achieving a more three-dimensional and immersive virtual world experience, on the other hand, some manufacturers are trying to find another way. Wang Congqing seems that various types of sensors may have shortcomings, or a strong sense of wearing, affecting the immersion experience; or sensitivity, feedback and the user's own feelings are different, therefore, the brain-computer interface or will become a higher level sensing form to enhance the user's sense of immersion in the virtual world, but Wang Congqing also said that this is farther than the improvement of the existing stereoscopic sensing products in the current market The technical imagination, its technical perfection will take at least ten years, before there must be a better product form out.
Shaking hands with friends in the virtual world, a move that can be easily achieved in the real world, is still a long way off before most people can perceive in the metaverse. Li Jie believes that from now on, it will take at least 5-10 years to mature stereoscopic sensing technology. Yuan Yu said that if it is optimistic, it will take about 10 years to achieve large-scale tactile interaction of users in the virtual world.