In January, VR Gyro twice introduced XR's standard technology eye tracking - "Breeding Two Listed Companies, 20 Years of Precipitation Tobii Uses Eye Tracking to Leverage Multiple Markets" and "New Generation XR Headset Standard, Detailing the Value and Challenges of Eye Tracking Technology", which has received the attention of many readers and triggered a lot of discussion.
Recently, we had a conversation with Johan Hellqvist, Vice President of Tobii, to discuss issues of interest to everyone.
Tobii Vice President You Haokui
VR Gyro: Hello Mr. You! We're excited to have the opportunity to discuss with you where XR is, the metaverse, and how eye tracking will bring new experiences to it.
You Haokui: Hello! I am also honored to have the opportunity to discuss this fascinating topic with you on the occasion of the chinese New Year.
VR Gyro: In recent months, the metaverse has become a hot topic of discussion. What do you think the metacosm is? What do you think of this concept?
You Haokui: As everyone knows, the metacosm is not a new concept. It came about because people made innovative connections between evolving technologies. It's a mixed reality (MR) space where people can get together to work, play, collaborate, and even make a living as themselves or avatars. It's a level playing field and anyone can enter from any device, anywhere. In the future, technology platforms will address the challenges of time and space, allowing users to experience authenticity and coherence in mixed reality.
Much of the popularity of the metaverse concept today is because big tech companies like Meta (the predecessor of Facebook), Microsoft, and Nvidia are using metaverses to describe the convergence of a range of technologies. Cloud services, WANs, and mobile networks have matured enough to provide operators with sustainable business models, and devices of all types have matured and begun to deliver on immersive promises. Shrinking form factors and lower prices are further driving the adoption of XR devices – creating new demands for content and an expanding market for developers. When you connect all of this, you get the metacosm and what it promises.
Image source: Network
VR Gyro: What does the Metacosm have to do with eye tracking and Tobii?
Yoh-hoo: As the metaverse evolves, we're going to see the ubiquity of VR and AR wearables, as they will provide more innovative immersive experiences. Tobii's technology will enable real connections that would otherwise be impossible in the metaverse because we can provide realistic eye contact and an understanding of human attention— core elements of communication.
From an early age, children learn to follow visual cues to understand the world. When you look into a person's eyes or observe how their lips move, the difference in intent interpretation is enormous. Therefore, only when the XR device is equipped with eye tracking can the real intention be effectively expressed in the communication between the virtual and real worlds.
Control devices such as keyboards or mice, which were not so intuitive at first, came to accept how this information was entered. But in the metaverse (especially in the AR environment), I think users probably don't want to wave our hands or control devices in an awkward way.
In real life, we use gestures, language, and eye contact to express intent, and so should experiences in the metaverse. Eye tracking that captures fixation, head movements, and facial features will enable fast authentication and intuitive communication using head posture and visual cues. Eye tracking will also support simple, intuitive gestures instead of finger pointing and other complex gestures that require careful manipulation.
Eye tracking is essential for people entering the virtual world using augmented reality devices. The information provided by our technology will help developers seamlessly weave virtual objects into the physical world. Therefore, eye tracking will help reduce the parallax between the real world and the virtual world, allowing users to feel real and coherent, and have a pleasant experience. In addition, eye tracking technology is also the basis for the development of multifocal displays and automatic adjustment systems.
VR Gyro: XR devices are the entrance to the metaverse, and they need to provide users with a sense of immersion and real engagement. For this reason, improving the comfort of the device experience becomes an important goal. What role can eye tracking play in this regard?
You Haokui: XR devices can improve user comfort through Tobii eye tracking, because the headset device can use the interpupillary distance measurement to match users with different physiological characteristics, so that the vast majority of users get a consistent and high-quality user experience, while also automatically adjusting the headset wear without affecting the user experience.
Users may also experience motion sickness, and high-resolution graphics can help solve this problem. This is where technologies such as dynamic fixation rendering play an important role in the operation of XR devices in split or native processing architectures.
In 2021, together with Nvidia, we released a new version of variable-rate supersampling VRSS 2 technology based on Tobii Spotlight. The two technologies work together to provide gaze information to drivers with minimal latency, enabling developers to create applications that benefit from dynamic fixation rendering without having to write a single line of code. The resulting stable high frame rate helps lower the stun threshold that some users experience when using VR.
(Source: Tobii)
A comparison of the same scene in VRChat shows that starting the Nvidia VRSS2 based on Tobii eye tracking can get a sharper image picture. The image above shows the rendering of fixations not turned on, and the image below is turned on
VR Gyro: What are the new technology trends in XR from a hardware perspective? Can Tobii ensure that solutions and integration platforms adapt to these new trends?
Yohokuei: Image quality and device weight have historically influenced the wearable experience, driving the development and adoption of folding optics (often referred to as pancake lenses in XR) to provide lighter, smaller headsets than traditional Fresnel lens designs.
In the shift in the gradual adoption of folded optical designs by XR OEMs, Tobii XR is well prepared for support. For example, most of our R&D work in 2021 will focus on developing a new generation of integration platforms to provide optimal solutions for this emerging architecture. Tobii's next-generation solution is designed to be compact and integrates without the need for intrusive changes to the headset, minimizing energy consumption.
Another important trend is that major hardware developers are adopting eye tracking in consumer-grade XR devices. Last year, two major XR headset devices natively integrated our solution: the Pico Neo 3 Pro Eye and the HP Reverb G2 Omnicept Edition. The success of Pico shows the market demand for all-in-one devices, while hp's success shows the market demand for multiple sensors. In addition, Pimax also announced a partnership with Tobii to bring the full potential of the metacosm to consumers.
A key differentiator for tobii's solutions for consumer device XR OEMs is that Tobii's integrated platform is broadly adapted to the vast majority of optical solutions, and eye tracking technology still delivers high accuracy and accuracy performance that is ahead of all other technology providers when it reaches a high level of user.
VR Gyro: What do you think is the biggest challenge when applying eye tracking technology in consumer XR devices?
Yoh-Kuei: As mentioned earlier, Tobii's technical goal is to be applicable to anyone, anywhere, and any device (the "three any principles"), and it's relatively easy to develop an eye tracking system that works in a controlled environment, but it's not easy to comply with the "three any principles." After more than two decades of tireless efforts, Tobii has been able to provide solutions for different eye shapes, colors, retinal reflectivity, and vision correction. For consumer-grade devices, our goal is not just to achieve more than 95% user coverage, but also to provide a powerful solution that can be reproduced in production equipment.
VR Gyro: Which eye-tracking applications do you think will first enter the metacosm?
You Haokui: Healthcare, remote collaboration, gaming, social, education, and simulation-based training have the potential to be pioneers in exploring the metaverse, where we've seen healthcare become a frontrunner in enterprise applications. I say this because in 2021 we see tremendous interest in Tobii Ocumen – an analytics toolkit product for data logging, organization, and advanced analytics applications to reveal human behavior.
In the consumer sector, given the current user interest and the way the platform evolves, social apps and games will experience rapid growth. I think these are the metaverse experience areas that consumers are most looking forward to. For Tobii, this means we will continue to work closely with XR OEMs, developers, and content creators to bring features like real social presences, realistic avatars, intuitive interfaces, and biometrics for authentication into the future metaverse.
(Source: Network)
SyncThink's EYE-SYNC product family features a Pico Neo device that integrates Tobii eye tracking and Tobii Ocumen
VR Gyro: Application development is indeed an extremely important part of the metaverse, can you talk about the importance of industry standards and OpenXR?
Yoh-hoo: One of the most effective ways to promote cross-platform development is through open standards, as they help prevent vendor lock-in and time-consuming and labor-intensive device adjustments. It also allows large tech companies and niche operators to work together. We know that OpenXR released an extension to the eye tracking standard back in 2020, and what we can expect this year is the implementation of the standard in supported solutions such as commercial devices and the Microsoft Windows MR platform. Tobii is a key member of the OpenXR Working Group, and we have been pushing for standards and cross-platform rollouts of eye tracking technology in OpenXR.
VR Gyro: At the start of the new year, what do you think we can expect in 2022? How will Tobii add value to the metaverse user experience?
Yoh-hoo: For me, I'm looking forward to integrating eye tracking technology into every XR headset device, which will require integrated technology like ours to achieve the goal of a high-quality metaversal experience that anyone, anywhere, and any device can get.
We'll see device types continue to diversify, but the main trend will be seen on lightweight all-in-one headsets. For developers, metaverse provides a solid platform for the development of commercial narratives and innovative applications.
We can expect to see some early prosumers, but as with most emerging solutions, the initial experience may be somewhat fragmented. We will also see more and more collaborative applications to help businesses push organizational boundaries and eliminate islanding effects caused by spatio-temporal constraints.
On the consumer side, we will see more consumer XR devices integrate Tobii eye tracking technology in 2022, making hardware performance better, allowing users to have more realistic avatars, more realistic eye contact and communication, easier interfaces and innovative gameplay.