During Facebook Connect 2021, the company, in addition to announcing the name change to Meta, confirmed another important piece of news: the headset code-named Project Cambria does exist. Meta CEO Mark Zuckerberg said it will be compatible with the Quest platform, but the headset isn't the next Quest.
In September 2018, Oculus Quest was officially released, on May 21, 2019, Oculus Quest went on sale, and Quest 2 was officially released in September 2020. There was a gap of two years between the two models, and a release time of one year and a few months, according to the time, Oculus's research and development progress is very rapid, and the update cycle is obviously shorter compared with HTC VIVE.
Now that Meta has made it clear that Project Cambria exists, it is likely that it is already in the second half of development, and the company said it will launch this year, but the exact timing of the release is still unknown.
1.Project Cambria
Project Cambria has been revealed many times so far, before Connect, user Bastian posted some videos on Twitter revealing Project Cambria, which were reportedly found in the "Seacliff" firmware, most likely an internal title.
It is reported that Project Cambria will be a high-end VR all-in-one machine, which is not the same level of product as the lower Quest2, and will not become a Quest Pro or Quest 3. Project Cambria also has features that quest 2 doesn't have: high-resolution color pass-through, eye tracking, face tracking, and no tracking ring on the handle.
High-resolution color pass-through: The Quest 2 can currently display the black-and-white real world in the headset after the user leaves the safe zone through the camera, while project Cambria will achieve higher definition, color-based pass-through. Realize the real switch between the virtual world and the real world, and further protect users.
Eye tracking: eye tracking is a technology that many VR manufacturers are trying to develop and use, and at present, domestic HTC VIVE Flow, Pico Neo 3Pro and other equipment have eye tracking versions to choose from. Eye tracking is thought to enable new input methods and to provide fovement rendering. So far, though, only confirmation that Cambria eye tracking will be used for the avatar function, and it remains to be seen whether developers will be allowed to use it for input and fixation rendering.
Face Tracking: Cambria has sensors to record facial movements, and face tracking is also used for avatar functions in conjunction with eye tracking, making 3D avatars more realistic in VR social interactions.
Handle without tracking ring: According to the first exposed video and renderings, the Cambria handle does not have a tracking ring, but the handle itself has multiple cameras, which may be used for optical tracking.
At that time, Project Cambria will be distinguished from the current Quest series, Project Cambria will focus on the high-end market, and the Quest series will become civilian VR equipment.
Project Cambria is most important for eye tracking and facial tracking, and Meta is currently developing full body tracking. Some developers have found that a new option for body tracking support appears on the Quest 2 Tracked Keyboard SDK Getting Started documentation page.
2.How does Meta view both technologies?
Andrew Bosworth, head of Meta XR, recently commented on eye tracking and full body tracking, which he believes are two of the biggest technical hurdles for VR.
Andrew Bosworth says eye tracking is far from mature. Meta has long been developing eye tracking to improve people's ability to express themselves, while also being used for gaze rendering. Biological differences between people are very large, especially astigmatism or other people's eyes are difficult to get full coverage. Coverage means that eye tracking is suitable for 100% of users, with particularly high requirements for center point rendering.
Capturing all body movements and transferring them to virtual reality has many advantages, as with eye tracking, enabling more realistic avatars and new forms of interaction. Bosworth says that for some of the use cases that people envision, external tracking of the body may be required. The biggest drawback here is the need for more components, which means extra cost and extra setup time.
In response to the new option for body tracking support on the Quest 2 Tracked Keyboard SDK Getting Started documentation page, Andrew Bosworth called it premature.
3. Two technical difficulties
Eye tracking and full body tracking development difficulty As Andrew Bosworth said, Meta is in full swing, but it is extremely difficult. Meta needs to face VR users around the world, taking into account the eye coverage of different people in different countries.
At present, eye tracking methods can be divided into probe coil recording method, infrared method, current recording method and video recording method, etc., each method has its advantages and disadvantages. Enough to prove that eye tracking itself has a certain degree of difficulty and difference, the technology has been developed to the extent that eye trackers in screens, webcams and glasses can capture and measure eye position and movement.
In the field of VR, it is mainly a problem of interpupillary distance and the difficulty of fove point rendering, and at present, fove point rendering has been successfully developed by tobii and other manufacturers. Interpupillary distance is the most difficult obstacle to research and development, and it is not yet known what solution Meta will adopt to solve the problem of eye coverage.
Full-body tracking can currently be done with some sensors, but most of them use external device connections for tracking, or require a separate camera for tracking.
The operation process of using external tracking devices is cumbersome, and it is not suitable for C-end experience, and external devices are generally difficult to achieve lightweight and other issues. Using a single camera to capture screen tracking can easily cause tracking failures.
4.Meta eye tracking and whole body tracking
In the future, Meta will continue to improve in eye tracking and full body tracking, and it is known that Meta is developing devices such as haptic gloves to achieve a more realistic VR experience. But prototype gloves are currently manufactured using a semi-automated process that requires skilled engineers and technicians to manufacture subsystems and mostly assemble gloves by hand.
Meta cautions that mass production of such gloves requires entirely new automated manufacturing techniques. Ideally, for maximum tactile accuracy and comfort, gloves should be tailored to the buyer's specific hand size and size.
Therefore, the cost of haptic gloves remains high, and eye tracking and full body tracking are likely to be used in VR equipment earlier.
Since Andrew Bosworth has mentioned it, it is very likely that eye tracking and whole body tracking will all be used in Project Cambria, but there are still technical obstacles. Meta product launches need to be available to global users, so they will be launched after the product and technology mature.
5. Summary of the full text
Current eye tracking and full body tracking are already being used in a number of fields, said Iowa State University Business Professor Joey M. Thompson. "If people can't focus on what's going on at the meeting, then maybe there's a way to improve the design of the platform to keep their attention and make the meeting more productive," F. George said. ”
The professor conducted an eye tracking study that showed participants paying attention to the person who spoke, based on eye tracking technology, but their gaze was more removed from the computer screen during group video calls than in large group meetings.
Eye tracking is also used in market research, website testing, gaming and user experience, learning and education, neuroscience and psychology, medical research, and more. In the XR field, the PSVR2 is said to also apply eye tracking technology.
In the future, eye tracking and whole body tracking will be more applied in the XR field, in addition to the above functions, eye tracking and full body tracking will enrich the VR/AR experience. At the same time, eye tracking and full body tracking will also be used in other fields, including office, entertainment and so on.
As artificial intelligence continues to advance, eye tracking and whole body tracking will also be combined with AI technology, but the day when eye tracking and full body tracking is fully used will still have a long way to go, which is not only a technical obstacle that Meta needs to face, but also a technical obstacle that we need to face.