Reporting by XinZhiyuan
Edit: Peach La Yan
【New Zhiyuan Introduction】Robot skin, pneumatic tactile gloves, Meta makes the future metaverse have a more realistic tactile interaction. Similarly, visual interaction cannot be ignored. A patent that has been filed shows that Meta is developing a mechanical bionic eye again.
Robot skin, pneumatic tactile gloves... Meta allows for more realistic tactile interactions in the future metaverse.
If you want to see the high-fidelity virtual world in the metaverse, visual interaction cannot be ignored.
No, Meta is going to develop a mechanical bionic eye.
For people? No...
Meta said that in the future, it will be placed in the body of a bionic human, and it seems that this is intended to make MetaHumans?
Recently, the company applied for a patent for a mechanical bionic eye called Two-axis rotatable mechanical eyeball.
The technology in the patent is said to be used to build Facebook's AR/VR applications in order to render 3D images and experiences faster in the metaverse.
In short, this patent can be called another progress of Meta in the metaverse.
Meta Next, Metacosmic Eye?
A "mechanical eyeball 100" document approved in December 2021 states that the metal sphere is a robotic eye similar to a human eye, with two axes of rotation that intersect at the center point.
Judging from the time of patent filing, this bionic eye patent was filed on June 11, 2019 and approved on December 7, 2021.
This patent details the design of a high-performance and realistic mechanical bionic eye.
In the future, the mechanical eye will be used as part of animatronic devices to more accurately track the movement of the human eye.
The description of the bionic eye design in the patent also seems to be quite complex, with parts intended to mimic the sclera, retina and cornea of the human eye, as well as the pupil and iris.
Why would Meta design a bionic eye alone?
Could it be for Westworld's AI-enabled bionics?
Meta said that the mechanical bionic eye was designed to match the robot's head, in order to make the eye appear more realistic.
The bionic eye will be equipped with sensors to help it "see" the real world, and can be trained and improved by machine learning.
Machine learning may be based on human eye movement data and then map this ability to the mechanical eye.
The patent also mentions that the eyeball will be placed in a mold similar to human skin. This mold may include both the head and the body.
Some netizens said, "So, are they MetaHumans?"
But exactly where the design is for, a Meta spokesperson said, "The company does not comment on the specific coverage of its patents or the reasons for filing a patent." In general, patents do not necessarily cover the technologies used in our products and services."
You know, the Meta bionic eye can also provide more practical applications.
The bionic eye can be used as a tool to more precisely track the movement of the human eye to observe how long it stays on screens and on images.
For example, eye tracking technology is often used in advertisements to observe how much users like ads.
Eye tracking technology could also be used to build Facebook's AR/VR apps.
For example, the Oculus headset that has been bursting into flames, if the future has the blessing of this technology, it is simply like a tiger.
As people familiar with the matter point out, this technology could accelerate the development of companies' augmented reality and virtual reality applications, which rely heavily on eye-tracking.
In any case, this mechanical bionic eye technology has strengthened Meta's determination to enter the metacosm.
In one fell swoop, the metaverse "visual touch"
This isn't the first time Meta has considered adding human sensory features to robots.
Back in November, Meta partnered with Carnegie Mellon University (CMU) to develop a lightweight, haptic sensing robot "skin" that can give robots a sense of touch.
The exploration of mechanical bionic eyes and artificial skin is said to be part of a broader robotics experiment conducted by Meta's AI division in 2019.
This skin is called ReSkin, the thickness is only 2-3mm, and the surface is similar to rubber with elasticity.
Looking closely at the picture, you can see that magnetic particles are embedded inside. When pressed/touched, deformation of the skin surface drives the magnetic field of the internal particles to change. These changes are collected by magnetometer monitoring and fed back to the AI model trained through self-supervised learning.
Zuckerberg said it could be used for virtual objects in the metaverse to get tactile sensations when we interact with them.
Meta's chief AI scientist Yann LeCun believes that robotics has become a way to study artificial intelligence, and he has said that robotics is an area that Zuckerberg boycotts.
LeCun has said, "This is where AI researchers are trying to get there." Perception, reasoning, planning, and acting form a complete closed loop that is constantly circulating, culminating in feedback from the environment."
Bionic eye main vision, ReSkin electronic skin main touch, and Meta's most iconic Oculus headset combine visual and auditory effects.
It can be seen that Meta has opened up the three major sensory experiences of human beings, such as vision, hearing and touch, on the road of entering the metaverse.
In fact, in terms of touch, some time ago, Meta also revealed its secret seven-year-old project "Pneumatic Haptic Glove".
This haptic glove allows you to experience the real touch when grasping virtual objects through air bags. For Meta, this glove and other wearable technologies will both lead us to a future where we can interact in VR and AR.
However, the experience in the true metaverse includes not only the three sensory characteristics of "visual touch", but also the ability to smell different smells and taste different tastes.
At present, in the Meta-universe technology system, the sensations that are not mentioned are smell and taste, which has higher requirements for overcoming technical problems.
However, there is nothing impossible in the future.
Who says meta-universe is PPT?
In November 2021, Facebook changed its name to Meta at connect and plans to invest hundreds of billions of dollars to enter the meta-universe.
Meta's move shifts everyone's attention to the virtual world it has created.
In fact, Meta's layout in the metaverse is not only the project mentioned above, but also the research and development of Avatar.
The metacosm is a 3D digital world with an immersive sense of space.
In this world, people create human avatars by wearing VR devices or chip implants, and integrate them.
In the metaverse, there are not only people, but also virtual objects, and more importantly, robots.
In addition to human-person and human-thing interaction, human-machine, machine-machine, and machine-thing interaction are also very important.
Related to robots is called the robot metacosm. The robot metacosm is the basis for robots to understand the physical world. The above-mentioned technologies such as bionic eyes and artificial skin are all designed to make robots more humanoid and realistic.
Meta has also made many attempts to improve the human experience in the metaverse, and there have been quite a few breakthroughs.
For example, the Codec Avatar project is a system that uses 3D capture and AI technology to create realistic avatars of people. The project started with the initial hd avatar and progressed step by step to a full-body avatar.
In the process, a full-body video codec was developed to allow human avatars in the metaverse to support more complex movements of the eyes, face, limbs, torso, and more. For example, the project team made XR input through an EMG wristband to accurately measure the way people move their hands.
Human incarnations have made many breakthroughs, so it is not a problem to map objects in the real world through physical means in the metaverse.
For example, the realistic simulation of clothes in the metaverse is very realistic. Not only does it perfectly reproduce the elasticity of the fabric of the clothes, but if you shake the clothes, the folds on the surface of the clothes are also vivid.
All meta projects are developed to enable people to have a better experience in the metaverse. This also confirms that Xiao Za's declaration of entering the meta-universe is not on paper.
The next step in meta-expanding metaversms, are you looking forward to?
Resources:
https://www.businessinsider.com/facebook-patents-animatronic-eyeball-robot-head-resembling-human-skin-meta-2022-1
https://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&p=3&u=%2Fnetahtml%2FPTO%2Fsearch-bool.html&r=119&f=G&l=50&co1=AND&d=PTXT&s1=facebook.ASNM.&OS=AN/facebook&RS=AN/facebook