Reporting by XinZhiyuan
Edit: Peaches Small salted fish
At the Connect 2021 conference, Xiao Za changed facebook to Meta and ran all out to the metaverse. In the eyes of many people, this is just a PPT of the metaverse. In fact, Meta has been working on the Codec Avatars project, and has been able to create realistic virtual characters, even the hair is rooted.
Zuckerberg has also come to challenge the "lightsaber disguise".
You think so? NoNoNo!
Xiao Za is not as cool as Tang Yixin, but it is a little more funny.
The sword is hidden behind the right hand, and the sword is bright in the left hand: a bottle of BBQ dipping sauce.
This is Xiao Za's avatar in the "Meta universe", and this bright sword video has been forwarded by many people.
Since Facebook changed its name to Meta and officially entered the metaverse, the popularity of the term "metaverse" has not diminished.
Along with the "metaverse", the word "Avatar" is frequently mentioned.
"Avatar", in the popular sense, is the image of people in the virtual world.
On December 10th, Meta, the pioneer of the "meta-universe", opened its own virtual world platform Horizon Worlds.
When users enter Horizon Worlds, they need to create a 3D avatar.
With avatars, users can play in this custom world.
Slightly embarrassed, though, the avatars were very stiff.
So, while the current 3D avatar is barely usable, it's still 108,000 miles away from sci-fi movies, such as the realistic avatars in "Head Player."
The Codec Avatars project represents Meta's efforts in the field of virtualization.
"Meta universe" is not APT, Meta is really doing it
Xiao Za's determination to change his name to Facebook is not a joke either.
If you want to achieve true interaction in the metaverse, the avatar can be the second vein of Ren Dou who opens the door of the metaverse.
The Codec Avatars project aims to implement a system capable of capturing and representing realistic avatars for XR.
Based on 3D capture and AI technology, Codec Avatars allows people to quickly and easily build their own avatars and make interactions in virtual spaces as natural as they would be in the real world.
The project started with a high-quality avatar demo and gradually realized the virtual avatar of the whole body.
Aside from simply scanning a person's body, a major challenge is getting it to move in a realistic way — not to mention getting the entire system to run in real time so that avatars can be used in interactive environments.
At Connect 2021, researcher Yaser Sheikh presented the team's latest achievement, the Full-body Codec Avatars.
He said Codec Avatars now support more complex eye movements, facial expressions, hand and body postures.
In addition, Meta shows the virtual avatar to achieve realistic rendering of hair and skin in different lighting conditions and environments.
Then, clothes can also achieve real simulations in the metaverse.
The elasticity is good when you pull it, and there will be wrinkles when you shake it.
Body details, clothes can be realistic restoration, and high-fidelity virtual space.
Meta built a virtual apartment where everything in the apartment was a reflection of the real world.
In this way, the user can move objects or interact in the virtual space in synchronization with the physical space.
For now, this seems to be a "best practice" scenario for companies to build a mapping real-world environment for experimentation.
Once in the metaverse, manual tracking is equally important. If you interact with a big arm in the physical world, it is not only tiring, but also unsightly.
To this end, Meta has been working on a more subtle and natural method, which is to make XR input through the EMG wristband.
Some time ago, Meta also launched a haptic activation glove, all in an effort to achieve new interactions in the metaverse.
How do I build Codex Avatars?
Meta's start of making Codex Avatars dates back 7 years.
In 2014, Yaser Sheikh, head of Panoptic Studio, a 3D capture lab owned by carnegie Mellon University's Robotics Research Institute, met Michael Abrash, chief scientist at Oculus, and the two had a very speculative conversation.
Michael Abrash
Yaser Sheikh joined Facebook (now Meta) in 2015 and has been leading the Codex Avatars research team ever since.
Yaser Sheikh
"The foundation for creating a realistic avatar is measurement," says Thomas Simon, a research scientist at Codex Avatars.
"Avatars need to rely on accurate data, which requires good measurements. Therefore, the key to constructing a real avatar is to find a way to measure the physical details in human expressions, such as the way a person squints his eyes or wrinkles his nose."
The Codec Avatars team at the Pittsburgh Lab used two main modules to measure human expressions: encoders and decoders.
First, the encoder uses a camera to capture what the subject is doing, and then assembles the captured information into a unique code that represents a person's physical and environmental state, which can be sent wherever it is needed at any time.
The decoder then converts the encoding to a video signal.
Ordinary smartphone cameras can also capture vivid videos. However, Codec Avatars is a combination of massive amounts of physical data and complex software, which is much more complex than expected.
The Codec Avatars project uses hundreds of high-resolution cameras to capture face details, each of which captures data at a rate of 1 GB per second.
What is this concept?
For example, three seconds of recorded data from these cameras is enough to fill a 512GB disk.
The Codec Avatars team continued to record for about 15 minutes, which was already at the limit of their existing hardware conditions, but as much data as possible also laid the groundwork for creating the most realistic virtual avatars.
The Codec Avatars team is using this data to train the AI system so that the AI can quickly build virtual avatars from the video.
Sheikh said: "We can now restore the 3D avatars of people with exaggerated hairstyles, and even the headshots of people wearing EEG collection hats."
"This is one of the most advanced methods in the world to automatically generate a real digital portrait of a human being." It offers a new way of virtual face-to-face communication that can be widely promoted in the future."
With the advent of Codec Avatars, will people become more and more receptive to the "metaverse" society?
Resources:
https://www.roadtovr.com/meta-reality-labs-2021-research-and-development/
https://www.theverge.com/2021/10/28/22751177/facebook-meta-codec-avatar-real-time-environment-rendering-neural-interface https://tech.fb.com/codec-avatars-facebook-reality-labs/ http://www.diankeji.com/vr/47334.html
https://gizmodo.com/mark-zuckerberg-is-a-bbq-sauce-obsessed-killer-in-new-m-1848135459