Recently, Microsoft's Mixed Reality & AI Lab research team developed the "FLAG: Flow-based Avatar Generation from Sparse Observations" solution based on the head and hand tracking data captured by the headset, and the head and hand tracking data obtained by the VR headset can generate the wearer's full-body 3D avatar.
It is understood that the tracking data obtained by headsets such as Microsoft HoloLens and Meta Quest is limited to the prediction of the position and direction of the head and hands, and cannot represent the whole body of the human body, so it is difficult to generate a realistic 3D avatar. The solution developed by the Microsoft research team can generate reasonable full-body human posture from these headset signals and solve this problem.
Specifically, the system analyzes a given head and hand movement data based on a flow model capable of achieving reversible mapping between the 3D pose distribution and the underlying distribution. In addition, it can not only learn the conditional distribution of the 3D human body, but also learn the probability mapping of the potential space from the observational data, and thus make uncertainty estimates of joints to generate reasonable postures.
It is worth mentioning that through experimental evaluation and ablation studies, this method is superior to the latest protocol of AMASS, a large motion capture dataset, which simulates multiple full-body postures in a SPPL mannequin, and is known for its extremely low error and less optimization.