A new way of interacting with VR is coming! The natural gesture recognition technology all-in-one machine program "Qi Ji" was launched

With the upgrading of technology in the VR industry, more natural interaction methods have also followed, and natural gesture recognition and tracking technology has attracted the attention of ToB and ToC with its more in line with people's real operating habits. The upgrade of gesture recognition technology can also facilitate the upstream and downstream relationships, increase the creative space of content developers, and increase the fun of the game to make its interaction mode more improved.

The 6DoF all-in-one solution "Qiji" developed by Quli has been equipped with the latest natural gesture recognition system, and the collaborators can directly call this module in the system to finally complete the recognition task of natural gestures. Quli Technology has also become one of the few companies in the domestic VR industry that has a natural gesture recognition product solution, and only Quest currently has this technology in the world.

Natural gestures under the Qi Ji equipment play the sound game

Fundamental technical principles

[Natural Gesture Recognition] is based on computer vision and artificial intelligence algorithms, which detects the user's hand area in the image collected by the camera and estimates its posture, so as to obtain full degrees of freedom natural gesture tracking technology. In layman's terms, it can be divided into 3 steps: first, what position is the hand to solve; second, what kind of posture is the solution hand; third, to identify what the hand is doing? The first two items are often put together to make gesture predictions, and finally to make time series divisions for moving and static gestures, and complete the final instruction implementation.

"Qi Ji" - a static gesture

From the hardware point of view, the detection equipment used in the common gesture recognition schemes on the market is roughly divided into four categories: pure infrared cameras, grayscale cameras, RGB cameras and depth cameras.

Each scheme has obvious advantages and disadvantages, such as depth cameras, although the depth camera can be based on depth information than other methods to obtain the position and shape of the hand area more simply, but the FOV is very small, the frame rate is very low, the cost is high, the power consumption is larger, and the current status of the VR all-in-one machine, gesture recognition and SLAM camera need to be reused, and the depth camera is difficult to use to do the SLAM algorithm, so there will be limitations in the field of XR application scenarios.

The principle of infrared camera is to build an infrared LED light source into the module, and the light is reflected back into the infrared camera of the gesture recognition module to achieve gesture recognition based on optical data. Due to the use of active light sources, it has good environmental adaptability and is relatively easy to detect the hand area, but it will increase power consumption at the same time, especially with the increase of the range of use of the opponent, requiring more and greater power consumption of infrared light, which is a very big challenge for the overall ID design and cost. This solution is better known for Its Leap Motion.

For the scheme of grayscale cameras and RGB cameras, RGB cameras have a better recognition effect because they can better distinguish between hand and environment with color, but there are also problems such as small FOV and low frame rate. The advantage of grayscale camera is that THE FOV is relatively large, the frame rate is high, the coverage is large, the adaptability is strong, and the cost is relatively controllable, and the disadvantage is that the difficulty coefficient is high, because in the world of black and white gray, the hand and the complex environmental background are accurately and quickly separated, and it is necessary to make sufficient efforts in the algorithm.

In The new 6DoF solution of Quli Technology, "Qiji", the fusion is based on the grayscale camera of the existing all-in-one machine for multiplexing. Quli Technology relies on the background of The Group's Yingchuang Technology, and has many years of experience accumulation and precipitation in the implementation of algorithms.

Natural gesture recognition operation of Qiji equipment

This gesture recognition technology algorithm after many periods of polishing and tuning, has now reached a common state, the algorithm framework can be applied to any camera equipment, in the future products, the new generation of VR all-in-one pc solutions may use color RGB camera and grayscale camera collocation (color is responsible for perspective, grayscale is responsible for 6DoF tracking). The essence of natural gesture recognition will still use a grayscale camera based on 6DoF, and the color camera is used as an auxiliary to supplement the accuracy of the opponent when stripped from the environment during the recognition process.

The difficulty of the whole gesture recognition technology is that the recognition of the hand and the face and the human body is different, because the complexity of the hand is relatively high, the self-occlusion is serious, the hand can move very quickly and the appearance can change significantly within a few frames. At present, the "Qiji" solution can achieve high accuracy and low latency because it relies on the group to build a self-developed data deep collection and labeling platform, based on the platform has a lot of reference data, and at the same time has self-research results in the sequence of the deep model, and has precipitated more than 6 years of technical accumulation and huge data reference value.

Future application scenarios

【C-end interactive entertainment】

The development of gesture recognition technology can better broaden the imagination space of content developers, and increase multiple gameplay and interactive experiences in content creation, such as directly operating by hand when playing music games, or a specific gesture in an RPG game can achieve magic attacks, and even in the future, a set of gestures can also complete the seal in "Naruto".

【B-end education office】

The application scenarios of the B-side are more extensive. In terms of intelligent office, as VR equipment gradually moves closer to office tools, natural gesture recognition can better complete the display of the presentation plan speech through specific enlargement and reduction of drag and drop gestures, and realize the real sense of portable office environment; and in education and teaching, it is also better to directly complete the disassembly and assembly of teaching courseware, combination and other requirements of teaching courseware with hand recognition technology, and more conveniently and effectively complete the operation interaction, I believe that in the future, with the further development of natural gesture recognition, it will be able to spawn more application scenarios. At the same time, we can also tailor our own solutions according to different needs.

In the future, Quli Technology will integrate the multi-party resources of Yingchuang Headquarters, gradually integrate more front-end technologies into its original solutions, better serve upstream and downstream partners, and provide them with all-round comprehensive technical support.

*Disclaimer: The corporate press release is not the official draft of the VR gyro, and the legal issues are not related to the VR gyroscope.