FBEC2021| Founder of Huanchuang Technology Zhou Kun: From binocular to quad-eyed is a trend

On December 10, 2021, under the guidance of Guangdong Game Industry Association, Guangdong Virtual Reality Industry Technology Innovation Alliance, Shenzhen Science and Technology Association, Shenzhen Internet Culture Market Association, hosted by Gyro Technology, co-organized by Shenzhen Science and Technology Development Exchange Center and Hengyue Maker Cube, and co-sponsored by the industry's leading media games Gyro, VR Gyro, Gyro E-sports, Gyro Finance, Gyro Media (FBEC2021) Grand opening at the Grand Opening of the Ballroom on the 6th Floor of the Sheraton Shenzhen Greater China Hotel!

At the scene of China Power 2021 5G XR Industry Summit, Zhou Kun, founder of Huanchuang Technology, delivered a speech with the theme of "From Binocular to Quadrangle, the Development of XR Spatial Positioning Technology".

The following is a transcript of the speech:

The title of my speech today is "From Binocular to Tetramuscular, the Development of XR Spatial Positioning Technology", which consists of four parts:

First, let's introduce the technology of AR/VR visual spatial positioning. Spatial positioning is mainly divided into two technical schools of external positioning and self-positioning, external positioning is called Exterior-in, which installs sensors and cameras on the outside to observe active objects or people from the outside to the inside. Its features: installation and use are more complicated, but the accuracy is relatively high, the viewing angle is limited, and it is easy to be obscured. More common like OptiTrack, the application areas are film and television motion capture, VR, industrial robots and so on.

The second is called self-positioning (inside-out), the camera or sensor is often set on the helmet, from the inside out to observe the vision, such as vSlam, characterized by portability, infinite angle of view theory, relatively low positioning accuracy, the application field is mobile robots and so on. In terms of VR, the first generation of Oculus Rift is the first generation of Oculus Rift, which uses external positioning technology to do positioning, installs the camera on the outside of the application environment, and locates the spatial position of the head and hand by successively observing the marker on the object worn by the user's head or hand.

In the past few years, it has been gradually replaced by The Side-Out, and the main reason for the replacement is the cost and ease of installation. Over the years, more and more companies have adopted self-positioning. There are two subdivided genres for self-targeting schemes:

One is based on Marker, hand positioning is Marker, characterized by high positioning accuracy, suitable for high-speed and high-precision positioning scenes;

The other is based on no marker, the positioning of the head is the way of no marker, the installation is more convenient, the accuracy is lower, and it is easy to be affected by the environment. Comparing external positioning and self-positioning, there are many indicators that illustrate the difference between them.

From the perspective of accuracy, the external positioning is higher and can be done in millimeters; self-positioning can only be done in centimeters. The delay is about the same. Movable range, external positioning must be arranged in the environment a circle of cameras, on this basis in order to achieve external positioning; cost, external positioning of the installation cost and use cost is relatively high, more troublesome.

Self-positioning is very convenient, so it is very popular on C-end applications, which is why Inside-Out is becoming more and more popular. External sensor, external positioning needs to set the camera, self-positioning does not need; anti-occlusion, external positioning will be affected, and self-positioning is not affected.

Tell us about some of the company's accumulation on inside-Out. The company began to study the positioning of AR/VR in 2015, began to study the external positioning, and later switched to self-positioning, we based on the binocular self-positioning technology, in the study of Inside-Out when we faced several aspects of the challenge.

First, the design of the light belt scheme on the handle is very difficult, don't underestimate the small light band above Oculus, there are many LEDs, the number of LEDs, the layout are exquisite, and its size and size are critical. People who do technology like to design the light belt a little bigger, a little bigger is easy to identify, but as a consumer scene is not very good, installation is not convenient, it is not beautiful to use, and the sales of C-end products are certainly unfavorable, so a balance needs to be formed between the two, and we as a technology company, we must always pay attention to the problem of patents when developing. Because AR/VR is a stage of global competition, Oculus and Microsoft have been laying out in this field for many years, so it is necessary to pay attention to the circumvention of patents, which is the first difficulty.

Second, the problem of initialization. Because the biggest pain point and difficulty of the CV scheme is the problem of the field of view, the Controller will exceed the line of sight, after returning from the range, it is necessary to initialize the camera, and if the initialization is not done well, it will delay initialization or caton, and it is best to complete the initialization work within one frame of the camera. This is very demanding for the initialization algorithm, which requires fast initialization so that the consumer does not feel the delay and stuttering, and quickly and naturally tracks the correct position of the handle.

Third, the handling of multiple Controller crossover, overlap, and occlusion cannot be allowed to drift and error after overlapping, which is a very big problem in algorithm design.

The front of these are five-star puzzles. There are three more puzzles behind:

First, in order to reduce costs and power consumption, it is impossible to separate the head tracking Camera and the hand Camera, Camera must achieve both head tracking and hand tracking, to be very multiplexed, which needs to be switched, which must avoid the problem of patents;

Second, the problem of high-speed tracking, the Controller in the hand reaches 7 seconds at the fastest, under the high-speed movement, whether the algorithm of our hand is suitable for the posture of the movement;

Third, move the algorithm to the problem of embedding the platform. Now we have to run all-in-one machines, platforms like Qualcomm and Kirin HiSilicon, the processing performance is still different, how to let the algorithm run to the platform, no longer occupy the CPU, you need to use the DSP well, this is also a challenge.

Now that the first generation of XR binocular positioning scheme has been developed, the launched product has realized inside-Out binocular tracking, THE FOV is 170 degrees, and it can be used at low cost, and the Camera microcomputer distribution can be.

This is the parameter we get in the process of practice, including the number of sensors, resolution, frame rate, field of view, maximum distance, dynamic angle, accuracy, dynamic position accuracy, budget delay, and the actual power consumption and CPU usage occupied, which is the result of our actual measurements.

Starting from the two eyes to the four eyes is a trend. Because we found a problem in the process of research and development and docking with customers: the binocular has a limitation, the FOV is not large enough, if the FOV is small, it will continue to exceed the field of vision of the FOV, resulting in users often encounter initialization problems.

In short, its effect experience is not as good as the four-eye solution, so we evolve to the four-eye solution. But evolution is not simply to replace two cameras with four cameras, stack the cameras, and there are many problems to deal with. Because the four eyes expand the range, so the overlapping area is smaller, the distortion of the lens is very strong, how to do it? It is difficult to achieve very high-precision positioning. Second, the amount of computation is much larger, the eyes have become four eyes, and the amount of exercise has doubled, but the amount of processing for the four purposes cannot be very large.

The performance was improved, the FOV changed from 170 degrees to 220 degrees, the vertical viewing angle was doubled, and the dynamic angle positioning error and position positioning error were also greatly optimized.

Let's take a look at the comparison between binocular and tetraphrasic, this is our demonstration video, the left is binocular positioning, and the right is the tetramuthic positioning scheme. On the left side of each image is the original image, on the right is the six-axis posture it restores, and we will take a look at the video of the demonstration.

During the demonstration, the user will have a variety of actions, such as occlusion, crossing, overlapping, and will also put the hand to the rear and then take it out, so that we can implement the initialization process. The tracking system is still able to track its effect and position very flexibly and stably. On the right is our four-purpose tracking system, which greatly expands its range by moving up, which makes the user experience better, avoids the problem of using IMU to track after it exceeds this baseline, and makes its experience more smooth and coherent.

In addition, the problems to be solved also include that users will encounter ambient light interference during use or participate in the exhibition, which needs to be handled relatively well in our algorithm. This is the result of our comparison, from the field of view to the positioning accuracy to the delay, you can see that we have a big improvement.

Our company was established in 2014, is a focus on high-precision positioning sensors company, since the establishment of the company has been doing high-precision positioning algorithms and algorithm chip research and development. At present, our main product line is the product line at the robot level, doing lidar, and the chips inside are also designed by us.

At present, we are the positioning of the head supplier in the industry. At the same time, the second line is in the field of AR/VR, starting to do AR/VR work in 2015, and cooperating with iQIYI in 2015 to provide them with external positioning solutions. At the beginning of this year, 200 million series B financing was completed.

This is the main business of our company:

First, the laser radar in the robot, including the three major manufacturers here, Stone, Coworth, xiaomi, two of which are my customers, shipments reached nearly 2 million units this year;

Second, in the field of VR, it specializes in providing customers with high-precision 6DoF modules;

Third, the industrial medical field provides high-precision positioning systems, and Commercial Aircraft Corporation of China is also our user.

This is our company's vision to become a global leader in computer vision spatial positioning.