Based on a specific frequency band coaxial light source, Apple AR/VR headset eye tracking patent exposure

On December 14, the United States Patent and Trademark Office published an Apple patent related to eye tracking for AR/VR headsets, which pointed out a scheme to detect pupils by adjusting coaxial illumination. The patent was inventored by Walter Nistico, Apple's deep learning and computer vision engineering manager, who previously worked for SM SensoMotoric Instruments, a computer vision company acquired by Apple. Prior to joining Apple, Nistico also worked with Google, Qualcomm, Sony, Nvidia and Intel on AR/VR projects.

According to Qingting Network, the scheme described in the patent is based on coaxial illumination technology, which calculates the characteristics of the eyeball through information such as visual field position, pupil shape, pupil diameter, pupil center, etc., such as eyeball shape, position and fixation direction. Coaxial here refers to the fact that the light source for a particular pulse band is about the same axis as the image sensor, which captures the light reflected back on the eyeball by the light source, which will carry information about the eyeball.

Apple pointed out that existing eye-tracking technology uses image sensors to generate grayscale images of the eyeball by identifying light intensity during exposure, and to detect pupils by comparing the difference in grayscale between the pupillary and iris regions in the image. However, the accuracy or effectiveness of this eye-tracking technique is not good enough, especially when the contrast between the pupil and the iris is not high. In contrast, the image sensor for Apple's coaxial illumination scheme captures brighter pupil images.