If the gap between the camera and the camera is the most obvious, it should be the shallow depth of field shooting effect. The effect of shallow depth of field, that is, we often say that the background bokeh effect is not so simple, the shallow depth of field effect mainly relies on large aperture and large size photosensitive elements, which are contradictory to the small size of the mobile phone itself, so it still needs to rely on algorithms to achieve.
At present, the method of realizing the shallow depth of field effect of the mobile phone is still to use the keying algorithm to shoot the focus subject independently, and then the other areas are blurred and simulated to obtain a photo similar to the optical shallow depth of field effect. Therefore, the most critical technology here is keying, in the portrait shooting scene, the most difficult to "cut" is the hair area, because there is almost no regularity and the hair is extremely thin, many mobile phone algorithms will be exposed in this area. In response to these scenarios, Google, which has always been known for algorithms, recently released a set of solutions for its own Pixel 6 series mobile phones.
According to Google AI Blog, this new portrait mode solution uses Alpha Matting technology, which was previously used in film production for accurate keying and background replacement. Google fed the original RGB photo as well as the alpha mask low-resolution keying into the neural network, using MobileNetV3 to predict higher resolution, more accurate masks, helping subjects separate from the background.
Judging from the comparison chart of the effect published by the blog, the keying improvement of this set of algorithms is still immediate. The sample model's hairstyle is currently "hellish" difficulty for any mobile phone, and Google can finely outline the edges of the curls, an effect that has never been seen before. As can be seen through the locally enlarged contrast chart, due to the more fine cutout, the depth of field at the edge of the hair is also quite precise, almost with a fake real effect.
In addition, to match the light of the keyed subject to the background, Google also used a spherical device for sample collection, which is equipped with 331 custom color LED lights, a series of high-resolution cameras and a set of custom high-resolution depth sensors. Using a variety of angles of lighting and camera shooting to simulate what kind of light effect the portrait should have under different lights.
In response to this improvement, Google also made a demonstration. In the demonstration, Google will enter the image after a series of processing, and recombined into a new background, the overall face lighting conditions and background is more consistent, compared to the original picture, its light in the background will not appear particularly abrupt.
It is worth mentioning that this year's Pixel 6 mobile phone is the most powerful generation of camera hardware google has ever had, before that, Google has been using the rather old CMOS algorithm optimization, and its actual imaging effect, although still excellent, is still difficult to match the latest top camera flagship. Perhaps also aware that the gap in hardware is difficult to cross domains, Google used a large-footed main camera on the Pixel 6 to improve the overall shooting ability. But the actual market feedback does not seem to be able to match the expectations of the outside world, which seems to be a strong evidence of the importance of computational photography.
Although the hardware of the mobile phone is important, in the automatic mode, the algorithm is still the most powerful reliance on the user to get the quality shooting effect. Previously, there was news that key members of the Google mobile phone computing photography team had left, and this new portrait algorithm obviously gave Google mobile phone users a confidence, but at present, the content we have collected is only Google's official test, and the specific effect still needs to wait for the Pixel 6 series to be installed, and then judged by external testing.