The third-generation Snapdragon 8 mobile platform was released in October, which brought a huge ripple effect: Android flagship mobile phones were released in the fourth quarter of this year, and the competition was extremely fierce in terms of price and product power.
▲ Realme GT5 Pro lens module
For example, the ultra-cost-effective flagship mobile phone Realme GT5 Pro released some time ago provides a very sincere configuration in its early 3000s price segment:
- Snapdragon 8 Gen 3 (Snapdragon 8 Gen3)
- 12+256GB starting configuration
- Up to 4500nits brightness, dual-screen jointly developed with BOE
- 100W wired + 50W wireless charging
- Snapdragon 8 Gen3 + IMX890 + ArcSoft algorithm composed of ceiling-level telephoto
Wait, why did the Snapdragon 8 Gen3 appear twice?
SoCs are not only CPUs and GPUs, but also ISPs and so on
Ten or twenty years ago, when we talked about chips, the basic default was CPU, but the idea that chips are about equal to CPUs has long been outdated, because large chips have developed into platform-like SoCs (System on Chip), and small chips are even more varied, some are responsible for power management, some are responsible for screen display, and so on.
When we talk about SoCs, the focus tends to be on CPUs and GPUs, which contribute the most when it comes to score-sharing games.
When we hold up the phone, open the camera frame, press the shutter, and then get a photo, and then upload it to social networks, in this short process, the SoC is almost "multi-department collaboration", using a lot of modules.
In the SoC, there are many other parts that are not involved in the scoring of software scoring, but are crucial.
The Snapdragon 8 Gen3, which was recently released, is quite complex, with NPU and AI engine for AI computing, a modem for 5G connectivity, a FastConnect mobile connectivity system for Wi-Fi, Snapdragon Listening for audio, a hardware and software combination for security, and an ISP for image signal processing.
The ISP (Image Signal Processor) is one of the reasons why the Snapdragon 8 Gen3 is a recurring feature in the Realme GT5 Pro spec table.
▲ Schematic diagram of ISP operation, picture from CSDN
It is not enough to have a good lens and a good image sensor, but also the ISP needs to convert what the lens sees and what the sensor receives into what the screen can display and what the human eye can see.
This process is quite complicated, first of all, the light passes through the lens, incident on the photosensitive area of the image sensor, the sensor undergoes photoelectric conversion, and obtains the original image in Bayer format, that is, the image with the suffix .raw, and then the original image in Bayer format is processed by ISP, and the output spatial domain image data is sent to the subsequent video interface unit, and then supplied to the screen for display. The role of an ISP is to optimize, enhance, and process the image and video signals captured from the phone's camera, as well as to manage the sensor interface and subsequent processing of the image data.
The process is like a converter connecting the image system to the display system, but the ISP does a lot of work, including but not limited to image quality improvement, such as color enhancement, white balance adjustment, and noise reduction, which is crucial in night shooting, without a strong ISP to reduce noise and grain, the night scene of the flagship phone will be very mushy, in addition, including multi-frame compositing behind HDR shooting, slow motion shooting, stitching in panoramic mode, etc.
In addition, the low threshold for mobile photography also depends on the credit of ISPs, and many people may not be able to take a clear photo when they get a Leica medium format camera, because the focus, aperture, etc. need to be manually adjusted. On the mobile phone, the ISP uses real-time calculation, laser focus, light sensor and other components to achieve autofocus, auto exposure, auto aperture, auto white balance and other functions, helping users to take clear photos at the moment of turning on the mobile phone camera.
When we criticize some mobile phones for slow focus, poor white balance, auto HDR will still be overexposed, and the night scene image quality is not pure, it is likely that the problem lies with the weak ISP.
And the Snapdragon 8 series chip and imaging flagship phones are always accompanied by the main contributors, and of course, the ISPs hiding in them.
Therefore, the intention of the example of the Realme GT5 Pro at the beginning is here, because the combination of Sony IMX890 image sensor + Snapdragon 8 Gen3 + ArcSoft algorithm is of typical significance.
The Snapdragon 8 Gen3 ISP has 3 ISPs that support 3 cameras to shoot HDR video at the same time, with a color bit depth of 18-bit, a camera that supports up to 200 million pixels, and 8K 30fps or 4K 120fps video recording, and 720p 960fps for slow-motion video recording.....
This means that the Snapdragon 8 Gen3's ISP has very good data processing capabilities.
But that's not all the Snapdragon 8 Gen3 imaging capabilities are capable of.
AI is the magic of technology, and Snapdragon Sight is the mantra that makes photos better
If it was 5 years ago, some manufacturers said that their photos were at the SLR level and studio level, they would definitely be denounced and exaggerated by professionals and public players.
But in 2023, many manufacturers have begun to promote SLR-level portraits and studio-level portraits, and in some subdivided scenes, the performance of image flagship mobile phones seems to have touched the threshold of professional imaging equipment.
That's the power of computational photography, which allows moving images to see the taillights of professional images from bends, and the aperture is not wide enough. The focal length is not long enough, and the periscope zoom and then the digital zoom and then calculate the photographic optimization details can also get a good super telephoto photo.
But in 2023, the year of AI magic, the coupling of images and AI will naturally come naturally.
Snapdragon Sight Advances in Snapdragon imaging technology have also become more closely related to AI with the release of the Snapdragon 8 Gen3. The Snapdragon 8 Gen3's outstanding ISP performance and enhanced AI computing power make Snapdragon Sight even more powerful and become the technical foundation for many imaging flagships.
For example, Snapdragon Sight's DCG (Dual Conversion Gain) technology is a technology that can provide better image quality and faster imaging, which can be used for HDR imaging. Essentially, this is also a multi-frame compositing technique, but the multi-frame images of DCG are generated at the same time, rather than the pre- and post-generation of traditional multi-frame compositing.
There are also things like Night Vision mode, Dolby format support, which are essentially the result of ISP performance improvements.
However, technologies such as image expansion and object erasure also need the support of AI computing modules such as NPU.
Previously, we thought that these AI-like technologies were not part of the realm of imaging, but to some extent, these technologies that used to require complex operations on software like PhoneShop are now easy to access.
And the object erasure function is limited to pictures, and now the Snapdragon 8 Gen3 has begun to support object erasure in videos, and the keyword "computing power" is still at work behind it.
At the same time, with the blessing of AI, ISPs can evolve into cognitive ISPs, which can use Qualcomm's AI engine to perform real-time semantic segmentation when taking photos and videos, and identify various elements of images, such as buildings, boats, rivers, trees, people's faces, eyes, skin, etc. In the Snapdragon 8 Gen2, cognitive ISPs were able to recognize 8 layers of semantics in images, and even supported the ability to remove reflections from glasses, but now Snapdragon 8 Gen3 has increased the number of semantic layers that can be recognized in real time to 12 layers.
The more layers recognized, the clearer the understanding of the image and the greater the likelihood of post-enhancement.
For consumers, imaging technology is at an accelerated juncture in 2023 with the blessing of Snapdragon 8 Gen3: taking a good photo and a good video is becoming easier than ever, and at the same time, this photo and this video, under the magic of AI and with the blessing of Snapdragon Sight Snapdragon imaging technology, has more possibilities, which can be expanded, erased, and edited.
When we open the pages of many imaging flagship mobile phones and swipe to the video introduction section, we will continue to feel that "algorithms" are everywhere, and ISPs and NPUs support various "engines".
In fact, if you compare the technical preparation of Snapdragon Sight's Snapdragon imaging technology, you will find that it actually leaves a lot of redundancy for mobile photography at this stage. For example, 4K 120fps video shooting, 3 cameras simultaneous shooting and other functions have not yet been fully implemented, the reason, of course, the progress of upstream manufacturers must be ahead of the terminal manufacturers, to provide more potential for tapping, and not to become a constraint.
Looking ahead to 2024, the OPPO Find X7 series, vivo X100 Pro+, Xiaomi 14 Ultra, Samsung Galaxy S24 Ultra, etc., will further leverage the strength of Snapdragon 8 Gen3 and Snapdragon Sight imaging technology.
At that time, we will better understand the value of this core.