Yang Jing Xiao Zhen was sent from The Temple of Oufei
Qubits | Official account QbitAI
Can you believe that in order to make your phone directly shoot a blockbuster effect, engineers have done this kind of thing:
Invest 300+ manpower, and make a core.
Previously, they went from zero to one and spent two years to make the night scenes clearer.
This time, in only half a year, they iterated out the Plus version and upgraded the portrait to a new level...
When did the phone's imaging system get to this point?
All along, the image system has been the protagonist of the major mobile phone manufacturers at the conference, which is beyond reproach, after all, this is the most intuitive feeling for users.
But at first, it was just a matter of piecing pixels and comparing the number of cameras. Or more advanced, is to see what personalized algorithms, such as beauty to distinguish between boys and girls, scene recognition, night scene noise reduction and so on.
Now how can there be manufacturers specifically developed chips.
And this trend continues unabated, and has begun to iterate year by year mode.
Why does mobile phone image exploration now rely on chips? What else is there to play?
The mobile phone image gradually enters the inner volume
To understand the "inner volume" logic of mobile phone images, we need to focus on the entire history of mobile phone images.
Since the birth of the world's first camera-equipped mobile phone, the Sharp J-SH04 (2000), imaging has become a new track for mobile phone manufacturers to compete.
From the technical perspective, image upgrade is nothing more than hardware and software:
The hardware includes cameras, sensors, camera modules, chips, etc.; software is the algorithm.
However, from the perspective of time dimension, mobile phone images can be said to be a process of returning to the original.
On the surface, the first dozen years of mobile phone images have been in the era of hardware "inner volume".
At that time, the idea of various manufacturers was to simulate the camera, surpass and replace the camera.
The first to start the inner volume is the CMOS material as the representative of the image sensor, especially the 2010 iPhone 4 equipped with a back-illuminated sensor, and later directly increased the pixels of major mobile phones to tens of millions of levels.
Theoretically, noise can be greatly reduced when shooting in low light. And with the larger the CMOS area, the clearer the image, which is often said in the digital circle "the bottom of the first level crushes the dead", "the bottom is king".
But the space left for CMOS by mobile phones is really limited, and it is obviously not feasible to achieve the effect of the camera from a physical point of view (the sensor area becomes larger) in the short term.
After that, even if there was a breakthrough in the quality and quantity of cameras, they quickly became the standard of mobile phones, and did not stir up too many splashes in the industry.
At this time, some manufacturers began to focus on the advantages of the mobile phone itself, to create a technology suitable for mobile phone shooting scenes, and also explored a number of different paths.
In the end, it turns out that what can really differentiate mobile phone images is in the camera algorithm:
Auto HDR, night selfie, super portrait, big aperture simulation, stage mode, 4K video...
These overwhelmed functions, I believe you are more or less aware.
At this time, the imaging process of the mobile phone is no longer as direct as the camera, but before the final picture is presented, it must be optimized by real-time calculation of ISP, DSP, DPU and other processors.
This also brings challenges to the computing power of mobile phones, especially in the face of multiple cameras, and the amount of computed data is exponentially increased.
If the computing power cannot be supported, no matter how good the algorithm is, the user cannot perceive it.
At this time, there are two paths left for mobile phone manufacturers, one relies on the upstream SoC platform upgrade to optimize the algorithm, and the other is to actively develop compatible image chips to meet the implementation of their own algorithms.
One is very passive and may face incompatibility dilemmas; the other is high risk, and the cost of failure is unimaginable.
Obviously, the second is a more difficult and correct path, and it is also a way for mainstream manufacturers to choose.
If the limitations of the sensor force the mobile phone manufacturers to think about the possibility of the mobile phone itself; then, the limitations of the algorithm are to force the mobile phone manufacturers to return to the original thinking -
Understand exactly what users need to give a more personalized algorithmic experience.
Last year, the trend of this self-developed image chip was very obvious, of which vivo, the first echelon of mobile phone imaging, was the most representative. They took the lead in equipping their flagship X70 series with a self-developed V1 chip, and also proposed an industry benchmark - "dual-core standard".
The V1 chip truly solves a series of industry-recognized imaging technology bottlenecks, including real-time night scene shooting.
With the blessing of vivo's self-developed chip V1, the mobile phone can run the denoising and interpolation of real-time ultra-high-definition night scene shooting at low power consumption.
In fact, there are many cases of solving technical bottlenecks through self-developed chips.
And with the recent release of the vivo V1+ chip, we found that the possibilities brought by the self-developed image chip to the mobile phone are far from limited to the image itself.
What's the point of this Plus version?
The newly released V1+ chip has become an ASIC (Application-Specific Integrated Circuit).
Compared with the V1 chip, which focuses more on image processing, V1+ has to be improved compared to V1 in terms of performance and functionality.
Let's start with the performance parameters.
V1+ has an average data throughput speed of 8GB/s, theoretically up to 25GB/s, reduced power consumption by 72% (V1 is about 50% lower), and energy efficiency is increased by 3 times.
What is this concept?
Intuitively, this allows the phone to run more high-performance algorithms that were "unbearable" in the past.
For example, the three most performance-consuming algorithms in mobile phone images can be hard-hardcoaled into the V1+ chip in one go:
3D real-time stereoscopic night scene noise reduction, MEMC interpolation and AI super-resolution algorithms.
Among them, MEMC interpolation and real-time night scene noise reduction continue from the V1 chip, which not only allows the mobile phone to run the real-time denoising and interpolation of ultra-high-definition night scene shooting at low power, but also through further upgrading, directly use the algorithm to build the mobile phone into a professional-level "night vision".
This feature allows the phone to shoot real-time video in an extremely dark environment with a illuminance of less than 1lux (the illumination required for regular reading is about 300~400lux).
As for the AI super-resolution algorithm, it is the first implementation of vivo on the mobile side, which makes V1+ jump out of everyone's cognition of its "image processing".
Well, now the pressure comes to the mobile game manufacturers (manual dog head).
So the question is, how exactly does the V1+ chip do the above functions?
On the one hand, V1+ has optimized its own architecture, such as integrating an SRAM on the chip to alleviate the memory wall, reduce the power consumption of access memory, and further improve energy efficiency;
On the other hand, the V1+ chip is not "fighting alone" this time, but has carried out dual-core joint debugging with the Tianji 9000 chip.
The commissioning took nearly 350 days, the team size reached more than 300 people, and the whole process produced more than 30 patents.
I also played the image with "permutations"
Looking at the chip, Vivo also developed more image algorithms this time, and "arranged and combined" them to make more interesting functions.
In terms of portraits alone, Vivo said that "behind every portrait photo, there are dozens of complex algorithms hidden behind it."
The beauty of several people is just a small fight, vivo even directly for multi-person scenes to engage in a "texture portrait group photo" function, the maximum support for the simultaneous processing of 30 people.
In other words, ai face ultra-clear, three-dimensional skin color and skin texture optimization multiple algorithms can run for up to 30 people at the same time, and the algorithm performance (execution time, memory occupation, etc.) is up to 4 times higher than before optimization.
This way, you can see your face even if you stand on the edge of the group photo.
In addition to portraits, vivo has also developed functions such as "horizon photos", which run algorithms such as real-time position detection and stabilization correction compensation at the same time. After filming in this way, you can directly send a circle of friends (dog head):
In addition to these algorithms that make photos "look better", vivo also further improved the "accuracy" of photographic photos with the help of ZEISS Natural Colors 2.0 this time, of which the brightness accuracy can be increased by 16%, and the white balance can be increased by up to 12%.
Based on the AI perception engine, Vivo has developed the "intelligent white plus black minus" and "intelligent automatic white balance" technology, the former is to get a photo with the right exposure, and the latter reduces the influence of the light source on the color of the object, which was originally adjusted manually by the photography "old mages".
In summary, from the perspective of performance and function, vivo does not intend to benchmark the "professional camera" with parameters as most mobile phone image inner volumes in the industry.
After all, limited by factors such as hardware, the parameters of the benchmark professional camera, is not the "specialty" of the mobile phone camera, in contrast, the mobile phone image should think more about how to play a differentiated route.
In fact, compared with professional cameras, mobile phones are both competitors (taking pictures) and referees (displaying) of images, and should have a greater voice in the display.
Vivo perceives this, in addition to improving the image performance, but also improves the effect of the screen display.
Extend the color gamut of photography and display to display P3, and expand the color range by 25%; Android's self-developed XDR Photo technology can increase the brightness of the high-brightness area of the photo by up to 350%, truly displaying the effect of the image on the screen.
Yes, in addition to the lens and algorithm, vivo even the screen is actively rolled up, which can be described as copying all the possible space of the mobile phone image and rolling it up.
Looking at the entire development process of domestic flagship machines, such desperate "inner rolls" are not uncommon.
There is not much room left for domestic flagships
Looking back at the entire process of mobile phone technology development, "inner volume" seems to be a very common phenomenon.
It does not appear only in the image, but in every "technical domain" on the phone.
Starting from the hardware performance of mobile phones, the memory evolved from tens of MB to 4GB to 8GB, and the number of chip cores was stacked more and more; to the later screen display, from 1080p to 2k to the 4k developed by some manufacturers now, the clarity was getting higher and higher; and then to various innovative functions...
Almost every time a new field is developed, manufacturers will flock to the development of related technologies, lest they be left behind.
This has also become a default development direction for mobile phone manufacturers to "tacitly":
As long as the new technology is followed up in a timely enough manner, the parameters are qualified enough, and the material is piled up enough, it looks like a "top mobile phone in the industry", and the user will buy it.
It can be perceived that once each technology approaches the mature pole, the speed of the "inner volume" will be accelerated, and the image "inner volume" is only the embodiment of this process.
On the one hand, the cost of new technology research and development is high, the opportunity for trial and error is small, and once vigorously invested in research and development, the consequences are likely to be input and output, just like the image sensor path that was stacked with small pixels but eventually disappeared.
On the other hand, compared with self-developed mobile phone technology, factors such as market and sales volume are often more likely to become the focus of attention of mobile phone manufacturers.
Instead of developing more technologies or holding more patents in your hands, it is obviously a more secure way to continuously launch new machines based on mature, market-proven technologies, or fight price wars.
However, the result of such "inner volume" also leads to mobile phones not being really competitive, and once the market changes, it may be left behind.
For example, last year, affected by factors such as lack of cores, at the end of last year, Apple successfully occupied the top spot in the Chinese mobile phone market with a market share of 23%.
In this case, there is not much room left for domestic flagships to make changes.
Whether to continue to maintain the "inner volume" of parameters, or to find a way to innovate on existing technologies, manufacturers have their own ideas:
For example, the launch of flagship sub-brands, or the separation of low-end models and high-end models, or the division of a piece of business dedicated to flagships...
But no matter how the brand-based change is, as long as the new technology is not really launched, the essential path is still the same: return to the inner volume.
In this case, taking the user demand as the guide and actively developing new technologies may be the only way to get rid of the inner volume.
Looking back at the development process of the vivo X series, it is not uncommon for this case of self-research to rush out the "inner volume":
Hi-Fi music phone, 2K screen, dual 2.5D curved glass screen, front-facing soft light dual camera, off-screen fingerprint, micro-gimbal stabilization lens...
What you can see is that they are not rolling for the sake of industry rolls.
Because compared to the "blood road" rolled out by stacking hardware and parameters in the mobile phone market, vivo's choice is more like a win-win result between manufacturers and users - manufacturers roll themselves, and users benefit.
After all, the ultimate demand for any product is in the user. Whether the product is good or not, the user knows that the closed loop between the user and the manufacturer is formed in this way, and the industry can develop healthily and sustainably.
Today, this V1+ chip, which is about to be used in the X80 series, may also be a new path that vivo has tried again to break the "inner volume".