Reports from the Heart of the Machine
Author: Zenan
When we buy the next mobile phone, "can you open the light chase" may be an important selection criterion.
When it comes to the most well-known graphics rendering technology in today's video games, it may be ray tracing.
Whether it's Minecraft, World of Warcraft, or Battlefield, more and more well-known games are adding light chase effects. For players, though, there's a trade-off between image quality and fluency — turning on light chasing often means halving the frame rate, even on Nvidia GPUs with dedicated RTX cores.
"Chasing the light off" is the front-row solution for players after feeling stuck in the picture. It has been 3 years since practicalization, and light chase is still just the capital for high-end players to show off their hardware.
However, at the OPPO Developer Conference on October 27, OPPO conducted a demonstration of mobile phone light chasing, and the picture effects that have been requiring high hardware have been realized on mobile phones:
This is the first time we have seen such a fine picture effect on a mobile phone. Before ray tracing technology could be widely rolled out on PCs and consoles, it was turning to hundreds of millions of smartphones.
Wu Henggang, President of OPPO Software Engineering Division, said: "It is expected that in the first half of next year, we will be able to provide ray-traced capability access for hardware-supported mobile phone devices."
Seeing such a scene, we can't help but ask: Is this really a achievable thing? Is the ray tracing value worth it? To answer these questions, we have to start with the technology itself.
Ray Tracing: Effects like never before
The first computational application of ray tracing dates back to 1968 and was well known in 1980 by Turner Whitted's pioneering research. David Krik, a famous computer scientist and former chief scientist at Nvidia, once said, "There used to be a joke that ray tracing is the technology of the future, and this future will never come."
In May 2019, ACM SIGGRAPH, a top computational graphics institution, presented a Best Doctoral Dissertation Award. The laureate, Dr. Lingqi Yan, graduated from the University of California, Berkeley, and is currently an assistant professor at the University of California, Santa Barbara (UCSB). His doctoral dissertation speech goes like this: "Each chapter of his dissertation can be a doctoral dissertation in itself."
A chapter of Dr. Yan Lingqi's dissertation on global illumination eventually led to Ray Tracing, a "ray tracing technology" that is now commercially available, which has taken the picture quality of video games to a higher level.
The future has become a reality, with Nvidia's Proposed Turing graphics card, a single GPU can already achieve real-time picture light tracking rendering.
Ray tracing means setting up a light source and then simulating its projection effect: reflections, refractions, shadows, and so on appear in the natural environment. If people want to reproduce the real environment in the virtual world, they need to find ways to achieve the intuitive perception of these goals by the human eye.
In the current widely used methods, ray tracing probably follows the process of emitting a ray of light in virtual space, traversing the entire scene, looking for objects that may be ray-traced, processing differently according to different results, and then coloring it. In this process, we also need to perform recursion, that is, go back to the original step, emit more light, and get more reflections and shadows.
There are five new shaders in DirectX and Vulkan's ray tracing: Ray Generation, Intersection, No Intersection, Traditional, and Arbitrary Intersection Shaders to simulate reflection, refraction, transparent materials, and so on, respectively.
In short, light chasing is through a complex process, consuming a large amount of parallel computing power of the chip, trying to generate a lighting effect similar to the real world. As the scene continues to increase and the complexity of the picture increases, the effect of ray tracing will become more and more obvious.
At present, most of the applications of ray tracing technology are AAA game masterpieces. You'll find this option in the latest and hottest games, and you'll have to update the GPU driver if you want to open it.
Paladin 7's ray-tracing effect.
Although it is very difficult to calculate, in the pictures shown in many AAA masterpieces, we can find that opening the light chase is like opening the door to a new world.
How to cram light chase into your phone
Similar to the trajectory of the development of AI technology, ray tracing has also undergone theoretical formulation and improvement, and after many years, it has been gradually put into practice because of the improvement of the computing power of the chip. However, in recent years, the performance development of traditional silicon chips has gradually approached the physical limit, and it is no longer difficult for us to carry higher computing power by simply using smaller transistors.
How is ray tracing, which is difficult to achieve on desktop computers, applied by OPPO to mobile phones?
First of all, it is a real effect, in addition to the two demo videos displayed by OPPO at the developer conference, there are also real machine demonstrations in the exhibition area of the developer conference, and the light chase is achieved on the mass production mobile phones currently sold on the market.
OPPO shows the "realistic" reflection of light on smooth material surfaces, as well as more subtle shadows:
There are also the most obvious effects such as specular reflection and transparent body refraction. Ray tracing on PC games, almost all of which can be presented here, has reached a high enough frame rate (around 30 frames):
OPPO said that OPPO ray tracing is based on the Vulkan API, a technology for the next generation of mobile light tracking hardware, and has been prototyping on the mobile platform. This technology can clearly restore many details in the real world when the hardware computing power is limited, improving the picture quality experience. In addition to gaming, ray tracing technology can also be applied in 3D live wallpaper, image filters, AR and other fields.
At present, most of the graphics rendering of mobile phones uses rasterization, and 2D projection is used in the location where lighting needs to be expressed, although the low power consumption demand is guaranteed, but many times the performance effect is more limited.
Global ray tracing is obviously a task that is difficult for mobile phone computing power to carry, so OPPO chose the "hybrid rendering" method. Minor changes to the original rendering method do not pursue light tracking of each pixel, but focus on places where the effect is obvious, such as the shadow of the object, the gloss of the weapon, and so on. This way, you can improve the image effect and keep it lightweight.
In some scenes, we can use rasterization to generate preliminary images and then use ray tracing to create more realistic, moving effects.
On the road to implementation, unlike the previous hardware-bound ray tracing technology such as Nvidia and AMD, the open source optical tracking technology based on Vulkan is compatible with more devices in the industry standard, and it is also convenient for developers to access.
OPPO's core advantage is that it has a unified scene traversal and light intersection operation, not only supports Android mobile phones, but also supports a variety of platforms such as PCs, and the short-term goal is at least full support for MediaTek and Qualcomm platforms.
In addition to ray tracing and heterogeneous computing presented at the developer conference, the Phantom platform will include more features.
"An important reason for choosing to do it at this point in time is because the industry finally has a standard, which means that different platform vendors will promote the implementation of light chasing in a mutually compatible way," said Tian Yizhen, senior algorithm product manager at OPPO.
In less than a year, from research and development to landing
The ray tracing capabilities of the Phantom Platform come from the Seattle Center of OPPO American Research Institute, and in the course of less than a year of research, OPPO has cooperated with MediaTek, Unity, Tencent Tianmei Studio, etc. At present, OPPO has completed the technical demonstration of two scenes with Tianmei on the Call of Duty mobile game.
Prior to OPPO's technology demonstration, MediaTek proposed earlier this year that ray tracing could be achieved on the Tianji 1200. On ODC, the model showing the Light Chase Demo is the Reno6 Pro. In 2021, there will be mobile phone manufacturers to achieve the original expectations.
"Some game companies, app developers, and even chip providers like MediaTek are doing light chase, but they're doing it from their own perspectives," Mr. Tian said. "For OPPO, we can get specific requirements from game manufacturers in the process of co-development, and at the same time coordinate the hashrate invocation method to the hardware platform, which is the most suitable for completing the landing of this thing."
At present, in the game production process, the light chase pipeline is very different from the game pipeline, which reduces the willingness of developers to apply new technologies. Oppo's proposed phantom platform hopes to improve the graphics rendering capabilities of traditional pipelines in small steps, while greatly reducing the difficulty of use, and developers do not even need to know the implementation details inside the light chase.
Based on Vulkan, the industry's only ray-tracing standard, OPPO will provide developers with an open source interface and plug-ins to popular game engines such as Unity, a technology evolution that is suitable for future technological evolution and supports a wide range of hardware.
In the future, for developers, the light chase effect may be called an interface in the game design process, and only one key in the game engine can be opened on the OPPO mobile phone to directly optimize the existing scene.
For users, they do not need to update specific drivers, or download special game clients to get better light and shadow effects.
OPPO has become the first mobile phone manufacturer to implement ray tracing technology on mobile terminals, but it currently has no plans to launch a dedicated gaming phone: ray tracing, heterogeneous computing and other technologies are hoped to be available to all users.
Available to everyone and efficient enough, this is the goal of OPPO's optical tracking technology. It is expected that by the first half of next year, OPPO will provide this technology to developers through open application access. As early as 2023, there will be a light-chasing application on the mobile phone side.
Finally, if I'm a Protoss player, when will I be able to turn on the Light Pursuit option in the game? OPPO said: If Mihayou is looking forward to adding such special effects, they can put forward their own optimization needs to OPPO, and we look forward to working with game manufacturers and application developers as soon as possible."
This day should not be far away.