With the blessing of more powerful AI and related technologies, what kind of changes will occur in the game industry?
On Wednesday, April 13, Yang Xiaofeng, an analyst at Founder Securities, published an industry research report titled "The AI Revolution of Gaming: StableDiffusion to Nerf".
In the report, Founder Securities said that the gaming industry can achieve "3D out of nothing" based on existing AI models - GPT, Stable Diffusion and Nerf.
The function of GPT text generation artifact is well known to the public, and here Founder Securities highlights the Stable Diffusion and NeRF models.
AI-generated 2D drawing: Stable Diffusion
According to Founder Securities, Stable Diffusion is a potential text-to-image diffusion model that can realize the "Wen Sheng diagram" function. However, the current input content is similar to the program code and has a certain threshold.
Stable Diffusion provides open-source models that users can further train in a variety of styles.
At the same time, using the controlnet plugin allows Stable Diffusion to generate multi-view images.
AI drives "2D to 3D": NeRF models
NeRF, or Neural Radiance Fields, is a technology that uses multi-view images to reconstruct three-dimensional scenes, that is, using 2D pictures to build 3D scenes.
At present, the models built based on NeRF technology mainly include Google's Mip-NeRF, Block-NeRF, LOLNeRF, and instant NGP developed by NVIDIA.
According to Founder Securities, instant NGP can greatly improve the efficiency of NeRF, in some cases by more than 1,000 times.
Instant NeRF enables second-level 3D scene generation in a single-GPU (RTX3090) environment. In HD resolution, synthetic and even real-world scenes can be trained in seconds, rendered at 60 frames per second.
In addition, 3D modeling software service provider Luma AI has recently developed NeRF-related apps, which have been launched in the APP Store, which greatly reduces the threshold for the use of NeRF:
With only a mobile phone, you can render three-dimensional scenes anytime, anywhere, and easily use NeRF.
On March 24, 2023, Luam received $20 million in Series A financing, with NVIDIA as one of its investors.
GPT→Stable Diffusion→Nerf: Making 3D out of nothing
According to reports, there are currently two paths from text to 3D models.
Using the "Bunsen diagram" model + NeRF to generate a 3D model, GPT can play an important role in this. For example, using the stable diffusion+controlnet plugin, a multi-perspective 2D map is generated, which is then modeled in 3D by NeRF. However, since there is still a certain threshold for the prompts of the current Wen Sheng graph model, GPT may be trained to master it, and then achieve satisfactory multi-view 2D maps generated by natural language.
Use text to generate 3D models. This type of model is often a combination of text-to-image diffusion models and NeRF. The principle is that a 2D map is generated from a text-to-image diffusion model, and then a 3D model is generated by NeRF.
AI drives cost reduction and efficiency increase
At present, there are four common game art production methods in China, namely 3-shaded 2 production, 3D modern generation production (traditional hand-drawn), 3D traditional next-generation production and 3D next-generation production.
But overall, the current cycle of making a 3D game character is about 30-45 days, which requires many steps and a large number of participants.
Using NeRF allows for rapid modeling, resulting in increased efficiency and cost savings.
In terms of time, the speed of AI generating pictures is greatly improved compared to hand-drawn drawing; NeRF modeling can also complete multiple steps above at once, which greatly improves the time.
Manpower-wise, Ventu and NeRF can do the work of multiple roles while also being faster.
The main idea of this article is from Yang Xiaofeng of Founder Securities, the original title: "The AI Revolution of Games: StableDiffusion to Nerf"