Virtual production demonstration site
The screen plays the male and female protagonists talking in the ascending "space elevator", surrounded by spaceships roaring by and space base scenes that gradually change as the elevator rises.
Finally, the elevator rises to the top floor of the space base, and the male and female protagonists are placed in a fantasy space base full of scientific and technological sense - all kinds of high-tech equipment are lined up, the spacecraft hovers above the heads of the male and female protagonists, and the faint light and endless darkness in the distance reveal the mystery of space...
Do you think this is a clip of some sci-fi blockbuster that has gone through post-production?
In fact, this is a clip of the first domestic movie concept film "Unknown Planet" that uses full LED virtual shooting. At the "2021 China Science and Technology Film and Television Summit Forum (Chongqing)" held on October 20, an LED virtual studio covering an area of more than 200 square meters stood behind the venue. With the "start of filming" by director Zhou Xiaofan, the original team of "Unknown Planet" reproduced the filming of the male and female protagonists meeting in the "space elevator" in the film. Through virtual production technology, the scene is completely rendered in real time using the virtual engine, and the rich and fantastic pictures taken on the scene are presented on the big screen in real time, which is the "what you see is what you get" of virtual production.
2021 China Science and Technology Film and Television Summit Forum (Chongqing) scene
On October 20, Chinese film and television geeks gathered in Chongqing "2021 China Science and Technology Film and Television Summit Forum" to welcome new opportunities in China's virtual production era, and the small editor of the science and technology association channel was also invited to participate.
In recent years, many films have used virtual production techniques, such as Star Wars, The Lord of the Rings, Avatar, The Lion King, Whitesnake 2: Green Snake Robbery, and so on. What exactly is virtual production? What technical blessings does it need? How to bring more creativity to film and television production?
fresh?
Virtual production refers to a range of computer-aided production and visual filmmaking methods. Simply put, it is filmmaking in a real-time interactive way, combining virtual reality and augmented reality with computer-generated imagery and game engine technology to make the picture look like it was shot on location.
In 2019, with the broadcast of "The Mandalorian", the word virtual production was a big hit, was widely discussed, and gradually began to break the circle. In fact, virtual production is not a particularly new thing, "more than a decade ago there was this concept", wang Chunshui, director of the virtual production laboratory of the Beijing Film Academy's Future Imaging Advanced Innovation Center and director of the Beijing Image and Graphics Society, who entered the field of virtual production in 2009.
Stills from The Mandalorian
During the filming process, "The Mandalorian" introduced and used LED surround screens (background walls) on the basis of the original virtual production. The intuitive, figurative, and touchable virtual production entity of the LED screen also makes the concept of virtual production clearer and more concrete in the eyes of the public.
According to Zhu Ziwei, CTO of Dawa Future Imaging Technology Co., Ltd., more than a decade ago, some people used projection, television or other methods to achieve such a virtual screen, but at that time, many people did not understand what problems this could solve. Today, virtual production spans broadcast, film and television, stage, advertising, animation, games, simulation and other industries, and is born and applied in various fields.
From a technical point of view, virtual production can be divided into visualization, performance capture, real-time hybrid green screen and real-time LED wall.
Long ago, TV news weather forecasts were a manifestation of real-time hybrid green screen virtual production.
In terms of film, the graphics and effects of Avatar caused a sensation around the world when it was released. It used more than 4,000 servers, more than 35,000 CPUs, and 2.88 million hours of rendering time. It captures the tech dividend and presents a realistic unreal world with a virtual camera.
Following Avatar, the "real interaction" between a young human boy raised by wolves and various animals in the forest in 2016's "Fantasy Forest" is post-production through mixed virtual production technology. Although it still retains the traditional production method of special effects films - actors perform in front of the green screen, there is almost no background and "partner" hints, but the virtual production technology it applies is more mature, combining live actors with virtual animal characters, virtual environments, and virtual props, capturing performances and actions through rehearsals, and combining with real-time synthesis in the later stage to form a virtual set. This also makes the graphics of "The Jungle Book" look more like a live-action movie.
Stills from The Jungle Book
The LED screen background wall used in "Mandalorian" replaces the green screen, making real-time shooting, real-time rendering, and real-time output possible. The use of LED screen background wall for virtual production, can achieve the effect of what you see is what you get, that is, in the process of filming, you have also completed the production of post-visual effects, and the captured footage can become part of the finished film. Just as Dawa Technology showed at the conference site of the "Unknown Planet" "space elevator" bridge section of the real-time filming and production.
Led virtual studio built at the conference site
Can achieve real-time film production, because virtual production is not simply a picture or a video to the LED screen to play, during the shooting process, the real camera will be perfectly matched with the virtual camera in the virtual world, so that the real camera through the LED screen to shoot the grand virtual world.
Schematic of virtual production
progressive?
Although virtual production is not a new thing, the emergence and application of real-time LED screens have brought more convenience and possibilities to the workflow and production effects of virtual production.
In the past, green screen and compositing have always been a solution for virtual production of fantasy and science fiction scenes, but there have always been some technically difficult problems in the later stage. For example, when the character is wearing a highly reflective costume or wearing a reflective mask – just as each character in Unknown Planet wears a highly reflective helmet in space – how ambient light and shadow will be presented on reflective objects to create a more realistic, realistic scene is a huge problem for later light and shadow production. In addition, the actors perform in front of the clueless green screen, which not only tests the actor's performance ability and experience, but also tests the composition and lens design of the main creators.
Under the surround real-time LED screen, the light and shadow elements will be reflected in real time to highly reflective clothing, helmets and other objects, directly presenting real-time light and shadow effects, avoiding the high cost and difficult light and shadow production in the later stage. At the same time, the actors can also have a clearer understanding of the environment in which the story takes place in the context of the surrounding, and with hints and references, it will be conducive to their control and adjustment of their performances. The main creators can also control the lens and screen according to the real-time presentation of the picture. In the on-site shooting of "Unknown Planet", director Zhou Xiaofan can control the lifting and lowering of the "space elevator" in real time through the tablet in his hand.
The director can control the lifting and lowering of the "space elevator" in real time
50% of the footage in the first season of "The Mandalorian" was shot in the new way of real-time LED screens, which used an LED screen wall consisting of 1326 individual LEDs, about 6.1 meters high, covering the ceiling, presenting a 270-degree semicircle, and the available performance space is about 23 meters in diameter. During the filming process, the director can make specific creative decisions based on the visual effects of "what you see is what you get", and it is said that it can be modified in real time to generate new digital assets (virtual scenes presented by LED screens). In fact, the first season was not filmed as fast as it was in real time.
"At that time, it would take at least a day to change a set of digital assets, which could be reduced to 20 minutes when the virtual engine was upgraded to version 1.5, and it could be reduced to 20 seconds if the current Hive AI data engine was used." Sun Dan, co-founder of Naguang Intelligent Technology Company, said.
In addition, the virtual production of real-time LED screens will also bring more convenience. The shooting team did not have to look around for scenery and sets, a large film could be completed in one studio. In terms of time, compared with traditional production, virtual production can also greatly compress time. The core proposition of the first season of "The Mandalorian" is that the actual production must be completed within 6 months, which is why it uses real-time LED screen virtual production. With the blessing of this technology, "the entire project of the concept pilot film of "Unknown Planet" was only 18 days, and the shooting was only about 3 days. Zhu Ziwei introduced.
The studio where The Unknown Planet was filmed
debate?
Does the imaginative frontality shackle the director's inspiration and creativity?
Real-time LED screen virtual production puts digital assets such as environment and background into the film before the film shooting, which eliminates the need for post-processing and can achieve "what you see is what you get", but this is indeed a test for the director.
From the perspective of technology, the development of technology can make it very convenient for directors and creators to get the content they want quickly. "After Unreal Engine 5 came out, a shot was rendered overnight, and I gave feedback to the technical side, and they changed it right away, and the next day they could come up with a new film." The famous director Li Weiran bluntly said that the update of technology will make it convenient for creators. In addition, the current technology can also support the director to make modifications on the shooting set.
Virtual scenes can be modified in real time
In the view of the famous animation director Zhao Ji and the technical director of Dawa Technology Laboratory Wang Xin, the use of real-time LED screen virtual production technology requires the imagination to be front-loaded, that is, the inspiration and ideas are quickly realized and determined in the creative process. This approach may have certain limitations on the imagination. Zhao Ji believes that "front-facing" is an ideal way to work, but he is worried that the creation will lack inspiration and additional possibilities in the early stage of the actor's side and actual shooting. In the future, these two aspects need to be rebalanced and explored.
In addition, using the unreal process's plug-in and process management, the director can make unlimited modifications to the virtual scene that has been set on the spot, "but when shooting on the spot, the director suddenly came to the idea of adding a scene, which is not allowed in virtual production." Sun Dan said.
Is it really a cost saving?
Cost savings are also a major advantage of real-time LED screen virtual production that is often mentioned. All shooting can be easily done in the studio, eliminating the need for travel expenses, venue fees, set fees, etc. But some people also question that for action movies, from the house to the outside of the house and then hit the roof, the entire space structure needs a more complex environment, so how big is the shed based on the LED screen, and the cost of the shed and the production of virtual space may be more expensive than building a physical scene.
At present, the cost of LED sheds is still very high. The cost of the studio of Naguang Technology to taipei Shih Hsin University's tailor-made practical education base is about 2 million US dollars, and this studio does not meet the requirements of film big screen production. If you want to present a perfect picture effect, led screens are very critical. Science fiction film director Yu Gang introduced that in order to present the real virtual scene, they tested a lot of screens, and there is a world-class screen that can currently achieve the true and false background that cannot be seen by the naked eye, so as to achieve a very real presentation. Of course, the cost of different levels of screens is also different - the price difference between the screen that can be used and the top screen is dozens of times.
Smart studio of Shih Hsin University in Taipei
In Sun Dan's view, at present, in the field of virtual production, it is often mentioned that it can save a lot of costs, but in fact, it can only produce more things under the same cost. In addition, real-time LED screen virtual production has certain scene limitations, and some scenes may directly use traditional shooting methods than using virtual production technology to save money. "Virtual production has a rate of application, and sometimes it is better to use traditional methods to solve. For example, we shoot "Family with Children", make a real scene on it, many people around the machine toss, it is better to make a scene to shoot; shoot the Huangguo tree waterfall, you can directly shoot the real scene material, and then put in the LED screen, this is also an application direction, not everyone thinks that you must make a three-dimensional image to use the LED screen. ”
In the future, it is expected that the development of LED sheds will be large-scale, functional segmentation, and process division of labor will help reduce the cost of virtual production, make virtual production widely used in film and television production, and bring revolutionary changes to the development of the film and television industry.
Edit | Tannie
Typography | Radish Juan
Audit | Six winged butterflies Susu
The image comes from the Internet
Please do not reprint the article without authorization.
Disclaimer: The content of this article only represents the personal views of the author and does not represent the position of the platform, and everyone is welcome to discuss it in the comment area.