Dialogue Opening Ceremony Real-time Special Effects Team Leader: The idea comes from the reflection on the ice surface, which is the first in the world

Beijing News Express (reporter Ma Jinqian Liu Xin) Just concluded the opening ceremony of the Beijing Winter Olympics, the LED ground screen display interactive technology that covered the whole field shocked the audience.

As the performance progressed, the actors on the field "bloomed" under their feet and produced crystal white snowflakes, which had the effect of "shadow following". Tong Lei, dean and professor of the School of Sound of Beijing Film Academy, introduced that this is achieved through the integration of real-time capture, interactive presentation and other technologies, and nearly 700 actors have achieved time-free presentation on more than 10,000 square meters of LEDs, which has also achieved the world's first.

The Beijing News spoke with Professor Tong Lei, dean of the Beijing Film Academy's School of Sound. Beijing News reporter Liu Xin photographed

The real-time interaction idea comes from the reflection on the ice

Beijing News: How to achieve the effect of "shadow following the shape" on the ground screen?

Tong Lei: The secret of the ground LED effect "Like a Shadow" contains two parts of technology.

First, real-time motion analysis technology based on artificial intelligence identifies actors through video cameras and passes the information to the next system. The second system is the responsibility of the film school, which obtains information through the real-time interaction engine, generates the desired effect according to the director's artistic creative needs, and presents it at the feet of the actors without delay. In the eyes of the audience, the effect of "shadowing" under the feet of the special effects has been realized.

In this kind of theatrical performance occasion, nearly 700 actors have achieved no delay in the PRESENTATION of more than 10,000 square meters of LEDs, which is probably the first in the world.

Beijing News: What is the source of inspiration for this idea?

Tong Lei: About half a year ago, director Zhang Yimou had a preliminary idea: Can the effect of ice reflection be realized on the ground screen? After receiving the demand, we did a lot of demonstration research, trying to initially achieve this effect from the physics of the ice reflection, but at that time, the algorithm was relatively rudimentary, there was a certain delay, and the number of actors that could be carried was also limited.

In the director's creative process, we found that there may be more effect needs, so we made more than a dozen effects such as glittering, aurora, pushing away snowflakes, pushing away clouds, etc., and the plan was handed over to the director, who adopted it according to the overall design. There are a lot of run-ins in this process, not only in terms of scientific and technological feasibility, but also in terms of the need to use technology as a means to show art.

Such an effect cannot be achieved by pre-production or precise rehearsal, and it is completely dependent on technical means. The actor can play freely, the play process will affect the display effect, the display effect will also boost the actor's performance, which is what we call interaction. This interaction precisely reflects the natural romantic mood and is also very much in line with the director's original creative intention.

Countless detail parameters are adjusted to meet the creative needs of the director as much as possible

Beijing News: Where are the technical difficulties in real-time capture and interactive display?

Tong Lei: The large area of the ground screen, the large number of actors carried, and the requirement of real-time capture interaction without delay require very strong computing power and engine support behind these technologies.

These technologies may have been used very maturely in their respective fields, such as capture technology is commonly used in athlete motion capture analysis, real-time rendering is commonly used in large game scenes, and our idea this time is to perfectly combine the two technologies to form a "one-stop" link from capture to recognition to presentation rendering, in order to ensure that the effect is perfectly presented.

We also hope that through this technology, we can realize a creative presentation of art that is different from the traditional and reflects the characteristics of interaction. The current effect has undergone several months of refinement and correction, for example, in order to meet the needs of broadcasting, the color of the snowflakes on the ground screen has undergone several versions of the adjustment, and this adjustment has occurred countless times throughout the rehearsal process.

Beijing News: We all know that director Zhang Yimou has very high requirements for his own creative realization, what kind of run-in have you experienced in this process?

Tong Lei: I am a filmmaker myself, and I am very subjective in the field of artistic creation, and I have never been perfect, so I never want my ideas to be agreed upon by others without any objection. We can only use our imagination as much as possible and give as many good ideas as possible to the director. In this process, feedback and modification opinions are inevitable, and it is precisely the continuous exchange of corrections to achieve the perfect effect.

After each rehearsal, the director makes suggestions for each show. Our happiest moment was that we did not ask any questions or suggestions about this part of the seminar, because it showed that our effect was up to his expectations. What we are looking for is to realize the director's creativity as much as possible and to impress the audience as much as possible.

Participated in two Olympic Games and was responsible for the fireworks display in 2008

Beijing News: You were also involved in the opening ceremony of the 2008 Summer Olympic Games in Beijing, what was you responsible for at that time? How are the two Olympic Games feeling different?

Tong Lei: In 2008, I was responsible for the fireworks control of the opening ceremony of the Olympic Games, in cooperation with Teacher Cai Guoqiang.

In 2008, the fireworks display inside and outside the Bird's Nest was jointly completed by five or six units, and the control methods and control instructions of each unit were different, which required a general control center to coordinate each family, accurately cooperate with the performances, music and time points in the venue, and the error was controlled within one or two frames. In response to this demand, I made a pyrotechnic control system that year to coordinate the accurate firing of various firework systems.

If you want to say differently, in 2008 I was still a young lad, and 14 years have passed, our technology and expression have improved, and we have higher requirements for ourselves. This Winter Olympics is very different from the 2008 Summer Olympics in terms of form and style, and the requirements for our on-site technology are definitely getting higher and higher, and in terms of subjective will, we must also strive to do better than 2008.

Beijing News: We have noticed that many teachers and students of the film school participated in the Winter Olympic Games, what positions are they in?

Tong Lei: The Beijing Film Academy is a well-known professional film school in the country and even in the world, which not only cultivates a large number of outstanding film talents for the national film industry, but also has a large number of teachers and students participating in major activities such as the 70th anniversary of the founding of New China and the centenary of the founding of the Party.

At the opening and closing ceremonies of the Winter Olympics, we still sent a large number of teachers and students to participate. First of all, there were nearly 200 students participating in the instructors of the delegations and the actors of the opening and closing ceremonies; the real-time special effects team I was in was completed by 8 teachers and students of our school; there was also a team responsible for the filming of the opening and closing ceremonies documentaries, with eight or nine people, also from the film school. There are also many teachers and students in other venues, such as the media center, who do technical support. The college as a whole has invested a lot of manpower and material resources, cooperated with and improved various safeguard measures, and made every effort to escort the successful hosting of the Winter Olympic Games.

Beijing News reporter Ma Jinqian Liu Xin

Edited by Chen Jing Proofreader Fu Chunyan