Use AI technology to cast time "magic" on old animations

On one side are faded, stuttering, and out-of-frame animations, and on the other side are emerging computer algorithms. The old and the new, are thus combined. Restoration technology can already make old animations 4K effects. Let the traditional animation reawaken the audience's childhood memories with a high-definition and smooth appearance.

"Whether it's fixing a cartoon or fixing an old image, it's not just about improving its clarity. At the end of the day, what we want to fix is the memory behind this content and the emotional resonance that generations have had over time because of these memories. Ren Lifeng, president of Watermelon Video, said in an interview with a reporter from Science and Technology Daily.

Not long ago, the 4K restoration version of 6 classic animation works such as "The Legend of Nezha", "Hulu Brothers" and "Black Cat Sheriff" was repaired by the watermelon video platform and launched on the watermelon video. In addition, the 4K commemorative version of the classic animation blockbuster "Tianshu Qitan" has been released in theaters nationwide on November 5, and the traditional animation that carries the common memory of generations of Chinese is quietly coming with a new look.

AI technology provides more possibilities for animation repair

When it comes to animation restoration, people's stereotype is that the studio restorer cleans, scans, and grades the film, and the whole restoration process is completed, often taking months, even half a year, or several years.

In the past, the medium for storing films including cartoons was mainly film, and the early films were all nitric acid sheets, which is a more flammable substance than paper, although after 1960, the nitric acid sheet base was basically replaced by acetic acid sheet base and a polyester sheet base, but no matter which kind of film base (a plastic film, is the supporter of photosensitive film), it is difficult to keep it at room temperature for a long time.

In order to more safely preserve old animation (old films), as well as to rescue and restore damaged films, animation restoration technology is also constantly innovating. With the rise of deep learning and AI algorithms, the "old piece" repair has gradually shifted from artificial to artificial + intelligent repair mode.

"The traditional manual repair is mostly to remove dirty spots, scratches, jitter, etc. on the film, it just makes the picture cleaner, equivalent to 'washing the face' of the picture, which is a kind of icing on the cake." However, AI technology can do 'out of nothing', filling in the details missing in the picture, its efficiency is higher, the means are richer, and it also provides more possibilities for animation repair. Zhao Shijie, a researcher at the Volcano Engine Multimedia Laboratory, told reporters.

Taking the well-known "Hulu Brothers" as an example, Zhao Shijie showed reporters that the film is a combination of ink and paper cutting, and the difficulty lies in the diversification of restoration types - both to ensure that the characters are sharp, while also taking into account the artistic effect of the ink background. Therefore, AI technology is required to accurately identify the foreground and rear scenes, use super resolution and intelligent interpolation algorithms to complete the conversion of video from low definition to high definition and generate intermediate frame content.

In addition, the repair of old cartoons is mainly the underlying multimedia tasks such as images and videos, such as noise reduction, super score, color enhancement, etc., many of which combine with AI's deep learning algorithms, which can greatly improve the efficiency and quality of restoration. "In the past, some repair processes that took a long time to manually deal with, such as eliminating screen scratches, noise and other imperfections, professional masters can only repair one or two hundred frames a day, and AI algorithms combined with powerful computing power can accelerate tens of thousands of times." Zhao Shijie said.

Find a balance between technology and art

"Despite the blessing of AI technology, repairing classic old animations is still a heavy and difficult task, and the biggest challenge is to retain the artistic style and beauty of the original animation." Zhao Shijie said.

There are six types of damage to image film, including scratches, noise, and mildew. The restoration work is to eliminate and correct these problems on the film, while restoring and improving the original texture of the film as much as possible, so as to "repair the old as old".

In the restoration process of "Hulu Brothers", it is necessary to improve the clarity of the foreground and take into account the artistic effect of the ink background, so that the landscape maintains a hazy artistic sense. Zhao Shijie admitted that when repairing "Hulu Brothers", they discussed the plan with the producer many times, and after repeated testing and tuning, they achieved the desired effect.

In order to prevent "accidental injury" of artistic effects, the algorithm that eliminates defects usually does not set a high intensity, so some defects are missed and need to be marked with manual assistance. For old films with serious damage, it takes more labor to completely eliminate flaws.

Zhao Shijie told science and technology daily reporter that at present, AI technology can not completely replace human labor. On the one hand, the restoration of old animation is not blindly improving the picture quality, the restoration process needs professional art guidance and artificial assistance to achieve the ideal artistic effect; on the other hand, the restoration in the color enhancement and other links, but also the use of some multimedia processing technology without AI intervention, not all technical means to use AI algorithms.

The restoration of "Hulu Brothers" is only a typical case, the volcano engine repair data shows that the algorithm can directly eliminate more than 95% of the flaws, and the rest should be supplemented by manual labeling, and then adjust the algorithm to do secondary optimization.

Industry experts have said that classic works condense the wisdom and spirit of old artists, and finding a balance between technical empowerment and maintaining artistic effects is a key problem that must be answered well in the restoration of old animation.

The restoration of old films is entering the homes of ordinary people

In 2014, after the Shanghai International Film Festival created the first 4K restoration unit, 4K restoration entered the public eye. Subsequently, professional institutions such as the China Film Archive and the China Film Digital Base successively launched 4K restoration plans.

Different from the lengthy content restoration such as movies and TV series, traditional animation films such as "Hulu Brothers", "Monkey Fishing Moon", and "Little Tadpole Finds Mother" are only more than ten minutes long each episode, and the narrative is complete, and the workload of restoration is relatively small.

According to public information, since the establishment of the Shanghai Fine Arts Film Studio, more than 80% of the animation films within 30 minutes of more than 500 animation films have been produced, which means that under the condition of existing restoration technology, a large number of traditional animation films can be "reborn".

"According to the plan, in the next year, the 4K restored version of 100 traditional cartoons such as "Haha Mirror Flower Edge" and "Shuk and Beta" will be launched on Watermelon Video one after another." Ren Lifeng revealed.

Why do Internet video platforms have to repair old animations? Zhao Shijie said that the picture quality problem also affects the video platform with massive submissions, "In order to improve the user experience, the video platform needs to optimize the low-quality submissions, of which artificial intelligence and other technologies can be used in the repair of classic old movies and old animations." Zhao Shijie said.

It is understood that based on the accumulation of technology and capabilities, volcano engine and other technology companies have launched "intelligent processing" toolkits, through AI algorithms to achieve the effect of enhancing video quality, and has been applied to video creation platforms.

"The technology is mature, and all video platforms will also open the restoration portal, provide free AI restoration support for ordinary users, and carry out free restoration of historical images with historical value, so as to change the old appearance of video works treasured by the public." Zhao Shijie said. (Reporter He Liang)

Source: Science and Technology Daily