To help improve the quality and efficiency of content creation, Tencent officially released the AI intelligent creation assistant "Tencent Smart Shadow"

"Hello everyone, I am a digital tycoon, and I am honored to discuss with you a topic that has attracted much attention: can AIGC (AIGC (AI Generated Content, which refers to the use of artificial intelligence technology to generate content) replace humans in creation? History proves that human creativity and creative style are constantly evolving with the times and technological innovations. In content creation, especially in the creation of high-quality personalized content, the collaboration between AIGC and humans is a very necessary part. ”

On March 30, at the 2023 New List Conference, Yao Tianheng, deputy general manager of Tencent's content platform department, opened with such a speech through his digital human "tycoon". What is interesting is that this digital person is not only highly realistic in image, but also very real in terms of voice, intonation, lip movements, etc., and even Yao Tianheng's usual expressions and movements are simulated without sloppiness.

Yao Tianheng, deputy general manager of Tencent's content platform department, and his digital human "tycoon"

It is understood that the digital human "tycoon" is based on Yao Tianheng's real person, using Tencent Zhiying's latest artificial intelligence technology, and learning a small amount of real audio and video data to generate a doppelganger model. Taking this opportunity, Tencent also officially released a new AI intelligent creation assistant "Tencent Smart Shadow", which provides content creators with a series of intelligent creation tools to help creators improve quality and efficiency in content creation.

"In content creation, the industrial-scale output of PGC and the creative transformation of UGC have always been the core problems that plague creators, but the development of AIGC has greatly improved the productivity of creators, thus providing a feasible solution to the above problems. Combining AI capabilities and human creativity, AIGC can realize full-link closed-loop human-computer collaboration, including content creation, opinion establishment, material generation, content polishing and production. Tencent Smart Shadow is to make content creation simpler and more efficient by providing the ability of 'people', 'sound' and 'shadow'. Yao Tianheng introduced.

Liberating "human" power, Tencent Smart Shadow enables everyone to realize digital human freedom

In terms of "people", "Zhiying digital human" is the core function of Tencent Zhiying. Users only need to enter text or audio content and generate digital human broadcast video in minutes. Tencent Smart Shadow currently has opened dozens of digital humans with diverse styles, and creators can choose digital human images and costumes according to their needs, and add different actions and backgrounds, making it easier to make video production of live people.

Tencent Zhiying Digital Human Broadcast

In addition, Zhiying Digital Human can also achieve "image cloning". Similar to the digital human "tycoon" at the beginning of the article, users can get their own digital human avatar by uploading a small number of pictures and video materials, and the price is close to the people and convenient to operate, so that everyone can realize the freedom of digital people.

Tencent Smart Shadow also has access to digital human live broadcast, which can directly replace real people, without time and space restrictions, without the need to build any shooting venue, using virtual backgrounds to carry out broadcasting, interview, connection, interaction and other operations, to achieve 7*24 hours of uninterrupted broadcasting.

Tencent Smart Shadow Digital Human Live Broadcast

At present, Zhiying digital people have applications in multiple scenarios. Not long ago, Tencent Zhiying provided technical support for the People's Daily New Media Two Sessions reporting digital anchor "Ren Xiaorong", bringing personalized news broadcast content with "thousands of people and thousands of faces"; During the epidemic last year, Tencent Smart Shadow also cooperated with the Voice of the Yangtze River Delta of Shanghai People's Radio to create the country's first provincial-level radio digital human anchor "Chang Xiaojiao" to help explore media content production and video transformation. In addition, adhering to Tencent's concept of technology for good, Zhiying Sign Language Digital Human also serves the Winter Olympics and the King of Glory KPL competition, reporting and explaining the event for the hearing-impaired.

Blooming "sound" machine, Tencent Smart Shadow helps creators make dubbing efficient and concise

In the innovation of AIGC in the direction of sound creation, Yao Tianheng introduced the three functions of intelligent text dubbing, timbre customization, and intelligent voice change.

In order to meet the dubbing needs of different creators, Tencent Smart Shadow text dubbing function provides hundreds of timbre to choose from, and natural speech can be generated by entering text, which is simple and convenient to operate, and is suitable for various scenes such as news broadcasting, short video creation, and audio novels. The data shows that Tencent Smart Shadow can complete the dubbing and release of a 1,000-word manuscript within 2 minutes, and can manually adjust the effects of voice multiplication, local speed change, multi-tone words and pauses, and also supports multi-emotion and dialect broadcasting, making the audio sound more vivid and natural.

Tencent Smart Video text dubbing

In addition, based on Tencent Zhiying's voice recognition and replication technology, creators can quickly edit audio and video content. First of all, Tencent Smart Shadow can automatically remove redundant words through speech recognition and text smoothness. In terms of audio rewriting, through techniques such as sound cloning and audio training, some sentences in the original video can be modified. Yao Tianheng introduced, "This text-to-speech method can reduce the cumbersome recording process, and help creators quickly and accurately complete a large number of dubbing work in scenarios such as video creation, hot release, book refurbishment, and post-slip of the tongue processing, so that high-quality content can reach the audience faster." ”

Tencent Smart Shadow also provides an interesting voice changing function, through which creators can convert audio into designated vocals while retaining the original rhythm, helping creators liberate productivity and make videos more expressive. In the live demonstration, Yao Tianheng took the narration dubbing of Ye Wenjie in "The Three-Body Problem" as an example, Tencent Zhiying reproduced Ye Wenjie's voice in advance, and through Zhiying's voice change technology, the voice recorded by any person was converted into Ye Wenjie's voice, which sounded very infectious. "When faced with a large number of content production tasks, the creative team can work together to help KOLs with audio interpretation; Or in the process of video processing, a certain video needs to be recorded and modified, through this function, there is no need to let the actor rework, any staff can assist in completing the dubbing update. ”

Let "shadow" follow, Tencent Smart Shadow makes the automatic generation of intelligent video a reality

In terms of video image editing AI, Yao Tianheng pointed out that AIGC's text automatically generated video technology is a revolutionary technological innovation that brings unprecedented opportunities and challenges to the video industry. Compared with traditional video production methods, AIGC technology greatly improves the production efficiency and quality of video content, and also provides creators with more creative space and freedom.

For what kind of intelligence is valuable to creators? Yao Tianheng gave three key words: "understand you better", "more exquisite" and "more vivid".

"Know you better" means that a good tool must accurately understand the creator's intent, whether it is text, speech or hand-drawn sketches, which can be quickly transformed into actual creative results by intelligent tools. "More beautiful" refers to the improvement of creative quality, such as good smart tools to help creators better tell their story through various forms of assistance, such as a good sentence, a beautiful image, or a wonderful shot. "More vivid" is the requirement for content expression, compared with pictures, text, video is a more vivid form of expression, good tools can help creators produce more interesting and vivid content.

With the blessing of AIGC technology, creators can directly convert their text into video content through Tencent's Zhiying article conversion capability, without cumbersome material collection and processing. In addition, Tencent Smart Shadow's segmented material presentation allows creators to quickly process storyboards, add points, filters, special effects, etc., thereby greatly shortening the cycle and cost of video production. At the same time, Tencent Smart Shadow also provides a large number of copyright materials to provide creators with more choices and inspiration. This allows creators to focus more on the planning, creativity and marketing of creation, and improve the quality and dissemination of content.

Tencent Smart Shadow article to video

"In the future, Tencent Zhiying will become more intelligent, and creators can even create an anime or make a wonderful video on Zhiying with only one idea to make their stories more vivid." For the next development plan, Yao Tianheng said that Tencent Smart Shadow will further strengthen the research and exploration of AIGC technology, continuously explore its potential application value and possibilities, lower the threshold for the use of digital humans and voice customization products in terms of price and ease of use, and empower creators with AI, so that intelligent creation can follow the shadow, truly realize the diversification of content and let creativity bloom.