BEIJING, Oct. 24 (Xinhua) -- In September this year, Hua Zhibing, a virtual person who "studied" at Tsinghua University, released a video of playing and singing songs, triggering discussion among netizens.
In this 38-second video, a girl plays and sings with a guitar. The subtitles above this incredibly realistic picture show that the girl is a "virtual person".
Screenshot of short video
At that time, many netizens were surprised that virtual people could already be so real. However, since then, some netizens have found that there is still a "live-action version" of this singing video. Some netizens further questioned that the video of the virtual person Hua Zhibing playing and singing only changed the face of a video taken by a certain up main real person of station B.
Is the technology applied to it just "AI face-changing"? What other cutting-edge technologies are in the video? How is this different from other "virtual idol" short videos? The reporter conducted an interview and investigation.
The three words "Hua Zhibing", which appeared in the public eye as the name of a virtual person, began in June this year.
On June 1, Hua Zhibing, an artificial intelligence student jointly "trained" by Beijing Zhiyuan Artificial Intelligence Research Institute, Zhipu AI and Xiaoice Company, entered Tsinghua to "learn". At that time, the relevant information and videos released caused heated discussion among netizens on social media.
Hua zhibing video released in June 2021. Courtesy of The Wisdom Source Conference
More than three months later, the Xiaoice team released a video of singing with the image of Hua Zhibing.
According to a statement released by Xiaoice, in the video, the facial features of the characters, including faces, expressions, mouth shapes, etc., are all generated and fused by artificial intelligence Xiaoice framework X Avatar; the body and movements of the characters in the video, including handheld guitar playing and singing, are derived from the original video template recorded by the Xiaoice team members; the songs in the video are generated by the artificial intelligence Xiaoice framework X Studio.
However, some people on the Internet interpret it as just the application of "AI face-changing" technology. In the view of the Xiaoice team, the technology is fundamentally different from "AI face-changing".
On October 19, Xiaoice company released a note stating that the video belongs to the hyper-realistic video production line product in the Xiaoice conference. The new productization technology enables the complete virtual generation of non-existent facial and sound technologies.
But Hua Zhibing is not just "changing a fake face".
The note also points out that even the face replacement in the video is not just an AI face replacement. Previous technologies can only achieve facial replacement between real humans, and the accuracy is insufficient to achieve content-level video production. In addition, the reporter learned that the songs in the video are also generated by artificial intelligence "creation".
Hua Zhibing frontal photo. Courtesy of the Xiaoice team
So, compared with the "virtual idol" works with "two-dimensional wind" that we usually brush on the short video platform, what is the difference between the technology applied by Hua Zhibing's short video?
"Now the vast majority of 'virtual idols' that everyone sees on the short video platform have nothing to do with artificial intelligence half a dime." They apply motion capture technology. In the view of Li Di, CEO of Xiaoice, Huazhibing and the Xiaoice framework behind it are fundamentally different from those "virtual idols".
The reporter asked a number of "virtual idol" related industry figures for verification, and also got a similar answer.
In short, most short videos have two-dimensional style "virtual idols", and more are video works that transform people's movements and facial expressions into two-dimensional styles through motion capture.
Although it looks virtual, behind it, there is a real person. The actions and reactions of these "virtual idols" are consistent with the people behind them.
But Hua Zhibing is not.
This brings us to another question: if motion capture can make people in videos look "virtual", why should we develop artificial intelligence?
The answer is about cost.
Just in September, the Ninth Generation Xiaoice released. Among them, the "Xiaoice short video content packaging pipeline" launched has greatly reduced the production cost of short video.
According to reports, the two-dimensional short video in the "Xiaoice short video content packaging pipeline" has realized the artificial intelligence self-pickup from text generation to short video generation full link and full pipeline, and does not require any manual participation in the middle, and the production cost of short video has been as low as 3 cents per minute. The three-dimensional short video has basically realized the entire link from biological feature generation to short video generation.
Humans only need to provide a few keywords in this process.
A few days ago, Xiaoice company made it clear in a public statement, "We believe that virtual people will become one of the main providers of video content in the future, and security, controllability, no privacy risk and infringement risk are its prerequisites." Therefore, Xiaoice has been exploring this trend direction and doing extended research on this field from different aspects. "The Huazhibing project is cooperating in the pre-training model, hoping to use the intelligent model as the core to test what technical and application surprises the pre-trained model can bring."
In addition, when Hua Zhibing "enrolled" at Tsinghua university in June this year, public reports showed that the team would continue to train Hua Zhibing's creative ability in the fields of music, painting and poetry, as well as the ability to interact based on emotion.
The reporter noted that the ninth generation of Xiaoice released last month has made new progress in some of these areas.
Taking painting as an example, the new edition Xiaoice introduced a model of Chinese painting.
Chinese paintings "created" by artificial intelligence. Video screenshots
Although Xiaoice has previously been able to "create" Western-style paintings based on keywords, Chinese paintings are obviously different. Li Di, CEO of Xiaoice, told reporters, "When we started training the model, there were many stamps on the 'created' works, because many sample data of Chinese paintings had Qianlong stamps in them. The algorithm did not know that this poke was not the focus of Chinese painting. ”
By training artificial intelligence to process the entities in the picture and observe the composition, Xiaoice can already master the "creation" of a considerable part of the subject Chinese painting.
In terms of interaction capabilities, artificial intelligence has also made progress.
While most people also think of AI as a tool for receiving instructions and giving feedback, some AI can already ask humans questions.
Artificial intelligence MERROR image. Video screenshots
Last month, an account called AI_MERROR posted a video of ai-powered people "talking" to humans. In this video of more than 5 minutes, the artificial intelligence MERROR can ask humans questions such as "Please introduce the world from your point of view" and "When was the last time you felt embarrassed"; when asked by humans "What was your state when you died", MERROR will reply "Sorry, let's change the topic".
From this perspective, artificial intelligence has become more and more human-like.
At the same time, under the framework of Xiaoice, more and more kinds of artificial intelligence have begun to appear, some of which have their own unique styles, such as "Shandong Big Brother".
Artificial intelligence "Shandong Big Brother" image and its works. Video screenshots
This is an artificial intelligence that can speak "Shandong Mandarin", can draw large peonies, and looks like a strong man. For the technical team, the most challenging thing is how to get the artificial intelligence to speak "Shandong Mandarin".
"We hope that 'Shandong Big Brother' has his unique textual language habits, not to take a manuscript and read it." Because he is from Shandong, he will have some specific language and rhetorical methods, and Shandong people are used to saying inverted sentences. Li Di told reporters that "Shandong Big Brother" can already master these dialect skills, and the team is currently studying how to make artificial intelligence speak Guangxi dialect.
In the virtual social platform "Xiaoice Island" where humans and artificial intelligence coexisted, which was previously launched by the Xiaoice team, there are more artificial intelligence with different styles, different accents, and different skills.
Artificial intelligence is beginning to tend to be "a thousand faces".
In a trance, you can't even feel who is human and who is artificial intelligence in a virtual environment.
As a member of the artificial intelligence Xiaoice team, Li Di's biggest concern at present is not what technical bottlenecks are difficult to break, but how should we deal with the relationship between humans and artificial intelligence when artificial intelligence is getting closer and closer to humans.
At the end of the interview, he told reporters that it is important and necessary to make rules in the ethics of artificial intelligence, but at present, in the field of artificial intelligence, the most missing rules are also the rules of artificial intelligence ethics. And this can no longer be done by technology alone. (End)
Source: China News Network