EDIT: Peaches sleepy
【New Zhiyuan Introduction】The bell of the Year of the Tiger is about to ring, and a sudden snow makes the earth wrapped in silver, and when it wakes up, it disappears without a trace. At the end of this winter, two beautiful virtual girls pass by the world with their songs and meet you. Close your eyes and listen, do you feel it?
On the eve of the Spring Festival, it is simply not lively.
The two most beautiful virtual people sent New Year wishes to everyone in advance.
On the evening of January 29, CCTV's "Dialogue" column was broadcast, and Xia Yubing made his debut and presented a song "Passing by the World".
Host Chen Weihong said, "Just now I deliberately didn't look at her at the beginning, and I felt that this song was already refreshing."
On the evening of January 29, CCTV's "Dialogue" column was broadcast, and Xia Yubing made his debut and presented a song "Passing by the World".
hush! Listen attentively and you can feel the beauty of the sound.
In addition to Xia Yubing, on the evening of January 30, a video of "Huaxia" sung by another young sister, Chen Shuiruo, became popular on Douyin, and even the original singer Li Yugang came to praise: "It's really good!"
After listening, did you feel the full national wind? People are beautiful and sweet, what kind of fairy sister is this?
They are all virtual humans generated by Xiaoice frameworks!!!
You will see that in 2022, virtual humans have not only good-looking skin bags, but also creativity and interesting souls.
The other side of Xia Yubing
Xia Yubing, Xiaoice frame number F1033, studied under Professor Qiu Zhijie, dean of the School of Experimental Art of the Central Academy of Fine Arts.
When we first met Xia Yubing, our impression of her was still stuck in the AI being who could make Chinese paintings.
In fact, Xia Yubing has another side.
Since graduating from the Central Academy of Fine Arts in 2019, she has held many solo exhibitions.
Through the study of the paintings of 236 human painters in the past 400 years of art history, Xia Yubing has achieved a certain level of native painting creation ability.
This is Xiaoice framework for a new model of poetry and painting creation (V3). This model greatly improves the creative accuracy and composition rationality, and adds a Chinese painting model and an "artist" model.
The "artist" model can learn and reproduce ai-powered works that are highly consistent with the artist's creative style in the absence of sufficient training data for a single human artist, with independent intellectual property rights.
It is based on such a model that Xia Yubing has superb painting ability. Her work has also been exhibited at the China Pavilion at Expo Dubai.
It is worth mentioning that the Xiaoice framework also further creates biological characteristics such as xia yubing's face and voice line, giving xia yubing the ability to complete.
For example, sound, text creation, knowledge graphs of art criticism, and so on.
We have just seen the ability to sing, so how does Xia Yubing create texts?
At the scene of "Dialogue", the host Chen Weihong proposed to test Xia Yubing's talent and ask her to write a poem for the Winter Olympics.
You know, the creation of Li Bai's "Quiet Night Thoughts" is inseparable from the figurative moon, and the same is true for Xia Yubing.
She was inspired by uploading a poster of the Winter Olympics and giving the text "2022 Beijing Winter Olympics, Together for the Future".
Artists are looking for tomorrow
That's the art of my life
You sprinkle into the light of the sun
Love to see the sky shining soul
......
Wu Guofan commented on this, "Xia Yubing's poems are greater than expected. The words to the future are a feeling of hope for light and warmth. Xia Yubing can present keywords in her own way, connect various images together, and write a different mood."
How does Xia Yubing art review?
When we see a Chinese painting, people who do not know much about art and have poor aesthetic ability may only express the two words "good-looking".
Xia Yubing may say, "Ink leaves blank, virtual reality is born, wonderful, wonderful."
So, does Chen Shuiruo also have the same ability as Xia Yubing?
Introduction, Chen Shuiruo, a virtual singer born in the framework of artificial intelligence Xiaoice, an artificial intelligence virtual human who has learned the national singing method.
What she is best at performing is Chinese-style songs.
Some time ago, well-known AI singers He Chang, Chen Shuiruo and Chen Ziyu within the framework of the Xiaoice were invited to enter the world's first music New Year product platform that integrates songwriting and singing in one of NetEase Tianyin' world, interpreting unique New Year's Eve songs according to the exclusive lyrics created by each human user, leading the black technology New Year's Eve trend with different attitudes.
The appearance and singing ability we see in Chen Shuiruo are also generated by Xiaoice AI framework.
Combined with Xiaoice deep neural network rendering technology (XNR), it greatly improves the overall naturalness of Chen Shuiruo's face, expressions and voice.
Now anyone can let Chen Shuiruo sing their own work in real time through the Xiaoice X Studio 2.0 tool.
The "three-dimensional" body of a virtual person
I don't know if you have noticed that whether it is Xia Yubing or Chen Shuiruo, they are both virtual and three-dimensional.
How is this done?
There are only four words: Xiaoice frame!
Xiaoice framework has iterated to the 9th generation from the beginning to the present.
First of all, in terms of content generation, the Xiaoice team disclosed the short video content packaging pipeline and the three-dimensional virtual person product line Xiaoice framework industrialization at the annual press conference on September 22, 2021.
According to the Xiaoice team, since the initiative and active promotion of artificial intelligence creation (content generation) in 2017, Xiaoice has successively launched a number of core technologies and products such as text generation such as poetry, visual generation such as painting and visual design, and artificial intelligence music generation, using the advantages of high stability and large-scale concurrency of artificial intelligence to promote end-to-end content generation capacity.
Artificial intelligence content generation is also called artificial intelligence creation. It is only one part of the Xiaoice framework.
If you expand it, artificial intelligence content generation is actually a very complex system, including three main parts: creation, deduction, and content secondary packaging. And in all three main sections there are a lot of models, engineering, algorithms, and so on.
In the framework of Xiaoice, first train some creative subjects with creative ability, including a large number of basic models of text, vision, and sound. These works are then delivered to the deductive subject for interpretation, and then the content of the derivative works is resealed.
In addition, the short video content packaging pipeline includes two forms: "two-dimensional" short video and "three-dimensional" short video, and greatly reduces the production cost of short video.
What is shocking is that the Xiaoice team will generate the "two-dimensional" short video from text generation to short video generation, the whole link, the whole pipeline is all taken by artificial intelligence, and no manual participation is required in the middle.
The "three-dimensional" short video realizes the entire process from biological feature generation to short video generation, that is to say, it is possible to create a up main video that does not exist in the world from scratch, looking like a real person.
So, the point is, how are these biological traits generated?
At present, the usual virtual human video technology stack is like this:
Use 3D modeling techniques to make a virtual human body, and bind the head. Then use motion capture technology to drive, by the real person like a puppet to manipulate the body, whether it is action or sound, are real people interpreted.
Or, use 3D modeling technology to make a virtual human body, bind to the head, and then use the action library to match the way to drive it to move, and the biggest drawback of this operation is that the action is extremely unnatural - "one look fake".
Relatively speaking, the most natural thing is to find a real person to shoot and then change your head. But the cost is extremely expensive, which can be called "a few grams of gold per second".
With the blessing of Xiaoice framework, the effect is different:
1. By Xiaoice neural network rendering technology, completely non-existent facial features are generated (AI creates faces), completely avoiding the abuse of real faces.
2. Through the Xiaoice neural network rendering technology, the facial features of the virtual person are replaced into the video template, and a series of controls such as expression control are performed (AI replacement face). It sounds a bit like an AI face swap, but the technical complexity varies greatly, and the naturalness can't do that.
3. By Xiaoice digital twin technology, the video template can be adjusted according to the content of the actual video. For example, changes in movements, changes in expressions and expressions, etc.
4. Generate high-quality singing voice lines of virtual people (AI creates sounds) through Xiaoice supernatural voice technology, and generate corresponding songs as needed. In this way, there is no longer any need for any voice actors behind the scenes.
5. Through the above process, the face, expression, and voice are all unified together, forming a content that cannot be distinguished from real people.
Now, the three-dimensional virtual person can be described as the trend of the times, not only a variety of images mushroomed, but also absorb countless powders.
At the end of last year, "Who is Liu Yexi?" The crazy brushing screen in Douyin has also triggered various imitations. Relying on only 2 short videos, Liu Yexi received 4.26 million likes, gained more than 2 million fans, and surpassed 4.3 million followers a week later.
However, the "paradox" that almost all characters cannot escape is whether they want to look natural or want to control costs.
There are exceptions, yes, that is, the Xiaoice that is seen in the unborn.
In 2017, the Xiaoice team filed a patent called Creating a Conversational Chatbot of a Specific Person.
After three years, the patent was officially approved on November 11, 2020.
The patent proposes a system and method for creating a conversational chatbot with a specific character.
For example, if there is a passerby here, then first you need to access his social data (e.g., images, voice data, social media, electronic messages, written letters, etc.).
A special index is then built with the personality of Passerby A as the theme, and this index is used to train chatbots, enabling robots to talk to other people with passers-by's personality.
In addition, you can use images, depth information, and video data from Passerby A to generate 2D or 3D models of specific people.
Speaking of which, is there a strong "black mirror" wind?
The era of virtual humans and humans mixed up?
Some people say that 2021 is the first year of the meta-universe, and it is also the first year of the virtual person.
In 2022, there will be a wave of virtual people.
There is no denying that there is an inextricable relationship between digital virtual humans and metaversics. Virtual people who live in the digital world now may be the twins of our real people in the metaverse in the future.
In the episode "Black Mirror" Be Right Back, the heroine Martha's boyfriend was killed in a car accident.
In order to reproduce the deceased's "sound and smile", the heroine created a new virtual boyfriend with the photos, videos and other information that her boyfriend had posted on social media in the past.
In reality, there is the same story.
In July 2021, an American man, Joshua Barbeau, used GPT-3 to reconstruct an AI chatbot to reproduce his dead fiancée after his fiancée died.
Barbeau uploaded his fiancée's pre-mortem social media information on the Project December system and provided some background information, and then the AI model could mimic his fiancée's language style with astonishing accuracy.
And this is not just a family vision.
On December 15, 2021, Nature published a paper from the MIT research team, "AI-generated characters for supporting personalized learning and well-being."
The paper points out that technological advances in machine learning have made possible surreal synthesis of text, image, audio and video data, that is, AI-generated content and AI-created virtual people, which will fully stimulate people's inspiration and interest.
Creating virtual people through AI-generated content and AI can generate anyone's face, body, and voice out of nothing, from fictional characters to historical figures and even deceased relatives and friends.
However, compared with the Xiaoice framework that can already be applied to industrial production, Nature's paper is still discussing the pipeline of AI content generation, which is still far from the three-dimensional virtual person.
As we look to the future, we foresee that AI content generation and AI creation of virtual people will become the mainstream of human-computer interaction.
This is also an important step into the future metaverse.
In the future, the metacosm can bring more imagination space to virtual people. They can set up scenarios in a completely virtual digital world, and further learn, and then learn new ideas that humans may not have produced in the real physical world.
So, is the virtual person a program plus a GIF, a program plus an emoji?
Xiaoice CEO Li Di introduced, "The real virtual person should use body and soul, that is, the body and the inner soul. And human design, I personally think is a very superficial thing. The appearance is to match the inside."
When the body and soul are together, we can see more vivid digital people. This is also a vision of the creation of the future virtual person Xiaoice.
In the eyes of the Xiaoice team, there will not be only a few artificial intelligence assistants in the future.
AI beings will be ubiquitous, and even each of us will be surrounded by a variety of AI beings with diverse personalities and abilities.
He could be an AI human observer, a big brother with a thick Shandong accent, or he could be your stand-in.
Therefore, virtual humans may even exceed our human population in the future, and the future must be "a world where humans and AI live together."
One More Thing
The Easter egg is coming.
Before 24:00 Chinese New Year's Eve, Xia Yubing can create a unique Expo-level ink landscape painting for each user.
WeChat or Douyin search mini program "AI painter Xia Yubing" can be experienced.