Produced | Tiger Sniff Commercial, Consumer and Mobile Group
The author | hollow triangle
Caption| virtual digital person AYAYI, the official website of Burning Wheat Technology
On March 25, the Douyin @ Liu Yexi account updated the fifth episode of the short drama "Ground Branch Maze", in the context of science fiction + fantasy, virtual digital people and real human beings fell on each other, and cold technology and warm humanity burst out of the spark. In the comments section, in addition to the praise of "you can actually watch movies on Douyin", there is also the curiosity of "asking the author how much this video will cost" and the expectation of "is it time to update".
In the past six months, the "Liu Yexi" account has actually only released 11 works, except for a few promotional videos of about 30 seconds, only the initial debut video and 5 4-minute positive films. As we all know, the Internet has no long-term memory, and such a production is far from satisfactory for short video creators with Vibrato as the main position.
In fact, for domestic users, the past six months have been the period of the highest sense of existence of virtual digital people since its birth.
On the one hand, the technology giants represented by Meta (Facebook) have once again thrown out the concept of metaversics, triggering market pursuit, "virtual digital people" are regarded as the medium for future human beings to enter the metaverse, and then stand on the public opinion outlet; on the other hand, the Beijing Winter Olympics will become a display stage for the current virtual digital human technology, whether it is the virtual twin of sports entertainment stars (such as Gu Ailing's virtual twin Meet Gu), or the public service-oriented virtual anchor (such as CCTV AI sign language anchor), All of them have made the public have a more intuitive and in-depth understanding of virtual digital people.
Meet Gu 3D Avatar stylized digital minds/Image: Web
The data can also reflect the popularity of the domestic virtual digital person industry. According to Qixinbao data, the number of new domestic virtual digital person-related enterprises in 2020 was 36,080, and in 2021, the data reached 66,293, an increase of 80% year-on-year. According to the data of Tianyancha, there are about 300,000 "virtual digital people" related enterprises in the mainland.
The number of virtual digital person projects currently being incubated is not known for the time being, but according to the parallel model of a team and a technical link supporting multiple virtual digital person IP incubation at the same time, this number should be tens of thousands, or even more.
If we blur the history of virtual digital people in the pure concept stage, then Crypto Future Media's "Hatsune Mirai" launched in 2007 can be regarded as the first widely known virtual digital person, and the virtual digital people we have seen before are mostly virtual idols and Vtubers with two-dimensional images. In this round of boom, virtual digital people have gradually begun to diversify their development: at the external perception level, relying on a new generation of engines and technologies, hyper-realistic digital people and virtual avatars have become difficult to distinguish between true and false and real human beings; at the functional application level, virtual employees and virtual anchors with more service attributes have begun to show a trend of large-scale application.
In the "2022 China Virtual Person Industry Development Research Report" released by Ai Media Consulting, the development process of the domestic virtual person industry is divided into four stages: the embryonic stage (1980-1999), the exploration stage (2000-2014), the primary stage (2015-2019), and the development stage (2020 to the present), and the characteristics of the development stage are, "There are breakthroughs in technology and demand, and virtual idols attract young people with unique personalities." ”
The development process of China's virtual person industry / Photo: Ai Media Consulting
At the current stage of development, the entire industry has entered a period of rapid expansion, but there is still a distance from maturity. In the near future, virtual digital people still do not have a widely recognized authoritative definition and standardized classification; in the long run, the metaverse concept is still early, and virtual digital people lack environmental support.
So from now on, the digital people who have hatched and will soon hatch may face the same problem - how to live? Like many new things in the Internet era, digital people will also experience the test of "mushrooming after the rain - big waves - survival of the fittest", so technology iteration, long-term operation, and commercialization path may be the most important issues for the industry in the near future.
Note: This paper adopts the classification method in the "China Virtual Digital Influence Index Report" of the Communication University of China. Technically, virtual digital people can be divided into two categories: intelligent drive and real person (medium person) drive. In terms of application, it includes three categories: identity type (such as real person virtual avatar), service type (such as virtual employee), and performance type (such as virtual idol). ”
Technology ahead?
The production of virtual digital people is a relatively complex process. Taking the virtual digital astronaut Xiaoyi launched by Xinhua News Agency and NExT Studios, a subsidiary of Tencent Interactive Entertainment, as an example, the NExT technical team introduced the production process of hyper-realistic digital people.
Scanning: The Photogrammetry scanning system is used to scan a full range of specific actors, including face models, bodies, clothes, etc., and then generate a point cloud model.
Modeling: For body and facial features, the point cloud model is converted to a low mode to facilitate subsequent binding production. At the same time, handmade scans are wrong or cannot be scanned, such as the inside of the mouth, eyes, teeth, hair.
Binding: Based on FACS (Facial Expression Coding System), MotionBuilder (3D character animation software) and Maya (3D modeling and animation software), complete the binding of faces and bodies to generate simulated avatars.
Animation: The use of face and body motion capture scheme, combined with the post-artificial animation refinement, to achieve the dynamic display of the avatar.
Rendering: With Unreal Engine, character rendering and animation output are completed, and what you see is what you get.
The production of virtual digital people, like other industrial processes, not only includes product iteration, but also accompanies the innovation of production tools. For example, through the development of face-based expression model (Blendshape) production tools, the number of expression models is greatly increased to solve the problem of stiff facial expressions of virtual images; through automated algorithms, the problem of character wearing molds caused by the different proportions of virtual characters and real actors is solved.
Since 2017, the NExT Studios team has gradually explored two hyper-realistic virtual human production pipelines, named xFaceBuilder and xMoCap, corresponding to the face and body production of virtual digital people, in addition to the digital human field, both of which are also applied to Tencent's PC games and mobile game projects.
Virtual digital astronaut testimonial/Photo: Xinhua News Agency
At present, most large technology companies have technical teams to test the waters of virtual digital human incubation. For application parties that have actual needs and only lack technical reserves, Baidu Intelligent Yunxiling Digital Human Platform provides a relatively simple one-stop access solution.
Li Shiyan, head of Baidu's intelligent cloud AI human-computer interaction laboratory, introduced that the Xiling platform can meet the low-cost and rapid customization of digital people, and provide operational support for human design, business and content. Among them, 3D cartoon digital people can be generated by face recognition and photo modeling minutes, and 2D plane digital people can be generated by uploading real video hours. In terms of 3D high-definition digital people, non-specific digital people (no real or virtual specific reference images) support minute-level generation and support AI-driven; special-specific digital people (with specific reference image requirements) have the highest cost, requiring 10 to 30 days.
It should be noted that the service needs of digital people and virtual employees generated quickly and cheaply fit well, and the training can basically be seamlessly connected to the current application scenarios. However, for performance-oriented needs such as virtual idols, generating virtual images is only the first step, and more refined polishing and long-term operational support are essential.
CCTV AI Sign Language Anchor/Image: CCTV News
A number of virtual digital human teams that have communicated with Tiger Sniff believe that the industrial production of digital people is coming. It can be said that the current virtual digital human technology is developing rapidly and iteratively, at least in appearance has been infinitely close to what is thought, so that some hyper-realistic digital people are identified as fake precisely because they are too perfect at the aesthetic level.
But at the same time, the bottleneck in the interaction of digital people is obvious, and the existing AI technology is not enough to support digital people to develop independent behavior, and independent thinking is even further. Only under the trend concept and technology drive, the development of digital people is one step ahead of demand, but it is precisely because of the bottleneck of technology that digital people have not yet opened up their own survival path. Reflected in the present, if every digital person wants to become a long-term "person", it is more dependent on the behind-the-scenes operation team to support.
Operational follow-up is weak
The application of identity virtual digital people is not considered for the time being, and the current performance virtual digital people are mainly Based on To C, while practical digital people are mainly based on To B and To G applications. Staying active and keeping an eye on it is the sword of Damocles that hangs over every virtual digital person's head.
At present, the exposure methods of virtual digital people are mainly in the following ways: publishing offline videos and pictures (such as Liu Yexi, AYAYI and other hyper-realistic digital people); talent live broadcasts, stage performances (such as A-SOUL, Luo Tianyi and other two-dimensional virtual idols); participation in large-scale activities and special scene reports (such as CCTV Xiao C, Xinhua News Agency XiaoYi and other media virtual employees). The challenges faced by the above-mentioned digital human IP are also problems that the entire industry must face.
One is cost. On October 31 last year, Liu Yexi became popular with a short video on the theme of "Catching Demons", rising by 3.63 million fans overnight, ranking first in the "China Virtual Digital Human Influence Index Report" released by the Communication University of China. However, in the past 5 months, the total number of videos released by Liu Yexi's official account is 11, the update frequency is twice a month, and the "urging" in fan messages appears frequently.
AYAYI, ranked fifth in the Impact Index Report, has been active in the print media, maintaining a high exposure frequency through hard photos. However, looking at its Little Red Book dynamics, it can be found that the high-frequency photos did not continue to increase popularity and interaction, but fell from the initial peak, and the comment area below the dynamic also received feedback from time to time that the quality of the new film was not as exquisite as the early photos.
Phenomenon-level virtual digital person Liu Yexi / Image: Network
According to Baidu Li Shiyan, "The cost of using traditional methods to produce CG-level videos and posters is very high, and one minute of video may be short for a month, and 2 to 3 months is also normal." The present instruments of production have not been particularly well done, limiting the productive forces, which in turn further limit the innovation of the relations of production. In the future, through artificial intelligence deep learning, the production cycle can be compressed to a certain extent. ”
Tencent's NExT Studios team also confirmed the cost bottleneck, "Hyper-realistic digital people are the crown jewels of virtual digital people technology, which is much higher than the production difficulty of secondary paper people, the domestic industry accumulation is limited, related technical talents are also scarce, and the production capacity of most domestic manufacturers is suppressed at a relatively primary level." But at the same time, the team only needs to have a long board in one or two links, and it is easy to stand out.
At present, most of the virtual digital production companies on the market are entrepreneurial companies, with strong demand for short-term financing and realization, so they will pursue rapid production or scale, which is difficult to take into account for Seiko. A hyper-realistic virtual digital figure may have to invest at least 1 million to 2 million production costs to not be ashamed to see people, and even if it is invested, it may not be able to make a satisfactory role. ”
According to NVIDIA's official blog, the 14-second video of Huang Jenxun's virtual person appearing at the press conference has a total of 34 3D artists and 15 software engineers participating in the collaboration, totaling nearly 1,000 man-hours.
The second is content and operation. For virtual idols such as A-SOUL and Luo Tianyi, the development path that can be used as a direct reference is star artists, so business capabilities and fan operation capabilities are two hard indicators. It's just that the domestic Vtuber industry started late, and has been confined to the minority, professional middle-aged people, creative teams that can customize content for virtual idols are scarce; at the same time, there are only a few operation teams that can understand "the second dimension generates power for love", can play the fan economy, and can be familiar with the culture of the rice circle.
Lehua Entertainment, which has many years of experience in cultivating and managing traffic stars, can be said to represent a high level of domestic virtual idol operation. In the past year, Lehua has cooperated with Fang Wenshan and Xu Song to tailor new works for A-SOUL, and has also made mistakes that are not refined enough on more than one occasion - such as the "Jiaran Live Play Mobile Phone" incident and the "Suspected Exposure to Late People" incident.
Virtual idol group A-SOUL/Image: A-SOUL's official Weibo
In addition, there are also some cases that reflect the current barriers to digital human operations. In the middle of last year, the national style realistic digital _Ling released the Notes of the Little Red Book, saying that a certain brand of lipstick was "moist and not dry", which caused netizens to question. "Does the AI know if lipstick is good or not?" "AI is almost the same as the character with clothes, cosmetics, skin care products are free."
Lil Miquela, the originator of foreign digital Internet celebrities, has also been controversial - in 2019, Lil Miquela shared a process of being harassed when sharing a taxi with strangers through Vlog, but netizens believe that sexual assault as a marketing means is obviously out of bounds.
It can be seen that the user's trust in digital people is mainly concentrated in audiovisual perception. Although digital humans are physically close enough to humans, the gap between reality and virtuality is still difficult to cross.
However, in Li Shiyan's view, time can solve many problems, "because of the development of computer technology, has been pursuing the digitization of the information of the physical world, if there is enough development cycle, it feels that everything else can be digitized." ”
The third is to break the homogenization. At present, there are a large number of virtual hosts, virtual anchors, Vtubers, etc., which are the scenarios with the most AI-driven applications. 7 * 24 hours on the job, anytime, anywhere work characteristics reflect the advantages of technology. However, as mentioned in the "Impact Index Report" mentioned above, the current virtual anchor technology has more leverage and less autonomy, which needs attention and improvement. "Homogeneous competition" is an important challenge for virtual anchors, whether it is professional media operation, or enterprise or individual operation, its hosting and live broadcast functions are similar, and to highlight the siege, it requires stronger content planning and operation capabilities.
Tencent's NExT Studios team said, "Virtual digital people are an emerging product, and it is normal to attract some attention, heat and praise in the market. With the increasingly advanced digital human technology and the lowering of the threshold, such as the launch of tools such as NVIDIA's Omniverse Avatar and Epic's MetaHuman Creator, users can see more and more things of the same level, aesthetic taste will also increase, and the demand for excellent works will be magnified. At that time, users will not only be attracted by the beautiful image of virtual digital people, but will further see the performance, connotation and even spirit behind TA. Then the kind of digital people who only rely on a skin and a picture, and lack the ability to operate deeply behind the image, will definitely quickly recede and die on the beach. ”
The business path is unclear
According to the "Virtual Digital People Deep Industry Report" released by Qubit, by 2030, the overall market size of virtual digital people in the mainland will reach 270 billion, of which identity digital people (including virtual idols, virtual avatars, etc.) account for about 175 billion, and service-oriented digital people are about 95 billion, and the market is still in the cultivation stage. As for the application level, virtual anchors that replace live services and virtual idols with IP potential are current market hotspots.
In the data provided by Ai Media Consulting, the virtual idol industry maintains a stable growth trend, in 2021, the overall market size and core market size driven by virtual idols are 107.49 billion yuan and 6.22 billion yuan, respectively, and it is expected to reach 186.61 billion yuan and 12.08 billion yuan in 2022, respectively.
Tencent's NExT Studios team also believes that virtual idols and Vtubers may be the most promising direction for commercialization at present, "After all, they can interact directly with users, and the imagination space is larger than that of one-way CG videos." ”
At a time when the prospects of virtual digital people are unclear, the virtual idol group A-SOUL has injected a lot of confidence into the industry. On March 8, Lehua Entertainment submitted a prospectus to the Hong Kong Stock Exchange, and through the interpretation of the prospectus by various institutions, it was speculated that A-SOUL's revenue in 2021 should be no less than 17 million. Although this figure accounts for less than 3% of Lehua's total revenue, it is far from being comparable to Lehua's traffic stars, but it is already the best result achieved by virtual digital people so far.
Domestic virtual idol industry forecast / Photo: Ai Media Consulting
As for other performance-type and service-oriented digital people, in addition to traffic monetization, there is no clear business path at present. Virtual anchors with goods can save the labor costs of merchants, but whether they can replace the evaluation function of real people, whether they are competitive or competitive relationships with real people still need to be verified; with the help of the Spring Festival Gala and the Winter Olympics, virtual media anchors and reporters who have attracted widespread attention seem to have completed their mission and returned to the previous tepid state; and most of the "virtual employees" of the current enterprises are not much different from the previous intelligent customer service, and only a few use unique people to show the corporate brand image and IP for marketing.
Tiger Sniff also learned from the AYAYI incubation team, "At present, digital people only account for a quarter of the company's business sector, when AYAYI led the digital people industry, in fact, her task has been completed, we are now preparing digital labels, incubating digital artists, holding digital art exhibitions, launching digital collection platforms, etc." ”
Every class, and even every digital person, may be exploring their own way of survival. So what kind of digital people are likely to be close to success for some time to come?
In the more recent stage, tencent's NExT Studios team believes that the mainstream way to create virtual digital human IP may be as follows:
1. Stars with IP attributes. For example, the digital reproduction of real people such as virtual Teresa Teng, Mei Lanfang, and Will Smith, the operation idea is actually the reproduction of real stars, which is a form of digital people with the most technical and product challenges.
2. Virtual digital person IP linked with real people. This kind of IP operation method will have more professional and more global requirements for the operator, generally with the participation of traditional artist agencies, mainly to break through the existing user circle and avoid the risk of real people collapsing.
3. Create IP based on mature MCN influencers. In this dimension, due to the considerable cost of technology investment, virtual digital people will inevitably tend to be inferior.
4. IP created based on story and game virtual world. This kind of IP creation method is more in line with the traditional IP creation idea, from text to flat, three-dimensional, dynamic... The content form will also cover all dimensions from traditional paper media and multi-screen coverage to new media, which belongs to the form of exalting and high- The overall investment is huge, and the probability of success is funnel-shaped, only the enduring content and quality can continue to impress the audience and form a real IP.
5. Virtual digital person IP based on quadratic image. Although this kind of virtual idol is relatively low in technical threshold, it is difficult for users to reach, and it is not possible to succeed just by throwing money at people.
Domestic virtual person industry chain / Photo: Ai Media Consulting
In the slightly longer cycle, Baidu Li Shiyan believes that digital people are interactive carriers based on computer platforms and will present paragraph-style development. "The current era of flat computing is mainly based on service-oriented and performative digital human applications, and the next stage is a larger era, temporarily defined as the era of spatial computing." The information presented by the computing platform in the era of spatial computing is no longer flat, but real-time three-dimensional, when the behavior of core users will probably have four categories of socialization, acquisition of goods, information consumption, and access to services, then the virtual twin that can provide personalized interaction, can provide emotional value and content value is a must, and the service-oriented digital person who can provide humanistic emotional care and ensure efficiency is another application form. In the future era of spatial computing, there will certainly be a larger market than the current era of flat computing. ”
Write at the end
Just as the core contradiction thrown out in Liu Yexi's "Ground Branch Maze" short film - with the further development of virtual digital human technology, public opinion bubbles, ethical constraints and other issues may become potential risks, and human beings will also face the issues of AI and humans, digital humans and human beings.
In this regard, Li Shiyan believes that "in the current virtual digital human field, artificial intelligence is more like a production tool, and human creativity plus production tools can produce content more efficiently and at a lower cost." ”
Tencent's NExT Studios team also shared their views.
As a product of science and technology, the "soul" or "god" behind the virtual digital person still needs to rely on people or artificial intelligence to fully express. With the development of computer graphics technology, motion capture technology and AI technology, virtual digital people will make great progress in scene technology, emotional technology, and interaction technology, which will make future digital people closer and closer to real people, or even surpass.
Overall, the evolutionary path of virtual digital humans will move from anthropomorphism to homophotoization, and will eventually move towards superhumanization and realize the realization of authenticity and super-reality. In the anthropomorphic stage, the digital virtual person is a highly realistic three-dimensional animated character synthesized by the computer virtual; in the fanning stage, the appearance, emotion, communication ability, understanding ability and other aspects of the digital virtual person are the same as the real person; in the superhuman stage, the digital virtual person's ability will surpass the natural person.
The development of virtual digital people in the future cannot rely on technology alone, and to have sufficient competitiveness, it is necessary to create a new experience and a deep understanding of the entire application scenario. Meeting the needs of users, giving users an unprecedented emotional experience, and allowing users to accept virtual people is the only way for virtual people to succeed. ”