Generative AI is a technology that uses deep learning models to learn from data and generate new content, which can generate various types of outputs, such as 3D models, videos, animations, music, poetry, etc., based on inputs such as text, images, audio, etc. The advancement of generative AI technology has provided strong support for digital humans, enabling them to adapt to different scenarios and needs, showing unlimited possibilities.
A digital human is a virtual character based on computer graphics and artificial intelligence technology that simulates the appearance, movement, expression and voice of real humans. Digital humans not only have highly realistic visual effects, but also rich emotional expression and behavioral logic. These digital humans can not only serve as content creators and disseminators in entertainment, education, advertising and other fields, but also serve as a new entrance to human-computer interaction, interact with humans in the real world, and provide people with a bridge to connect closely with the virtual world.
In recent years, with the development and application of generative AI technology, the field of digital humans has also ushered in explosive growth. From games, film and television to social networking, live broadcasting, from virtual idols, virtual anchors to virtual customer service, virtual assistants, from 2D to 3D, from static to dynamic, from single to multi-modal, digital humans are appearing in our lives in a diversified and intelligent form. According to the "Analysis of the Status and Opportunities of China's AI Digital Human Market 2022" report released by IDC, it is expected that the size of China's AI digital human market will reach 10.24 billion yuan by 2026.
So, how does generative AI technology empower digital humans? How do digital humans drive market popularity? This article will explore the following aspects.
Generative AI technology provides strong support for digital humans
Generative AI technology, as a technology that uses deep learning models to learn from data and generate new content, can generate various types of outputs, such as 3D models, videos, animations, music, poetry, etc., based on inputs such as text, images, and audio. These outputs are not only of high quality and fidelity, but also highly creative and flexible. Generative AI technology can support digital humans in the following ways:
- Shape generation: Generative AI technology can quickly generate high-fidelity and high-diversity digital human shapes based on user-provided picture or video materials, or user-defined parameters and requirements. For example, NVIDIA released StyleGAN based on GANs (Adversarial Generative Network) technology, which can adjust style parameters according to the user to generate digital human face images with different styles and features; Tencent released Tencent Smart Shadow based on VAE (Variational Autoencoder) technology, which can generate realistic 2D digital humans online by collecting 5-minute video footage from users. SenseTime released SenseAvatar4, which combines GANs technology and 3D modeling technology, to generate realistic 3D digital humans online by collecting 5-minute video footage from users.
- Expression-driven: Generative AI technology can drive digital human facial expressions and lip synchronization, based on text or voice input provided by the user, or real-time facial expressions of the user. For example, NVIDIA released Audio2Face, which is based on GANs technology, which can drive the facial expression and lip synchronization of digital humans based on the voice input provided by the user; Huawei released an AI digital human based on the combination of GANs technology and 3D modeling technology, which can drive the facial expression and lip shape synchronization of digital humans based on text or voice input provided by users, or real-time facial expressions of users.
- Motion capture: Generative AI technology can capture the body movements and postures of digital humans based on video footage provided by the user or real-time body movements of the user. For example, NVIDIA released Video2Pose, which is based on the combination of GANs technology and 3D modeling technology, which can capture the body movements and postures of digital people based on the video footage provided by users; Alibaba released AliMoji, which is based on the combination of GANs technology and 3D modeling technology, which can capture the body movements and postures of digital people based on the user's real-time body movements.
- Speech synthesis: Generative AI technology can synthesize the speech output of digital humans based on text input provided by the user or speech samples provided by the user. For example, Baidu released Deep Voice based on deep neural network technology, which can synthesize the speech output of digital humans based on text input provided by users; Tencent released Real-Time Voice Cloning, based on the combination of deep neural network technology and GANs technology, which can synthesize the speech output of digital humans based on the speech samples provided by users.
- Text generation: Generative AI technology can generate text output of digital humans based on information such as keywords, themes, and styles provided by users, or the content of conversations between users and digital humans. For example, OpenAI released ChatGPT based on the combination of self-attention mechanism and Transformer model, which can generate text output of digital humans based on the content of conversations between users and digital humans; Baidu released Wen Xin Yiyan, based on the combination of self-attention mechanism and Transformer model, which can generate text output of digital humans based on keywords, topics, styles and other information provided by users.
Through generative AI technology, digital humans can not only have more realistic, more diverse, and more personalized shapes and sounds, but also have smarter, more flexible, and more creative text output. This enables digital humans to adapt to different scenarios and needs, showing endless possibilities.