By understanding one's own behavior, memories, preferences, etc., you can reproduce a virtual avatar that is unique to you, so as to achieve interaction that breaks the dimensional wall, meet friends, and get companionship...... Nowadays, the scenes depicted in science fiction movies are coming to life.
ON OCTOBER 14-18, 2024, GITEX GLOBAL BAY INFORMATION TECHNOLOGY EXPO WAS HELD IN DUBAI. At this globally influential science and technology event, Soul App, a new social platform, exhibited its latest self-developed multi-modal model, which has the characteristics of multi-modal understanding, real anthropomorphism, text dialogue, voice call, and multi-language. At the event site, the audience can generate 3D virtual digital avatars in real time through digital installations, and experience natural, smooth and immersive multi-modal interaction through real-time motion capture and restoration.
Tao Ming, CTO of Soul App, said, "At the globally concerned science and technology conference, we look forward to communicating with innovative companies and technology companies from home and abroad, showcasing the latest application practices and innovative solutions of digital entertainment of Chinese enterprises in the field of social networking, and jointly exploring new possibilities for social development." ”
3D digital twins, innovative interaction between virtual and reality
Now in its 44th year, GITEX GLOBAL has always focused on technology-driven innovation, attracting a wide range of attention and participation from large technology companies, global governments, innovative startups, professional investors, and more. At present, GITEX GLOBAL has developed into the largest and most successful computer communications and consumer electronics exhibition in the Middle East, and is also one of the three major IT exhibitions in the world.
Since the birth of ChatGPT, marking a new stage in the development of artificial intelligence, this year, GITEX GLOBAL 2024 has been upgraded again, spanning two large venues - Dubai World Trade Center and Dubai Ports, with more than 6,700 technology giants and innovative companies from around the world participating in the exhibition, showcasing the most breakthrough technological developments in the field of AI, new discoveries in intelligent connectivity, and new discoveries in digital entertainment, social networking, education, Benchmark cases of application in various fields such as health, bringing together the most cutting-edge technology trends and leading the new trend of industry change.
As one of the earliest Internet platforms in China to introduce AI into social relationships, Soul will bring its latest self-developed multi-modal large model to GITEX GLOBAL 2024, demonstrating the accumulation of AI technology and the latest implementation practice in social scenarios. This is also the first time that Soul has appeared at a large-scale international exhibition, and at the conference, the audience can focus on experiencing Soul's multi-modal AI interaction solution that integrates 3D virtual human capabilities.
In order to reduce the social pressure of users, Soul has not supported users to upload real avatars since it was launched in 2016.
In 2022, Soul integrated AI, rendering, image processing and other technologies, and launched its self-developed NAWA engine to provide technical support for users to create personalized 3D social images and scenes. Based on this engine, users can independently create vivid avatars, shape the avatar of the online world, and flexibly cooperate with various information such as voice and text to show each person's distinctive personality, accurately convey emotional temperature, and communicate and interact with others freely in various virtual and real fusion scenes, and feel a new social experience.
Combining the past technical reserves and breakthroughs in the research and development of large models, at this stage, Soul 3D virtual human capabilities have been comprehensively upgraded, forming a mature multi-modal AI interaction solution that integrates 3D virtual human capabilities - that is, a multi-modal large model direction that integrates text, voice, and action interaction, to achieve an interactive experience closer to human mode and more efficient, natural, and multi-dimensional information transmission.
At the conference site, users who participated in the experience can realize AI modeling of 3D virtual humans with high similarity, and quickly restore the characteristics of real faces in the 3D world in a few seconds by including more than 90 shape parameters and 6 attribute parameters of human faces, creating exclusive virtual avatars.
At the same time, based on real-time human action recognition, digital restoration and multi-modal dialogue interaction capabilities, the scene can realize the immersive interaction between 3D virtual humans and real people.
Multimodal end-to-end large model, super-anthropomorphic emotional experience
In fact, including language, dialogue, 3D virtual humans and other dimensions, Soul focuses on the direction of multi-modal large models.
In 2020, Soul officially launched the technical research and development of AIGC, systematically promoted the research and development of AIGC's key technical capabilities such as intelligent dialogue, voice technology, and 3D virtual humans, and promoted the rapid implementation of AI capabilities in social scenarios.
At present, Soul has successively launched the self-developed language model Soul X, as well as speech generation models, speech recognition models, voice dialogue models, music generation models and other voice model capabilities. In June this year, Soul also launched its self-developed end-to-end full-duplex voice call model earlier in the social industry, which has the characteristics of ultra-low interaction delay, fast automatic interruption, ultra-real voice expression and emotional perception and understanding ability, which can directly understand the rich sound world, support super-anthropomorphic multi-style language, and realize interactive dialogue and "human-like" emotional companionship experience closer to daily life.
In 2024, the Soul AI large model capability will be upgraded to a multi-modal end-to-end large model, supporting text dialogue, voice calls, multi-language, multi-modal understanding, real anthropomorphism and other features.
The launch of the multi-modal end-to-end model marks Soul's innovative breakthrough in human-computer interaction experience, and at the same time, the modal upgrade from text, voice to vision also means a subversive change in the way of interaction.
For example, Soul has launched the "digital clone" function based on its self-developed AI capabilities, which allows users to directly authorize the platform to set the image and characteristics of the digital clone based on chat records, post content, or in a customized way, and make the digital clone achieve the effect of replicating the real person to the greatest extent in the representation layer (image, voice, text style), identity layer (social relationships, long-term memory, personality information) and cognitive layer (decision-making, opinions, preferences). The "digital avatar" can not only achieve more personalized and multifaceted intelligent reply recommendations, help users break the ice for social networking, but also improve the efficiency of social communication in helping people build and cognitive decision-making.
Next, through the newly integrated 3D virtual human capability and multi-modal end-to-end large model AI interaction solution, the 3D virtual human independently created by Soul users can be used as a multi-modal all-round assistant in the digital world, fully empowering users to discover, establish and precipitate relationships in rich social scenarios such as platform group chat parties and instant squares, while expanding new relationships, providing high-quality, interesting and immersive human-computer interaction experience, and feeding back real and natural emotional companionship.
Tao Ming, CTO of Soul App, said, "As a natural traffic gathering place and interactive entrance, social networking is regarded as one of the best scenarios for AI to take the lead. We will continue to increase investment in AI technology based on the actual social needs and specific social scenarios of users, so as to bring long-term sustainable value to users. It is expected that by the end of this year, Soul's multi-modal end-to-end model will be upgraded again, and full-duplex video call capabilities will be launched, allowing users to truly conveniently and naturally experience multi-modal innovative interactions including text, voice, and vision. ”