Baidu "Xilang" officially opened for internal testing

On December 21, Baidu released the first domestic meta-universe product "Xilang" officially opened for directional internal testing, and users can enter the Xilang space for advanced experience with the invitation code. Baidu said that on December 27, "Greem" will be open to all users, when Baidu Create2021 (Baidu AI Developer Conference) will be held in "Greem", which is the first large-scale conference held in the metaverse in China.

At the "Xilang Advanced Exploration Day", Hou Zhenyu, senior vice president of Baidu, and Ma Jie, vice president of Baidu, systematically shared how "Xilang" was built and the opportunities and challenges facing the future development of the meta-universe industry.

Ma Jie said that visual, auditory and interactive problems are the three major technical problems hindering the development of the metaverse, and this time Greem officially upgraded to the -6.0 version. Relying on Baidu Brain's leading ability in the fields of visual, speech and natural language understanding and the powerful computing power of Baidu Intelligent Cloud, "Xi'an" took the lead in realizing 100,000 people interacting with the same screen and "10,000 concert-level" real sound effects restoration in China.

"The metaverse's demand for hash rate is nearly infinite. The AI-native cloud created by Baidu Intelligent Cloud provides the World with PFLOPS-level super heterogeneous computing power, hundreds of GB of bandwidth resources, and infinitely expandable storage to support this ultra-large-scale data transmission project. Hou Zhenyu said.

In addition, behind the functions of generating 3DAvatar avatars through a picture or sentence in the universe and summoning intelligent voice assistants anytime, anywhere, Baidu's series of intelligent technologies have created a more realistic and immersive metaverse participation experience for "Xi'an".

"After more than a year of development testing, The Version Number of Theramus is still a negative number, there are still many imperfections, and it still has a lot of room to grow from the goals we envisioned." Ma Jie said, "At present, the metacosm is still in a very early stage of industrial exploration, and its development is gradual, and it will take a long time for the entire community to build and mature together. ”

The development of the metacosm is facing three technological mountains

"Under the current network technology conditions, vision, hearing and interaction are the three mountains that hinder the development of the metaverse, and the challenges brought by technical difficulties are very large." Ma Jie said that the path to solving the metaverse problem needs to break through a series of technical bottlenecks and require the support of strong basic technical capabilities. These key technologies include: secure, autonomous and intelligent cloud computing technologies; continuous innovation in artificial intelligence, ARM cloud, edge computing, intelligent video, VR and AR; and the construction of content ecology and a huge market foundation.

It is understood that Baidu's efforts are to become an infrastructure platform for the metacosm, providing AI, cloud computing and other technology engines for Greem and other metaversal products. Baidu has been deeply involved in artificial intelligence technology for many years, and has industry-leading technological advantages in the fields of intelligent vision, intelligent speech, natural language processing, and knowledge graph. In October 2021, Baidu officially released the Baidu VR2.0 industrialization platform based on Baidu Brain. The platform includes two major platforms, VR content platform and VR interaction platform, the former around the material collection, edit management, content distribution and collection of equipment / material / unified protocol, so that the content consumption channel is smoother; the latter around the metaverse scene, avatar, multi-person interaction, VR headset / social network, to explore more possibilities of three-dimensional information in the metaverse.

Take the lead in realizing 100,000 people interacting with the same screen

Relying on Baidu's leading capabilities in the fields of vision, speech, natural language understanding and other fields, a series of intelligent technologies are empowering the world of Greem.

On the one hand, as the most natural and basic interaction within the metaverse scene, hearing provides important near-far locations and spatial cues. The tens of thousands of people held offline will have natural sound effects of indoor halls, stadiums, outdoor open-air stages and other spaces, but how to create a more realistic participation experience when the 100,000 people conference is placed online?

According to Ma Jie, based on the spatial sound field reconstruction technology, through the modeling of the acoustic characteristics of the 100,000-person stadium, the main venue of the Baidu Create Conference took the lead in the industry to realize the reconstruction of the stage center speaker's sound field of the super-large open air complex open venue, integrating the real-time background sound restoration of the voice of the 100,000-person real-time audience, and relying on 3D sound effect technology to provide users with immersive communication space sound effects covering the immersive effects of the surrounding users, realizing a more realistic sense of space, layering and immersion.

As a result, the participants are placed in the venue, and they can feel the effect of being in the middle of the 10,000-person venue through the live-action mixing technology, similar to the grand shock of 10,000-person concerts and football matches. At the same time, users can also listen to the chat conversations of participants within 5 meters in all corners of CreatorCity, and as the user's location changes, new chat content will be heard accordingly.

On the other hand, unlike the online games we are accustomed to, the identity system, social system and anytime and anywhere requirements of the metaverse require the platform to place users under the same server as much as possible and ensure a low-latency, high-immersion user experience. At the Baidu Create Conference, with the help of data transmission algorithm tuning and local rendering engine tuning, Baidu created the first super venue in the metaverse that can accommodate 100,000 people interacting with the same screen, realizing mutual visibility and real-time interaction based on different locations of participants under the condition of 100,000 user scale gathering, and improving the accuracy of data and optimizing the stability of the system through the construction of large-capacity high-concurrency servers.

Hou Zhenyu said, "The demand for computing power in the metacosm is nearly infinite. The AI-native cloud created by Baidu Intelligent Cloud provides the World with PFLOPS-level super heterogeneous computing power, hundreds of GB of bandwidth resources, and infinitely expandable storage to support this ultra-large-scale data transmission project. ”

It is worth mentioning that in this Baidu Create Conference, the audience can also participate in the conference through ARM cloud mobile phones in addition to mobile APP, PC computers, and VR headsets. No need to download the client, click-to-watch. Baidu's self-developed Panyu Hive Server has built a high-density, high-concurrency, high-performance, large-scale connected virtual reality computing power cluster for Greem.

In addition, based on the one-sentence avatar generation function created by Baidu AR multi-modal technology integration, users can generate a personalized avatar through a simple description of a sentence in the World of Koham, making the production of 3D virtual digital images "as easy to do as it is to say"; at the same time, through a small degree of dialogue-based artificial intelligence ecology, voice has become the main interactive method in the World of Greem.

Fully open to developers

During the Baidu Create Conference, the "Xi'aang Open Platform" was officially open to the majority of developers, and the City of Creators belongs to every developer. Based on the Xilang Open Platform, the 2021 Baidu "Xilang Cup" Developer Competition was held at the same time, and all the shortlisted developer works will be displayed in the developer block of the virtual space of the Baidu Create Conference.

Since Baidu first appeared in DEFCONCHINAParty earlier this year, Greeng has carried out continuous technical iterations, hoping to create a metaverse prototype that can be reached, experienced and improved. Ma Jie said that Xi'an will be officially open to developers, and invites the majority of partners and developers to promote The maturity of Xi'an and build a prosperous ecosystem for the domestic meta-universe.

"Bringing together the industry's leading AI, cloud computing and VR technologies, Greem's ultimate goal is to create a platform for our partners and developers, a platform where imagination can take root. In this second human space that will soon be parallel to the real world, we also look forward to its infinite possibilities and ultimately carry the bright future of HighTech-BetterLife. Ma Jie finally said. (Li Haoyue)

(Editing by Li Bo)