Meta is reportedly testing an AI system that lets people build parts of a virtual world through voice descriptions, and Meta recently demonstrated an AI-driven "Builder Bot" program developed for Horizon Worlds that allows users to build VR worlds entirely from voice.
It is understood that Builder Bot does not place objects through a user interface with a controller, but allows users to build a virtual world in the metaverse by saying what they need. In a pre-recorded demo video, Zuckerberg takes viewers through the process of making a virtual space with Builder Bot, starting with "Take Us to the Seaside," his series of verbal commands that let the AI create a seaside cartoon 3D landscape step by step around him.
Zuckerberg described Builder Bot as a "concept" and explained that "we still have a lot of challenges to solve." In addition, Meta did not explicitly say how much of the demo section was real, nor did it say whether the 3D model was dynamically generated or selected by the AI from the library.
Nowadays, there are already AI projects on the market that have implemented image generation based on text descriptions, including OpenAI's "DALL-E", Nvidia's "GauGAN2" and "VQGAN+CLIP". However, these projects can only generate 2D images without interactions, and the generation of 3D objects is still under investigation.
Meta says Builder Bot was able to do so thanks to SSL, a relatively new way to train AI models that the company has been working on in recent years. Not only that, Meta has also released several other new advances in AI technology, including plans for a universal language translator, a new version of a conversational AI system, and an initiative to build new translation models for languages that don't have large written data sets, which could also be used in the metaverse in the future to allow users to communicate with AI in the metaverse.