· Demis Hassabis, CEO of Google DeepMind, said Gemini combines the technology behind AlphaGo with a big language model with the goal of giving the system new capabilities, such as planning or problem solving, stronger than OpenAI's GPT-4 capabilities.
Demis Hassabis, CEO of Google DeepMind.
Google DeepMind, an artificial intelligence giant formed by a recent merger, has finally issued a substantial challenge to ChatGPT.
At last month's Google I/O developer conference, Google revealed for the first time its large-scale language model, Gemini, which it is developing. According to Wired on June 26, Demis Hassabis, co-founder of DeepMind and CEO of Google DeepMind, recently revealed further details about Gemini in an interview: The system combines the technology behind AlphaGo with a large language model, with the goal of giving the system new capabilities, such as planning or problem solving, stronger than OpenAI's GPT-4 capabilities.
In 2016, AlphaGo, developed by DeepMind, beat Go masters to make history. "You can think of Gemini as combining some of the advantages of the AlphaGo system with the amazing linguistic capabilities of large models." "We also have some new innovations that will be very interesting," Hassabis said. ”
New ideas may be tried
AlphaGo is based on a technique pioneered by DeepMind called reinforcement learning, in which software learns to choose what actions to take to tackle tough problems, such as Go or video games, by repeatedly trying and receiving feedback on its performance. It also uses a method called tree search to explore and memorize possible actions on the board. The next big leap in language models may involve making this technology perform more tasks on the internet and on computers.
Hassabis said Gemini is still in development and that the process will take months and could cost tens or hundreds of millions of dollars. OpenAI CEO Sam Altman revealed in April that it cost more than $100 million to create GPT-4.
Training a large language model like GPT-4 requires machine learning software called Transformer (a deep learning model developed by Google on which OpenAI developed GPT) that feeds large amounts of text from books, web pages, and other sources. It uses patterns in training data to expertly predict the letters and words behind a piece of text, a simple mechanism that proves to be very powerful in answering questions and generating text or code.
Developing language models of ChatGPT and similar capabilities requires an important extra step: refining their performance using reinforcement learning based on human feedback. DeepMind's deep experience in reinforcement learning could allow its researchers to empower Gemini with new capabilities.
Hassabis and his team may also try to enhance large-scale language modeling techniques with ideas from other areas of AI. DeepMind's researchers cover everything from robotics to neuroscience, and this week, the company demonstrated an algorithm that learns to perform manipulative tasks with a variety of different robotic arms.
Researchers at Google's DeepMind recently combined artificial intelligence with a robot called RoboCat and are expected to make a big leap forward in training robots on their own.
It is widely accepted that learning from the world's physical experience, as humans and animals do, is important to make AI more capable. Some AI experts believe that language models learn the world indirectly through text, which is a major limitation.
Google executives attach great importance to Gemini
Gemini is DeepMind's most ambitious project in the field to date. Foreign technology website The Information reported in March that spurred by the failure of Google's chatbot project Bard, in order to keep pace with ChatGPT, Google executives were directly involved in Gemini's research and development, including Jeff Dean, the company's most senior head of artificial intelligence research.
The completion of Gemini development may give Google an edge over generative AI technologies such as ChatGPT. Although Google pioneered many technologies and spawned recent AI innovations, it chose to carefully develop and deploy products. And in April, in response to increasing competition, Google merged its "Google Brain" lab with Hassabis-led DeepMind to become Google DeepMind.
Hassabis said the new team will combine two strong AI teams. "If you look at where we stand in AI, I would say 80 or 90 percent of innovation comes from one of the two." Hassabis said. "Both organizations have achieved some outstanding results over the past decade."
Hassabis himself has sparked an AI mania. In 2014, DeepMind was acquired by Google. Over the next few years, DeepMind showed how AI could do things that once seemed like only humans could do. When AlphaGo defeated Go champion Lee Sedol in 2016, many AI experts were stunned because they thought it would take decades for machines to master such complex games.
It's worth noting that Gemini isn't DeepMind's first foray into the realm of language models. Last year, the company launched Sparrow, a chatbot, claiming that it is less likely than other language models to give "insecure" or "inappropriate" answers. Hassabis said in an interview with Time in January that DeepMind would consider releasing a private beta version of Sparrow sometime this year, and it's unclear if those plans are still ongoing.