Editor: Layan Peach
AI agents may be the next frontier. Today, Noam Brown, the father of Depoo AI, announced his joining OpenAI.
Chief scientist Andrej Karpathy believes that AI agents represent a crazy future, and this is the time to return to neuroscience and seek inspiration from it.
He said that every time there is research related to agents, the OpenAI team has to study it carefully.
OpenAI's bet on AI agents is thorough.
Just today, Noam Brown, the father of Depu AI, officially announced that he has joined OpenAI!
Brown tweeted the heavyweight news, saying that his experience in the field of Depu AI will make LLM 1,000 times stronger than the existing GPT-4.
"I'm excited to announce that I've joined OpenAI! For years, I've been working on how AIs can learn to play poker-like games on their own, as well as train them to reason in games.
Now I have the opportunity to explore how to promote these approaches in a wider context! If successful, one day we can see that the new big language model performs 1,000 times better than GPT-4."
The father of Depu AI
Brown is a research scientist at OpenAI, working on multi-step reasoning, self-play, and multi-agent AI.
Brown previously worked at Meta, where Brown and his teammates developed CICERO, the first AI to reach human level in the strategy game Diplomacy.
Address: https://noambrown.github.io/downloads/diplomacy_science_all.pdf
Brown also applied his research to making the first AI to beat top humans in unlimited poker.
Brown and his advisors at CMU created Libratus and Pluribus, which beat top human puppet masters in a human-machine match.
At the same time, Libratus received the Marvin Minsky AI Outstanding Achievement Award. Pluribus also appeared on the cover of Science and was runner-up in Science's 2019 breakthrough of the year.
He was also named one of MIT Technology Review's 35 Under 35 innovators.
Noam Brown received his Ph.D. in Computer Science from Carnegie Mellon University. Prior to CMU, Brown worked in the Federal Reserve's International Financial Markets Division, studying algorithmic trading in financial markets.
Brown also reviewed the history of the simple version of game AI under Twitter, starting with AlphaGo.
In 2016, AlphaGo defeated Lee Sedol.
But the key is that AI thinks for 1 minute before making every move. How much does this improve it? For AlphaGoZero, this is equivalent to expanding pre-training by a factor of 100,000.
As you can see, the abscissa in the above figure is different versions of AlphaGo, and the abscissa is the Elo score of each version.
Here's a simple insertion of popular science, Elo scoring is a scoring system used to measure the competitive level of chess players, athletes, gamers and so on.
It was first proposed by American chess master Arpad Elo in the 50s of the 20th century in order to assess the level of chess players more fairly.
As you can see, the best level of humanity is probably in the 3500+ position, and the Zero version of AlphaGo with tree search is a great ride, and Elo is as high as 5000+.
And the same phenomenon Brown also found in De Pounce.
The discovery is also the first time that Brown's Libratus card AI has defeated humans.
Jim Fan: Game AI is important
Jim Fan also tweeted that a lot of the content in game AI research can be applied to LLM, which will be a key point.
"I believe that the next generation of big language models will borrow a lot from the research of game AI.
Noam Brown's addition to OpenAI, and DeepMind Gemini's indication that it will borrow technology from AlphaGo, among other things, are proof of this.
This actually makes a lot of sense. The strategy of self-play and self-play used in the training of game AI, as well as the tree search method used in reasoning, allow AI to outperform humans in games such as Go, poker, Dota, StarCraft and so on. The root cause is that these methods improve the ability of models to reason in a highly scalable way.
At present, we have seen many applications of this method in the LLM field. For example, Voyager, which is an algorithm that can infer time, allows AI agents to write code constantly.
Guide their skills in Minecraft.
In addition, the Tree of Thought LLM in context is combined with search engines to promote the advancement of reasoning.
At the same time, more applications are still on the way, waiting to be put into practice."
Resources:
https://twitter.com/polynoamial/status/1676971503261454340
https://twitter.com/DrJimFan/status/1677000660791992320