The fairy team behind GPT-4o: The project leader only has a bachelor's degree, and alumni of Peking University/Tsinghua University/Jiaotong University/University of Science and Technology of China are listed

36 Krypton

2024-05-17 17:52Posted on the official account of Beijing 36Kr

Text: Li Ran

Editor|Su Jianxun

Cover source: public information

After OpenAI threw GPT-4o to the world at a more than 20-minute press conference, many bigwigs didn't seem to buy its progress.

GPT-4o is just a small upgrade, native multimodal

Source: X

OpenAI co-founder, now the legendary god AK who has left OpenAI, commented on GPT-4o:

"They released a combined text-audio-vision model that handles all three modalities in a single neural network and can also perform real-time speech translation in special cases if requested by the user."

Boss Ma immediately echoed that this description of GPT-4o disenchantment is more accurate (sour).

However, when Sam Altman forwarded the story behind the development of the GPT-4o team, the outside world saw what kind of fairy team was needed to make the ability of the large model into native multimodality.

Source: X

The core team of 18 people creates a new history of human-computer interaction

Musk once said in an interview that the core significance of OpenAI's launch of ChatGPT is actually only to create an interface for human-computer interaction - ChatGPT, so that ordinary people can communicate with AI with text. And then all of a sudden, the average person realized what incredible things AI can do now. Because in fact, before the advent of ChatGPT, the technology of large language models has existed in the laboratory for a long time and has developed strong capabilities, but ordinary people have not had the opportunity to experience how powerful it really is.

By the way, it also made OpenAI a startup with a valuation of $100 billion.

If GPT-4o is allowed to fully communicate with AI through media and channels other than text, no one may be able to fully realize how much impact GPT-4o will eventually have.

The OpenAI Omni Team, which bears this heavy responsibility, has only 18 people, including 4 Chinese, and almost all of them are post-90s in the team, including the project leader.

团队领导 Prafulla Dhariwal

Source: X

The Indian boss who leads the Omni Team is presumed to be a post-90s generation based on his educational background, and most of the team members are doctoral graduates, but his educational experience is only a bachelor's degree.

来源：linkedin

He joined OpenAI directly as a research intern after graduating from his bachelor's degree. I've been working at OpenAI until now.

来源：Linkedin

Throughout his research career, he has participated in almost all cutting-edge research on machine learning, such as reinforcement learning, unsupervised learning, Scaling Law, etc., and has also participated in research including DALL· E 2，GPT-3，DALL· E3 and other key items.

来源：Linkedin

Before going to university, he represented India in the IMO (International Mathematical Olympiad) and also served as the coach of the Olympiad team. A proper young genius-level character.

From his experience, although in general, the threshold for an AI research scientist is a PhD, and the threshold for leading an AI research scientist may need to be "an AI research scientist with only an undergraduate degree".

Key researchers

The core of the team responsible for all aspects of graphics, audio, data, and post-training: James Betker

来源：Linkedin

He was listed first on the team list, and the team leader commented on him: as long as any task is given to him, he can do it for you!

来源：Linkedin

He worked at Garmin and Google before joining OpenAI.

来源：Linkedin

In particular, on his LinkedIn, there is a letter of recommendation written to him by a previous client:

来源：Linkedin

As James' project leader, I have always been impressed by his adaptability, attention to detail, and strong work ethic as he learns new skills. With minimal training, he was able to educate himself and become an expert on problems within our business area and the tools and platforms used to achieve our goals.

It seems that to work at OpenAI, either you are a genius, or you are the kind of person who impresses even geniuses.

Video Lead: Rowan Zellers

来源：Linkedin

This post-90s doctor's contribution to GPT-4o is to allow the model to see the video like a human.

来源：Linkedin

He came straight to OpenAI after graduating with his PhD.

来源：Linkedin

In the demo video, he is also the one who directly appears in the demo of the model's visual recognition function.

Source: X

He has been involved in several OpenAI projects such as GPT-4, and his previous research interests have also focused on multimodality.

来源：Linkedin

音频方向的负责人:Alexis Conneau

The project director commented on him that he was the first person in OpenAI to propose a real-life reproduction of Samantha in "Her", and ruthlessly executed his vision.

Source: X

This may be evident from his X cover.

Originally from France, he graduated from one of the top engineering universities in France before joining Meta and earning his PhD at FAIR.

来源：Linkedin

Then after working at Google and Meta AI for a while, he joined OpenAI.

来源：Linkedin

At Google and Meta, he has worked on projects and products that have impacted 1 billion users. The paper also won the Best Paper Award at EMNLP.

来源：Linkedin

Five Chinese made key contributions

Just like Sora, which exploded before, there has never been a shortage of Chinese in OpenAI's high-profile projects:

Li JING

来源：Linkedin

This Chinese brother who graduated from the Department of Physics of Peking University has participated in DALLE-, Sora.

He also contributed to the release of GPT-4o this time.

He received his B.S. in Physics from Peking University and his Ph.D. from the Massachusetts Institute of Technology.

来源：Linkedin

He himself has started a business and worked full-time at Meta for 2 and a half years before joining OpenAI in 2022.

来源：Linkedin

Jiahui Yu

来源：Linkedin

He received his bachelor's degree from USTC and his Ph.D. from UIUC. He is now the head of the perception team at OpenAI.

来源：Linkedin

He used to be one of the heads of Google's Gemini multimodality, and was poached by OpenAI in 2023.

来源：Linkedin

Yu Zhang

来源：Linkedin

He received his bachelor's degree from Shanghai Jiaotong University and his Ph.D. degree from MIT.

来源：Linkedin

He interned at Microsoft's Asia Research Institute before joining Google DeepMind and OpenAI from 2023.

来源：Linkedin

HUIWEN Chang

来源：Linkedin

She received her B.A. from Tsinghua University and her Ph.D. from Princeton University.

来源：Linkedin

Prior to joining OpenAI, he worked as a research scientist at Google.

来源：Linkedin

Qiming Yuan

来源：Linkedin

He is responsible for the pre-training data processing of the language in the GPT-4o team, and graduated from Tsinghua University with a bachelor's degree and a master's degree from Austin, Texas.

来源：Linkedin

Prior to joining OpenAI in 2018, he worked at Dropbox and Microsoft.

来源：Linkedin

View original image 85K

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

•