"ChatGPT is not innovative."
"This thing that OpenAI is doing is not an improvement over other labs."
In the past two days, Turing Award winner LeCun publicly "sang the opposite" with the popular trend, which instantly triggered netizens to watch.
You know, ChatGPT is powerful and fun, popular on the whole network, anyone who watches it has to sigh "the technology is too strong", but the AI boss "dare not agree" about this?
In this regard, many netizens said that LeCun is "acid" the achievement of OpenAI:
"OpenAI now puts both Google and Meta in the shadow of ChatGPT. That's why it's not a revolutionary innovation. ”
But some academics backed him up, saying that "ChatGPT is an improved version of InstructGPT from a year ago, and I must agree with LeCun."
What's going on?
"ChatGPT is not a technological innovation, it just combines well"
LeCun made those comments at a small gathering of media and executives on Zoom last week.
To say that ChatGPT is not innovative is in terms of the underlying technology.
He said that in the past, many companies and labs have built data-driven artificial intelligence systems such as ChatGPT, so OpenAI's work is not unique.
Although ChatGPT may be the most powerful of these, and the public is impressed by its ability to make a big difference in people's work and lives, it is really not revolutionary science.
He pointed out that in addition to Google Meta, there are 6 other startups with very similar technologies.
"These technologies are public, not rocket science, and if you want to find out, you can find that there is no secret behind it."
(Ps. rocket science, which refers to particularly difficult things.) ）
He called ChatGPT simply "a good combination."
Specifically, the following key technologies are "combined":
First of all, its core is the pre-trained Transformer architecture in a self-supervised learning way.
Among them, self-supervised learning is what "I LeCun" has long advocated. How early? At that time, OpenAI was not yet born.
What about Transformer, not to mention that. It was proposed by Google in 2017, and a large number of language models such as GPT-3 are based on it.
And when it comes to language models, its history is even earlier.
The first neural network language model was proposed by Yoshua Bengio about 20 years ago. (The concept of attention mechanism was also first invented by Bengio, then adopted by Google for use in Transformer, and then became a key element in all language models.)
In other words, ChatGPT not only uses other people's technology, but also has been around for a long time.
In addition, LeCun pointed out that OpenAI's programs use reinforcement learning, which is to rank the output of the machine by a human agent (like Google's page ranking) to improve system performance.
And this approach is not OpenAI, but the first of its kind by Google's DeepMind division.
All in all, LeCun believes that ChatGPT did not "come out of thin air", but was combined on the accumulation of so many technologies over the years.
So, it's not an underlying technological innovation:
It's not so much a case of a scientific breakthrough as a case of decent engineering.
Interestingly, LeCun also found the same example for ChatGPT.
That was the self-driving car that won DARPA's 2005 Challenge, from entrepreneur Sebastian Thrun.
He said that the technology used in the award-winning project is also not particularly innovative in basic science, but is very well designed.
Well, whether it's the live audience or you in front of the screen, you will definitely ask now:
You say OpenAI's ChatGPT is "no big deal," so why aren't companies like Google and Meta making it?
In this regard, LeCun "came prepared", smiled and asked himself:
They either can't or don't want to, because it's too expensive to do this kind of thing, and they don't get much benefit.
Netizen: You're sour
ZDNet compiled these views of LeCun into an article and published it, which was personally forwarded by LeCun himself.
And re-emphasize in the retweet that ChatGPT and other large language models LLM did not come out of nowhere, but the result of decades of contributions from different people.
As a result, no AI lab is already significantly ahead of the rest. (That is to say, OpenAI is still that OpenAI, not because of ChatGPT is better than others)
For this remark, some netizens commented:
LeCun, you're jealous in "acid."
Specific be like:
"LeCun's point of view is so interesting, that is, I want to say that Meta's technology is on par with other labs, and there is no product just because Meta's leadership is unwilling to do it."
"If Galactica [Meta's language model] had achieved ChatGPT-like success, you wouldn't say it was just the result of decades of contributions from various people. Your point of view sounds like you're jealous. I think if you can congratulate OpenAI, people will probably praise you. ”
Words can be considered very sharp.
So, just today, LeCun made this public statement:
I'm not criticizing OpenAI's work or their claims.
Instead, I want to correct the public and media perception, everyone said that it is a "new breakthrough", which is really too high.
Do you think ChatGPT deserves such a rating?
Rich color comes from the temple of Concave Fei, qubit | Official account QbitAI