laitimes

Wen Xin's word vs GPT-4 actual measurement! Baidu backwaters and a war exchange

The editorial office was published by Wofeiji

Qubits | Official account QbitAI

A day after the release of GPT-4, all the pressure was given to Baidu.

Just now, Baidu handed over the rolls.

In a word, Baidu's new generation of knowledge enhancement big language model was officially released in the conference room of Baidu headquarters.

In a quiet atmosphere, Li Yanhong stepped into the scene, with a little nervousness in his tone:

The expectation is that we are benchmarking ChatGPT, benchmarking GPT-4, which is a bit high (laughs).

Pregnant in October, we will take you to see what this AI big model Wen Xin looks like.

Wen Xin's word vs GPT-4 actual measurement! Baidu backwaters and a war exchange

Previously, some people turned meme pictures wildly to compare Baidu to the trash bin next to GPT-4.

Others advocate that Baidu is the hope of the entire village.

At the same time as the press conference, the value of Baidu Hong Kong's stock market fell sharply, and related topics also rushed to Weibo's hot search.

But there were also netizens who liked it in the live barrage:

So what is the strength of this Chinese version of ChatGPT?

Let's use the pre-recorded demo demonstrated by Baidu press conference to compare the newly baked GPT-4 and speak with strength first.

Wen Xin's words vs GPT-4

Like GPT-4, Wen Xin Yiyan is a multimodal large model.

Li Yanhong opened by demonstrating the five abilities that Wen Xin Yiyan possesses, including literary creation, commercial copywriting, mathematical logic calculation, Chinese understanding, and multimodal generation.

Wen Xin's words even showed a down-to-earth Sichuan dialect on the spot, and netizens at the scene burst into laughter.

What about other abilities? Let's expand on that.

Literary creation

In terms of literary creation, Li Yanhong moved out of "The Three-Body Problem" author Liu Cixin as soon as he opened the scene.

Let Wen Xin introduce Da Liu in one word, "After all, I am also Liu Cixin's fellow countryman":

Wen Xin's word vs GPT-4 actual measurement! Baidu backwaters and a war exchange

Looks like nothing wrong. Same question to GPT-4?

Well??? Directly moved Liu Cixin's hometown to Honghu City, Hubei Province. Hubei people ecstasy (doge)

Later, Robin Li showed a demo of continuing this article from a philosophical point of view:

Wen Xin's word vs GPT-4 actual measurement! Baidu backwaters and a war exchange

It looks decent and more rational. Old-fashioned, let's also make a wave of comparisons with GPT-4:

Do you want to watch the continuation of "The Three-Body Problem" by GPT-4 this wave, or is it a word from Wen Xin?

Let's take a look at the old partners who played Shi Qiang and Wang Miao in the "Three-Body Problem" TV series, Yu Hewei and Zhang Luyi, what do they have in common?

Wen Xin's word vs GPT-4 actual measurement! Baidu backwaters and a war exchange

No problem.

It seems that the speed of the display is a little fast, a little faster than Li Yanhong's speech speed (manual dog head).

Wen Xin's word vs GPT-4 actual measurement! Baidu backwaters and a war exchange

Commercial copywriting

Next, Li Yanhong showed Wen Xin's ability in commercial copywriting.

For example, give a name to a new company.

Wen Xin's word vs GPT-4 actual measurement! Baidu backwaters and a war exchange

And the name is not yet blind:

Let's take a look at the effect of GPT-4 naming?

It seems that GPT-4's ability to control the Chinese is still a little less essence in comparison.

As for writing a press release on company formation? It doesn't seem to be a problem for Wen Xin's words:

Mathematical logical reckoning

Mathematical ability is a major problem that tests generative large models. When ChatGPT first went online, it also turned over a lot of cars.

However, at the scene, the mathematical problems handled by Wen Xin were not complicated, and it was a common chicken and rabbit cage problem in primary school mathematics competitions.

Wen Xin's word vs GPT-4 actual measurement! Baidu backwaters and a war exchange

Easter egg is the first question that Li Yanhong showed on the spot, and Wen Xin said Gao Qisheng's classic line in "Crazy": This question is not right.

Modify the topic, OK, and throw it to Wen Xin again:

It seems that the answer is quite reasonable, step-by-step logical reasoning.

Li Yanhong said that these questions "dare not say that they can do 100% right, but at least they reflect Wen Xin's thinking process."

Chinese understand

Next, Li Yanhong focused on Wen Xin's ability to understand Chinese, and specially emphasized:

Wen Xin's understanding of Chinese culture should go beyond any pre-trained model.

It is an idiom, "Luoyang paper is expensive." How expensive is it? ”

Wen Xin's word vs GPT-4 actual measurement! Baidu backwaters and a war exchange

This is the effect of Wen Xin's words:

The economics behind this idiom are also explained:

Well, what about throwing this problem at GPT-4? First ask it, do you know what Luoyang paper expensive means:

Next, ask him what his corresponding economic theory is:

This wave seems that the Chinese understanding of GPT-4 does not seem to be inferior to Wen Xin's words.

So, let's take a look at the ability of the two to write Tibetan poems?

The first is the effect of Wen Xin's words:

Next, let's look at what GPT-4 has to say:

Eh, it seems that GPT-4 does not really understand the meaning of "hidden head poem".

In this wave of cultural understanding, it is true that Wen Xin's words are "small victories".

However, in English, Li Yanhong also admitted that although Wen Xin can handle it with a word, his ability is significantly inferior to Chinese.

This is also related to the training data that Baidu can currently use.

Multimodal generation

Finally, Li Yanhong also briefly demonstrated the ability of Wen Xin's multimodal generation.

First of all, let's take a look at the poster for the upcoming 2023 World Intelligent Transportation Conference -

In addition to the ability to convert text to Sichuanese shown above, Wen Xin can also convert text to video.

Li Yanhong instructed to "generate the above content into a video", and soon, in less than a few seconds, Wen Xin finished the subtitles and videos with one word:

Wen Xin's word vs GPT-4 actual measurement! Baidu backwaters and a war exchange

It is somewhat regrettable that in terms of ChatGPT's frequently praised programming, Wen Xin did not demonstrate the relevant capabilities live.

But Wang Haifeng revealed that Wen Xin's training data also includes code.

How does Wen Xin's words "run through"?

Just as ChatGPT was born out of OpenAI's GPT series, Baidu's ERNIE Bot launched this time is also based on Wenxin large model technology.

According to Wang Haifeng, Wen Xin's words are mainly born out of two major models:

Baidu ERNIE series of knowledge enhancement 100 billion models, as well as Baidu's large-scale open domain dialogue model PLATO.

On this basis, six core technologies are mainly adopted.

Three of these are well-known large model techniques, including supervised fine tuning, human feedback reinforcement learning (RLHF), and prompt construction.

Human feedback reinforcement learning is also a key technique for ChatGPT.

The other three are "Baidu's more distinctive" technologies, including knowledge enhancement, retrieval enhancement and dialogue enhancement technology.

Let's start with similar techniques to ChatGPT: supervised tuning, RLHF, and prompt building.

There is supervised fine-tuning, especially for Chinese aspects of data. Based on its understanding of Chinese Chinese culture and Chinese application scenarios, Baidu screened specific data to train the model.

As for reinforcement learning (RLHF) and prompt construction for human feedback, the operation is also similar to ChatGPT.

This was followed by Baidu's proposed technology to further improve the model.

Knowledge enhancement, including two parts: knowledge internalization and knowledge externalization. Among them, knowledge internalization "penetrates" knowledge into the model parameters; External knowledge means that the model can directly use external knowledge.

The search enhancement is related to the search technology accumulated by Baidu search engine.

Baidu will combine search technology and generation technology, first search the content, use the more useful parts for generation, and then integrate the output results:

Finally, there is the dialogue enhancement part, including the memory mechanism, context understanding and dialogue planning techniques accumulated by Baidu before:

In summary, the ability shown by Wen Xin's words was called "intelligent emergence" by Li Yanhong:

This phenomenon occurs when the parameters reach the order of hundreds of billions and the training corpus reaches enough.

At present, Baidu's AI technology can be divided into four parts, chip (Kunlun core), frame (flying paddle), model (Wenxin) and application.

The reason why software and hardware must be laid out, Baidu said, is to reduce costs:

Generative AI requires very high computing power and is quite expensive.

Therefore, if the four-tier architecture is optimized in synergy with each other, it can be made more efficient than others, thereby significantly reducing costs.

Robin Li believes that this is also Baidu's advantage:

There are companies with leading products on all four floors, and there is no other.

The hardware computing power behind Wenxin's large model is also provided through Baidu Intelligent Cloud.

At present, Wen Xin Yiyan has been connected to Baidu search, with the aim of improving the efficiency of search resources.

At the same time, products such as Xiaodu and autonomous driving Apollo, as well as companies such as iQiyi, have also been connected to Baidu's Wenxin Yiyan model.

Netizen: It seems that there is no need to retire early

As of press time, Baidu's Hong Kong stock price has rebounded after a wave of significant declines.

At the end of the press conference, the most reacted by netizens was "recording in advance is a bit of a bad feeling":

For this point, Li Yanhong's explanation is that the questions given are relatively long, in order to save on-site time, so the form of advance recording is used.

There are also many netizens who are not very satisfied with the ability shown by Wen Xin's words. Some people ridiculed that after reading it, they felt that "the days of early retirement look like they can be slowed down":

There is still 4 old hu behind GPT-20.

Some netizens felt that Wen Xin's press conference was like his own graduation defense (doge):

However, some netizens said that they hope to give domestic products a little time and a little patience.

At the end of the press conference, Wang Haifeng announced that Wen Xin's words will be tested externally from today, including individual users and enterprise users.

It is a mule or a horse, and I believe that in the future, it will be verified more.

One More Thing

By the way, some netizens said that they have obtained the qualification for internal testing of Wen Xin's words:

Hello, thank you for experiencing the words of Wen Xin, experience address:

https://yiyan.baidu.com/welcome, I hope you will give more comments during the experience, Wen Xin Yiyan invitation code: KFCVME50RMB, valid until 24:00 on March 16, 2023.

Well, Crazy Thursday (manual dog head).

Read on