I pulled the ChatGPTs together for a round of college entrance examination essay smashes

Recently, there are too many hot spots in the technology circle, which is roughly like this:

ChatGPT-3.5 released... New Bing release... ChatGPT-4 released... Wen Xin released a word...

Waves of AI news make people feel that "The Matrix" is about to shine into the real world.

Regarding their evaluation, I believe you have seen a lot.

But I don't know if you have ever been curious: science and engineering to the content, right and wrong, you can understand the level of these AIs at a glance, but the creative class, especially text writing, always makes people scratch their heads.

——It all looks like it's all written, but it all feels similar?

Today, Shichao wants to live the whole thing: invite them to write a college entrance examination essay together, and then ask a college entrance examination essay teacher to correct it ~

Although the college entrance examination essay topic does not represent everything, it is a relatively fair and familiar measure.

The test title we selected is the 2022 Tianjin College Entrance Examination Essay, with a full score of 60 points ↓ ↓

The invited teacher is a senior Tianjin college entrance examination essay teacher.

Next, the World Super will first show the AI composition, and then show the teacher's rating, and when interested friends look at the example text, you can also give them a score by yourself~

Player 1:

Teacher's analysis: The article intercepts ordinary moments and scenes in life, expounds the preciousness of ordinary life represented by pyrotechnics, and the writing is smooth and beautiful. The theme of the ending is sublimated into cherishing the current life, experiencing the beauty of the ordinary, and the intention is appropriate. The text is slightly detached from the subject when it comes to the discussion of courage and dedication. This article has basically reached the second level of college entrance examination composition, which is the middle and upper level of general candidates' writing. From the perspective of AI authoring, the quality of the article is 50% better than that of the average candidate.

Teacher rating: 45

Player 2:

Teacher's analysis: The article uses the author's own experience to explain pyrotechnics, and the entry point is desirable, but the expression of the connection between pyrotechnics and his own life in the text is slightly stiff, and the final sublimation should also be carried out around the theme of pyrotechnics. This article is the lower level of college entrance examination composition, which basically reaches the average score of college entrance examination composition, but from the perspective of AI writing, there is a language expression that is obviously not in line with daily writing in the penultimate paragraph of the article, which is a more obvious deduction point.

Teacher rating: 42

Player 3:

Teacher's analysis: There is a problem with the understanding of "pyrotechnics" in this article. The question was wrong. Mixing pyrotechnic gas with fireworks is judged to be off-topic. This article is judged to be a failing grade because it obviously has an incorrect understanding of the key words in the topic, and it is obvious that AI still has deviations from people in the analysis of similar words or words with implicit meanings in some cases.

Teacher rating: 28

Player 4:

Teacher's analysis: The beginning of the article basically meets the requirements of the topic and conforms to the understanding of pyrotechnics. However, in the following text, there is a situation where the topic sentence of the first sentence of the paragraph is completely detached from the argument that follows, and the main sentence is on topic, but the discussion part is off topic. This situation affects the overall score of the essay and is an off-topic essay.

Teacher rating: 38

Well, now you can guess who the corresponding AI is.

Player One: GPT-4; Number two: New Bing

Player 3: GPT-3.5 Player 4: Wen Xin

I don't know what you guessed?

The GPT-4 score was the highest, New Bing second, Wen Xin said again, and the lowest score was GPT-3.5.

Shichao also briefly chatted with this teacher about his views on AI composition.

First of all, if the candidate does not have a big off-topic and deviant phenomenon in the college entrance examination, but there is no particularly outstanding place, it will generally fluctuate around 42 points, and the writing is better, and 1-2 points can be added on this basis, that is, 42 points is the average of most human candidates.

So, if you want to get a high score, what ability do these AIs mainly lack?

The teacher told Shichao that these essays are mainly written around the topic itself, and if you want to become a relatively excellent college entrance examination essay (close to 50 or more points), you must extend the pyrotechnics, such as extending to the small moments around you in daily life, and say it in depth.

There is a commonality in these articles created by AI, that is, it has more in line with the requirements of the college entrance examination composition structure.

The front first gives a hat, the next is discussed in several paragraphs, and finally there is a sublimated ending, which is also a more common way to write, in these articles, this format is basically in line.

But when reviewing essays, structure is only one aspect.

AI also has a more obvious problem in these creations: conceptual confusion.

That is, mixing pyrotechnic gas with fireworks.

Especially the latter two, and among normal candidates, this relatively low-level conceptual misunderstanding is relatively rare.

The mistake that a normal candidate will make is: set up, that is, replace the existing concept with another concept he has prepared.

In addition to this, there are some bits and pieces.

For example, some compositions have something that does not conform to normal speaking habits and writing habits in language, and there are no punctuation marks.

Also, the number of words is not enough and needs to be corrected artificially.

In the questions entered by Shichao, the requirement of "not less than 800 words" is clearly included, but among them, New Bing and Wen Xin Yiyan only generated 400-500 words of composition when they first generated the essay.

The following one is Wen Xin's words, click to see the big picture ↓

If you want to use the college entrance examination score as a dimension, the teacher told Shichao that according to the standards of college entrance examination composition, the number of words is obviously not enough, which is basically about 15 points, not more than 20 points.

After reading these compositions generated by the AI, Shichao feels that his job is temporarily saved, after all, the current AI more often only understands the ideogram, but the connotation and metaphor behind the language is the most difficult part of writing and expression.

Of course, we do not rule out that if people induce AI well and give AI some good materials and angles, it may give a really good composition.

In any case, AI in creative writing, humans still have an advantage for the time being...