It is said that generative AI is very powerful in writing things, and I don't know how effective it is when it is used to write online articles. In this article, the author compares some problems of AI writing online articles and some style imitation effects for us through practical operations.
When many friends try to use AI to write online articles, they will encounter such a problem: what AI writes at the beginning may be good, but the language is plain and blunt when writing. When the length is extended, the AI begins to repeat, compare, a large number of inner monologues, pile up text randomly, and enter a state of convulsions.
So, is there any way to keep the style of the AI alive?
To solve this problem, we first need to analyze the possible causes of this problem, so as to prescribe the right medicine.
At present, there are many types of articles that AI can write, ranging from Xiaohongshu/Moments copywriting and short poems, to official account articles and papers. AI has a very good ability to process logical and knowledge-combing articles, such as generating email bodies, meeting minutes, papers, and the like. The language style of the AI itself is also very suitable for this kind of content.
However, when it comes to creating stories and outputting more emotional copywriting, AI will have all kinds of problems, and the situation mentioned at the beginning is a typical example. I think the reason is that there is logic in the story content, but logic is not the core driving force of the story, and most of the content in the story needs to be fictionalized and created out of nothing, which is actually a very different ability from the type of article generated above. Fiction ≠ fabrication, the comprehensive use of this kind of knowledge, the implicit decision-making mechanism is very complex, the human brain can easily do it, but it is difficult to summarize it into 123, which can only be understood but not spoken, far from being fully grasped by existing AI.
Therefore, we can consider the following aspects to solve:
- From the level of literary style, is it possible for AI to deeply learn the style of online texts, understand the characteristics of online texts in terms of words, sentence structures, dialogues, descriptions, paragraphs, etc., and use the constraints of style to avoid scribbling.
- At the level of questioning, it is still necessary to break down the whole task to make the tasks performed by the AI more focused, and not let the AI generate too much content at once, and see if this will be better.
In this way, we can have three sets of solutions, respectively for AI chat, agents, and AI writing software, and see how it works.
General idea
The main purpose of this test is to explore how ordinary online writers can use AI products to make the style of AI-generated online articles more stable and reduce the waste of time by AI gibberish.
Although it is aimed at three different types of AI products, the overall idea of this test is somewhat similar, mainly including the following aspects:
1. Let's first find the work of a prolific author, pick up some of the content and feed it to the AI to learn
I won't pick up the whole book here, but control it at about tens of thousands of words. Because our goal is not to train a large model to write online articles, but to hope that the AI can write in a stable style (or the author's own style), so that the amount of content that can be learned by AI is usually not too large.
2. Find the author's latest work, describe the corresponding story synopsis for the AI to write, and see what the AI can write
Generate the text in blocks, a small number of times, try to control it within 1000 words each time, and then assemble the generated content. Try to write two chapters in this way to see if the AI will have a convulsion and how different the content is from the original author's style.
3. Prepare the control group
Since the effect is to be measured, the control group is of course indispensable. Here, I divide the test objects in each set of solutions into 4 groups according to the number of words in the examples, which are 0k (no examples), 2k (examples of about 2k words), 10k (examples of about 10,000 words), and 50k (examples of about 50,000 words). An example of this is the existing work of the prolific authors mentioned above.
4. Prepare inspection standards
How do you measure AI-generated content style similarity? I found Lu Xun's "Kong Yiji" and "Diary of a Madman", and Zhu Ziqing's "Back", and uploaded them to Kimi for AI to analyze the similarity of their language styles.
Question: [Upload file] Please help me check the similarity of the language style of these two articles, and tell me what is the similarity of the language style of the two articles if the full score is 100 points?
Kimi gives the similarity of the language styles of "Kong Yiji" and "Diary of a Madman" as 70, and the similarity of the language styles of "Kong Yiji" and "Back" as 40. Therefore, we can use these two scores as a reference to evaluate the similarity between the original text and the AI-generated content.
Why did you choose this line of thinking? In fact, the key is that the current AI is too stupid. AI can't do it in a more intelligent way, and a more refined way is not acceptable to general online writers. Therefore, whether it is the general idea or the subsequent specific test plan, it is necessary to find a balance that is not too troublesome and can produce good results.
方案1:用AI chat仿写
Use AI chat to imitate writing, that is, upload online text examples, and then let AI write novels with reference to the style of the examples.
There are usually two ways to do this, one is to upload the file and give a synopsis of the story directly, and let the AI write it in the style of the example. The second way is to upload the file first, let the AI summarize the language style of the article, and then ask the AI to write according to that style.
My list of questions is as follows:
Direct copying
There is no sample version
Please play the role of a professional online writer and write the text of the novel according to the following synopsis:
(abbreviated)
Request:
1. Focus on dialogue and description to make the story more graphic
2. Expand the details in combination with the synopsis of the story to ensure that the details are rich and the plot is coherent
3. The output text content should not be less than 500 words, and there should be no insights and summary content at the end, waiting for me to give you a follow-up story synopsis
Sample version included
【Upload file】Please play the role of a professional online writer, the above is the online article you have written, please write the text of the novel in strict accordance with the language style of this article, the synopsis of the story is as follows:
3. The output text content should not be less than 500 words, and there should be no insights and summary content at the end, waiting for me to give you a summary of the follow-up story
Ask
Follow the synopsis below to continue writing the rest of the story:
(abbreviated)
The requirements are as follows:
1. The language style must be consistent with the above
2. Focus on dialogue and description to make the story more graphic
3. Expand the details in combination with the synopsis of the story to ensure that the details are rich and the plot is coherent
4. The output text content should not be less than 500 words, and there should be no feelings and summary content at the end, waiting for me to give you a follow-up story synopsis
…… (Repeat the question until you have completed 2 chapters)
Imitate after induction
Questions
【Upload File】Please analyze the language style of this article and introduce the characteristics of this language style in detail
Ask
Please play the role of a professional online writer and write the text of the novel in strict accordance with this language style, and the synopsis of the story is as follows:
After all the generated content fragments are generated, they are assembled according to their respective groups, and then the complete text of each version (2 chapters) is compared with the first 2 chapters of the author's original work, which is also scored by Kimi, and the language style similarity score is as follows:
Direct copying
Imitate after induction
*When the AI analyzes the language style and scores, it focuses on the theme and plot direction, although I later adjusted the question to ignore this part, but the AI didn't listen to it, and the score was still ridiculously high. Such a deviation has a large impact on the final score, resulting in a serious discrepancy between the results and expectations. (╯#-_-)╯~~~~~~~╧═╧ Why don't I believe this ghost score......
Forget it, forget it, AI is not very reliable, you still have to rely on yourself! Let me analyze the following sets of results for generating the body:
Direct copying
0k blank control group
The language style of the 0k version is relatively plain, starting from the second segment and ending with a typical AI-flavored passage. As the number of rounds increases, AI-flavored content takes up more and more space, including cumbersome inner monologues, repetitive sentences/paragraphs, etc. In terms of story expansion, there are cases where the AI of some fragments automatically continues to write the subsequent plot in order to make up the number of words, resulting in problems in the connection of the two fragments.
2k version
2k direct imitation version, the language style at the beginning, the appearance of the characters, and the 0k version are quite different. In terms of writing style, it is already clearly felt that it is moving closer to examples, and the proportion of content such as AI's inner monologue is greatly reduced, and only appears at the end of the last three plot clips. In addition, there is a bug, the second block has a large area of repetition, and among the 1600+ words generated, only the first 350+ words are available, and the back is all repeating the previous text.
Version 10k
10k direct imitation version, there was an unexpected situation, the plot was completely off track, and Shengsheng wrote a different story and character relationship according to the same story synopsis. In terms of style, this version is still a big improvement compared to the 0k version, and occasionally there are some AI-flavored inner monologues and looking forward to the future at the end of the clip, which does not account for a large proportion. Compared to the 2k direct copy, the improvement is not very obvious.
50 edition
The content written in the 50k direct copy version is very confusing, because it makes almost all the mistakes of the previous versions, and the AI flavor of the inner monologue, repetition, etc. takes up more space than the 2k and 10k direct copy versions, and a little less than the 0k version. Like the 10k version, the plot and character relationships are completely off-track, and there are a lot of meaningless character interactions mixed in (there is no plot, just these tricks are mentioned and disappeared to express concern for the protagonist). I don't know if I don't know, and a test is a big disappointment. Imitate after induction
2k version
The content generated by the imitation after 2K induction is similar to the 2K direct imitation version, both in terms of style and character personality. In this version, an important supporting role has been expanded to interpret some of the plots in the synopsis, which can be regarded as an innovation. As for the AI-style inner monologue, repetition and the like, it will be a little more than the 2k direct imitation version, and the overall proportion is still relatively low.
Version 10k
The imitation version after 10k induction is simply not too outrageous~
In terms of style, I think this version is closer to the original than the 10k direct copy, but a new problem has arisen, that is, the content generated by the dialogue in the later rounds will be repeated with the previous rounds. It's not a repetition of the previous story at the end, but a repetition of the inner monologue at the end of the previous round, and repeated several times, a magical callback. Of course,The problem of plot deviation in the 10k direct imitation version also exists.,There is also a situation where the characters before and after the collapse of the situation.,The logic is very confusing.。
50k version
The 50k post-induction copy is outrageous than all previous versions. How so? Maybe it's completely pickled by AI. The plot and characters are completely off, adding a lot of meaningless characters and plots, as well as an AI-flavored inner monologue that appears at the seams. I feel like it's not even as good as the 0k version, spicy eyes.
The above are the test results of AI chat. That's why I feel that AI scoring is extremely unreliable, watching AI scoring and manual analysis are completely different things.
Judging from the test situation of AI chat, there are several conclusions:
- The use of imitation writing will indeed be better than AI directly following the synopsis of the story, and the style of the generated text content will be more stable, reducing the taste of AI.
- Larger examples are not always better, and 2k, 10k, and 50k examples provide less effective reference, which is beyond my expectations.
- As the number of rounds increases, the quality of AI writing text is also declining rapidly, which is very obvious. It doesn't even take many rounds for this question to arise.
- Every time the AI generates content, it feels like it thinks it's the last time it serves you, and it has to have an ending imagining a better future, and there is a lot of drama that can't shut it up.
Overall, I don't think AI chat is a good choice for writing text, it's really too easy to convuls, and the style is extremely unstable. AI chat is suitable for doing some fragmented, shallow, and logical work, or generating some creative ideas, just write an article, it can't hold it at all~
This article was published by Everyone is a product manager by the author [I don't know], WeChat public account: [I don't know], original / authorized Published in Everyone is a product manager, without permission, it is forbidden to reprint.
Image from Unsplash, based on the CC0 license.