Why is long text important for large models?
Text: Sun Xin
Edited by Yao Yun
Header image source: Visual China
What does 2 million words mean? "The Biography of Elon Musk" is 300,000 words, "Dream of Red Mansions" is 700,000 words, and "The Legend of Zhen Huan in the Harem" is 1 million words. If you read 30,000 words in 1 hour according to the reading speed of 10,000 words in 20 minutes, then 2 million words will take about 66.67 hours to read.
And such a marathon reading is likely to be swallowed.
On March 18, Moonshot AI, a domestic AI start-up company, announced that it had made a new breakthrough in the technology of large model long context window, and Kimi intelligent assistant has supported 2 million words of ultra-long lossless context, and will start the "internal test" of the product from now on. Then, major companies followed suit and rolled up the length of the text. On the 22nd, Ali Tongyi Qianwen opened the long document processing function of 10 million words to everyone for free; On the 23rd, 360 Intelligent Brain announced that it can process 5 million words in the internal test.
That is to say, now, in about 10 minutes, Kimi, Tongyi Qianwen, 360 Zhibrain and other products can read the three books "Elon Musk's Biography", "Dream of Red Mansions" and "The Legend of Zhen Huan in the Harem", and can understand the content in combination with the context.
In fact, long texts have long been a battleground for AGI (Artificial General Intelligence): in November 2023, OpenAI released GPT4 Turbo, which supports input with a maximum context length of 128k; 14 days later, Antrophric urgently released Claude 2.1, announcing a breakthrough in contextual capabilities, from 100k to 200k; In February 2024, Gemini 1.5 announced that the long text window supports 100w length, which is 750,000 words.
So what do these numbers and standards really mean? What exactly is the ability to process long text contexts? We selected products such as Kimi, Tongyi Qianwen, and Claude for testing experience.
Antrophric's Claude, who has been recognized for his eye-catching ability to process long texts; Alibaba's Tongyi Qianwen can currently process 10 million characters, and the word order belongs to the first echelon; Kimi's long text context processing ability made the dark side of the moon out of the circle for the first time in domestic AI entrepreneurship.
Step 1: "Feed" Kimi, Tongyi Qianwen, and Claude to Musk
"If I have the opportunity, of course I am willing to communicate with Lei Jun." Kimi said to me as Elon Musk.
Source: Kimi screenshot
In order to better evaluate the ability to process long text contexts, we input about 370,000 words of "Elon Musk's Biography" to Kimi, Tongyi Qianwen, and Claude at the same time, and asked them to use Musk's thinking and tone to talk to us.
(1) Up to 500 files can be uploaded at the same time, and multiple formats are supported
When we enter a document, the parsing conditions and the time taken are similar.
Tongyi Qianwen parses documents, which can upload 100 files at the same time, each of which is no more than 150MB, and supports PDF, Word, Excel, Markdown, EPUB, Mobi, txt and other forms.
Kimi can upload up to 500 documents at the same time, each with a maximum of 100MB, and supports doc, xisx, PPT, txt, images and other file formats. It is worth noting that Kimi needs to apply for the internal test qualification of long text context ability, and it can be used for free after the application is approved.
Claude can upload up to 5 files at the same time, each file is no more than 10MB, and supports docs and images file formats. Therefore, when the document is large, we need to break it down into several docs files that are no more than 10MB in size and start parsing.
Source: Screenshot of Tongyi Qianwen
(2) Kimi, who is networked, understands Duanfei better
We entered about 1 million words of the novel "The Legend of Zhen Huan in the Harem" and asked them questions. "Who do you think is most likely to become a successful entrepreneur in "The Legend of Zhen Huan"?" A few seconds later, we each got this answer.
Kimi gave five answers: Zhen Huan, Shen Meizhuang, Concubine Hua, Concubine Duan, and Queen, and analyzed each answer. For example, Zhen Huan is because she is "able to survive in a complex harem environment and gradually improve her status", Shen Meizhuang is because she is "stable and thoughtful", Concubine Hua is because she "has a high status and power in the harem", and the queen is because she is "the master of the harem".
In Kimi's answer, the reason given to Duanfei is more interesting, and it is not directly written in the book. Kimi believes that Duanfei is because "although she is not as conspicuous as Zhen Huan and Concubine Hua in the novel, she is able to maintain a certain status and influence in the harem, which shows that her 'investment' in interpersonal relationships and personal development is also relatively successful."
Source: Kimi screenshot
However, Claude gave a character that does not exist in "The Legend of Zhen Huan in the Harem" - Concubine Chun, and set Concubine Chun with insight, judgment, open-mindedness, diligence and studiousness, good at handling interpersonal relationships, courage to innovate, tenacity and other character settings.
Source: Screenshot by Claude
On March 18, Kimi officially released a live broadcast, Xu Xinran, the head of AI Infra on the dark side of the moon, mentioned the 10,000-hour rule, that is, to become an expert in the field, we need to study for at least 10,000 hours. Now, in just 10 minutes, Kimi is close to the level of a junior expert in any new field.
In the live broadcast, Xu Xinran also entered about 1 million words of copies of "Heaven and Dragon Slayer" and 1 million words of the script of "The Legend of Zhen Huan", and in less than 10 minutes, Kimi became a "scholar" and "Zhen scholar".
We continue to test.
When we didn't provide Kimi with Elon Musk's biography, we typed "Please talk to me with Musk's thoughts and tone."
Kimi emphasized at the beginning and at the end that this answer is "imitating Musk's style" and does not represent the true views of himself and the companies involved. Faced with this problem, "Ksk", who incarnated as Musk, commented on Lei Jun and Xiaomi SU7 in this way. It believes that "Lei Jun is a very good entrepreneur" and "the Xiaomi SU7 car is a very interesting product". Finally, he also gave advice, "As a new company in the field of electric vehicles, Xiaomi still has a long way to go. They need to make continuous efforts in technological innovation, production efficiency, supply chain management, etc. ”
Kimi reads the previous version of Elon Musk's answer. Source: Kimi screenshot
We "fed" the book to Kimi, and when we parsed it, it gave a similar answer.
Kimi's version of the answer after reading "Elon Musk's Biography". Source: Kimi screenshot
We repeated the previous step and gave the same question to Tongyi Qianq. In the answer given by Tongyi Qianwen, the content about Xiaomi SU7 is: "Although the specific model and details may not exist in reality, it is based on the assumption that Xiaomi has entered the field of electric vehicles......"
Tongyi Qianwen asked the version of the answer after reading "Elon Musk's Biography". Source: Screenshot of Tongyi Qianwen
As concocted, we continued to test Claude, and the results were similar to those of Tongyi Qianwen.
It is understood that as a large-scale model product, Kimi has networking functions in addition to document interpretation. This means that Kimi can not only use the established data analysis, but also search for network-wide information based on the problem as the basis for data analysis. Currently, ChatGPT 3.5 and Claude 2.0 do not support this feature.
(3) The ability of the three documents is similar
In order to test the content comprehension of Kimi, Qianwen Tongyi, and Claude, we asked questions about the content of the book after entering the "Elon Musk Biography" into all three.
Source: Screenshot of Kimi's answer
Source: Screenshot of Tongyi Qianwen's answer
Source: Screenshot of Claude's answer
In response to the same question in the book, the answers of the three are similar and more accurate, and they all mention that Musk thinks this is a realistic version of the story of "Lord of the Flies", and he has lost a small amount of food and water, and he has lost 10 pounds.
(4) Kimi sees Musk's success traits, and Claude pays more attention to Musk's inner world
What you type in a chat box is called "context". The ability to understand between contexts is also crucial, and the fragmentation between paragraphs can lead to the loss of contextual information, which also has an impact on the accuracy of the answers given by the AI.
For example, if a user mentions a specific topic or question in a conversation, the contextual AI can remember that information and cite them in subsequent conversations to provide a coherent and relevant response. This ability is important for creating a natural, fluid conversational experience.
To test the contextual text processing capabilities of the three, we asked Musk about his school experience when he was 12 years old, and then asked "what is Musk's personality like?"
Claude gives the characteristics of fanatical persistence, ruthlessness, dramatization, tenacity, and dual personality, and analyzes them one by one. For example, fanatical attachment is due to "his prophetic enthusiasm and sense of mission for realistic grand goals, such as the development of electric vehicles and the landing on Mars." This fanaticism, which ignored his clumsiness, also prompted him to take risks and take extremely high risks."
Source: Screenshot of Claude's answer
Source: Screenshot of Tongyi Qianwen's answer
Source: Screenshot of Kimi's answer
Tongyi Qianwen also mentioned the grand vision, promoting the popularization of electric vehicles, and immigrating to Mars. The difference is that Qianwen Tongyi describes these "idealism and vision" as Musk's "belief that the problems facing humanity can be solved through technology".
Among these traits given by Claude, Tongyi Qianwen and Kimi, Musk's "negative" traits such as ruthlessness, dual personality, dramatization, and ambivalence are also mentioned. Claude summed up Musk as "fanatical, cold, dramatic, tenacious, but with a certain childlike innocence and vulnerability." Kimi summed it up as a "very unique individual" and believes that much of his success in the tech and business worlds owes a lot to these personalities.
Long text, the first step of AI "landing on the moon".
Why is long text important for large models?
"Why is a long text the first step to the 'moon landing'? It's essential. It is the new computer memory. Yang Zhilin, founder and CEO of the Dark Side of the Moon, once said in an interview with Tencent Technology that Long Context is the basic capability of large language models (LLMs).
Previously, Yang Zhilin used a figurative metaphor to describe long texts, and "supporting longer contexts" means that large models have more "memory".
In October 2023, Kimi was launched, and at that time it could support lossless contexts with a length of up to 200,000 Chinese characters. In 5 months, it will be upgraded to 2 million words, and the dark side of the moon will directly increase the ability of long texts to 10 times. According to the calculation standard in the field of AI, the length of 2 million Chinese characters is about 4 million tokens. At that time, Google's Gemini 1.5 and Claude 3, which were in the first echelon of long text, supported 1 million tokens, and Kimi's 2 million Chinese character context length surpassed the level of top overseas large models.
Similar to the "running score" when measuring the performance of mobile phones and computers, large models also have their own "running score" criteria, which are called tokens. It is the basic unit of input and output of a large model. According to OpenAI's relevant guidelines, 1k token is equal to 750 English words and 500 Chinese characters. The larger the token, the stronger the text processing power. It is understood that the current token of ChatGPT 4 is 32k, the token of Claude 3 is 100w, and the token of Kimi is 400w.
In other words, the longer the text that can be processed, the more material there is when the content can be extracted, and the more accurate it will be to help the user process the information.
Watchmaker: Sun Xin (Source: Introduction to the official websites of each AI product company)
According to statistics, the longest long document processing length comes from Alibaba Cloud's Tongyi Qianwen, and the strongest contextual long text processing ability is Kimi from the dark side of the moon, Google's Gemini and Kimi all support networking functions, but Ultra members need to pay, and the price is $19.99 per month.
It is worth noting that at present, the mainstream long text processing products in China, such as Tongyi Qianwen and Kimi, can be used for free internal testing, and with the increase in user usage, it means that the "training" data of large models is also increasing, and the AGI long text processing track is being opened.
Resources:
"The "Kimi Concept" Cools Down, Long Text "Can't Afford" the Next Step of the Big Model", Tencent Technology
"The explosive Kimi, whose business did you steal? , fixed focus