Rich color comes from the temple of Wofei
Qubits | Official account QbitAI
Finally, Tang Jie's team at Tsinghua University also made a move.
On the same day that GPT4 was released, Professor Tang announced on Weibo:
ChatGLM, a conversational robot based on a large model of 100 billion parameters, is now open for internal testing by invitation.
According to reports, ChatGLM is specially optimized for Chinese, which can be said to be friendly to Chinese people~
It seems that writing blog outlines, creating scripts, solving math problems, writing code, and even playing Gao Leng Royal Sister, it is all dead:
Write a blog outline about ChatGLM
△Cosplay chat plays Gao Leng Royal Sister
Qubits are fortunate enough to get the internal test quota, and a wave of actual measurements will be carried out later.
At the same time, another news was announced:
In order to better promote the development of large model technology with the community, Tang Jie's team also open-sourced the bilingual ChatGLM-6B model with 6.2 billion parameters.
Its biggest feature is that it can be deployed on ordinary computers, and a 2080Ti can be used.
Let's take a look.
Can write literary speeches, know He Kaiming's latest developments, but...
Let's take a look at ChatGLM, which is a new member of the large model family of Tang Jie's team born after half a year.
The alpha beta version is called QAGLM (full name qa-glm-v0.7).
Give it a try.
First, let's let it talk a little bit about the difference between it and ChatGPT.
It points out its own R&D organization and the characteristics of its Chinese.
So, let it write a thank you message in classical Chinese.
He said that although there is an additional "Yu" and an inexplicable traditional character appears, it is okay to read, and the ranking sentences used have enhanced the momentum.
Next, we threw the beginning of the Silicon Valley Thunderstorm article the other day to it and gave it a title.
It felt good, at least caught a few key pieces of information.
Unfortunately, the thesis challenge did not pass, and when we threw the link to GLM-130B to it briefly summarize the topic, it said not this one at all.
The operation with ChatGPT Huzou reference is simply a fight (manual dog head).
Next, test its mathematical ability.
This elementary school application question is no problem:
However, the chicken and rabbit were in the same cage, and it was stumped down, and in the end it actually calculated a negative number ==
In terms of programming, simple algorithm problems can also be solved.
What about inductive information capabilities? We gave a paragraph of English requirements, not difficult:
The result is correct:
It should be noted that currently ChatGLM can only make up to 5 round-and-forth rounds of conversations, with a maximum of 1,000 words entered each time.
It has a good grasp of new information, knows that Twitter's current CEO is Musk, and knows about Ho's return to academia on March 10, but has not yet discovered that GPT-4 has been released.
And, the current response speed is still very fast, no matter what the question, the answer is correct or not, basically within a few seconds to give the answer.
Finally, the qubit also makes it cosplay to see how good it is to coax a girlfriend:
Emmm, although it is a bit straightforward, but after listening to this paragraph, the "I" anger really disappeared.
So, the above are our test results, what do you think?
Based on a pedestal model with 130 billion parameters
According to the official introduction, ChatGLM refers to the design ideas of ChatGPT, injects code pre-training into the 100 billion pedestal model GLM-130B, and realizes the alignment of human intentions through supervised fine-tuning and other technologies (that is, to make the machine's answer conform to human values and human expectations).
The origin of this GLM-130B is worth saying.
It is a large-scale Chinese and English pre-trained language model jointly developed by the Knowledge Engineering Laboratory (KEG) of Tsinghua University and Zhipu AI, with parameters of 130 billion, and was officially released to the public in August last year.
Unlike the architectures of BERT, GPT-3 and T5, GLM-130B is an autoregressive pre-training model with multiple objective functions.
Its advantages include:
In the 30 major models in the world reported by Stanford, the GLM-130B was also the only model selected in Asia.
And got good results:
For example, in terms of accuracy and malice, it is close to or equal to GPT-3 175B (davinci), and the robustness and calibration error are also remarkable in all hundreds of billions of pedestal models (as a fair comparison, only compared to fine-tuning models without command prompts).
At a recent CCF conference, an audience asked: Why was ChatGPT not born in China? Are we not paying attention to this?
The guest moved the GLM-130B out (it was also selected for ICLR'23).
Now the GLM-130B has finally come in with "great use" as well.
Regarding the internal test, Tang Jie's team said that the scope will be gradually expanded in the future, and interested friends can wait a little longer.
A scaled-down version of 6 billion parameters is also open sourced
In addition to this chatbot ChatGLM, Tang Jie's team also open-sourced the "scaled-down" version of GLM-130B ChatGLM-6B this time.
GitHub has captured nearly 2k stars
Using the same technology as ChatGLM, the ChatGLM-6B is beginning to take shape Chinese Q&A and conversational features.
The features are as follows:
Of course, the disadvantage is that the capacity is only 6 billion, its model memory and language ability are weak, it is not good at logical problems (such as mathematics, programming), and multi-round dialogue may lose context and misunderstanding.
But its main focus is a low threshold, which can be used for inference on a single 2080Ti, and the hardware requirements are not high.
Therefore, if you are interested, you can download it and try it, research and (non-commercial) application development.
Portal:
https://chatglm.cn/
https://github.com/THUDM/ChatGLM-6B
Reference Links:
[1]https://weibo.com/2126427211/MxlsQ6w4A#repost
[2]https://chatglm.cn/blog?continueFlag=d70d7590143c950d12ac7283214d879d
— End —
Qubits QbitAI · Headline number signed
Follow us and be the first to know the latest scientific and technological trends