The rich color Xiao Xiao is from the Cave Fei Temple
Qubits | Official account QbitAI
Fudan large model MOSS, officially open source!
As the first ChatGPT-like product in China for open testing, as soon as the MOSS open source address was released, it rushed to the hot search of Zhihu:
According to the official website, MOSS is an open source dialogue language model with 16 billion parameters.
It is pre-trained by about 700 billion Chinese and English and code words, and if the accuracy requirements are not so high, it can even run on a single 3090 graphics card.
MOSS supports bilingual Chinese and English, has the dual ability of multi-round dialogue and using multiple plug-ins, and has "skill points" for search engines, literary diagrams, calculators, and equation solving.
At present, MOSS has 2.3k stars on GitHub, and the popularity continues to rise.
So, what projects are open sourced by MOSS and what are the current functions? Let's take a look.
Can solve equations, can also generate pictures
It is understood that this time MOSS open source is the third round of iteration version, called MOSS 003.
Of course, MOSS 003 is not open source all models, the main open is the pedestal pre-training model, and the final model will be open source in the near future:
Previously, MOSS has gone through two iterations, one is the open invitation for internal beta version of MOSS 002, and the other is the internal test version OpenChat 001.
The "first ChatGPT open test in China" witnessed by everyone in March is the MOSS 002 version.
How effective is this latest release?
The MOSS team released examples of its multiple rounds of dialogue, solving equations, writing code, generating pictures, and answering ethics.
This is the display effect of multiple rounds of dialogue, with the ability to trace the source of the data:
Here are examples of doing math problems, including the chicken and rabbit cage problem:
Also includes the ability to generate images:
There are also recent heated discussions among netizens about the ethics of AI:
Of course, there are also netizens who "smash the field" in this wave.
For example, in the official GitHub Q&A, some netizens are eager to know who is better than its Chinese ability and chatglm6b, and the following netizens ridiculed:
Are you here to smash the field?
This is also because the team previously said during the public test of MOSS 002 that the Chinese ability of MOSS is not too good, which is also related to the training data.
In addition, the team also open-sourced some training data this time.
As for the complete data, the Fudan MOSS team said that "all will be open source in the near future".
The first ChatGPT product released in China
MOSS is taken from the self-aware AI in The Wandering Earth.
It was developed by the team of Qiu Xipeng of Fudan Natural Language Processing Laboratory and officially released on February 20.
At that time, a number of large domestic manufacturers were still making high-profile official announcements to build ChatGPT, and no one expected it to be killed silently.
So even though MOSS has an order of magnitude smaller parameters than ChatGPT, everyone flocked to the server and overwhelmed the server.
A week after the release, Qiu Xipeng, a professor at Fudan University, proposed in his speech that if the optimization goes well, it is planned to open source MOSS at the end of March.
Because in his opinion:
The reason why people are easy to have the "stuck neck" problem is because of the lack of a good industry ecology.
In the case of ChatGPT, OpenAI only needs to focus on making models, and the computing power, data, and systems are handed over to other companies. On the other hand, many companies in China need to do all the links themselves when making large-scale language models, and the data and computing power of each company are also limited, and the result is that each company cannot do much. In the process of development, the high cost of large-scale language model training and online inference is also a major problem.
Therefore, they considered open-sourcing MOSS rather than commercializing it from the start.
With such a base, everyone does not need to repeatedly develop the underlying technology, and can connect different subdivisions on it.
Now, almost a month late, MOSS has done what it says.
Have you tried the Fudan MOSS open source model? How effective is it?