Tongyi Qianwen has open-sourced 32 billion parameter models, and has realized 7 large language models that are all open-source

On April 7, it was reported that Alibaba Cloud Tongyi Qianwen open-sourced the 32 billion parameter model Qwen1.5-32B, which can maximize the balance of performance, efficiency and memory usage, and provide enterprises and developers with more cost-effective model choices. At present, Tongyi Qianwen has open-sourced a total of 7 large language models, and the cumulative number of downloads in open source communities at home and abroad has exceeded 3 million.

Tongyi Qianwen has previously open-sourced 6 large language models with 500 million, 1.8 billion, 4 billion, 7 billion, 14 billion and 72 billion parameters and have all been upgraded to version 1.5, among them, several small-size models can be easily deployed on the device side, and the 72 billion parameter model has industry-leading performance, and has been on the list of HuggingFace and other models for many times. The open-sourced 32 billion parameter model will achieve a better balance between performance, efficiency, and memory footprint, for example, 32B is more capable in agent scenarios than the 14B model, and 32B has lower inference costs than 72B. The Tongyi Qianwen team hopes that the 32B open source model can provide a better solution for downstream applications.

In terms of basic capabilities, the Tongyi Qianwen 32 billion parameter model has performed well in multiple evaluations such as MMLU, GSM8K, HumanEval, and BBH, and its performance is close to that of the Tongyi Qianwen 72 billion parameter model, far exceeding other 30 billion parameter models.

In terms of the Chat model, the Qwen1.5-32B-Chat model scored more than 8 points in the MT-Bench evaluation, which is relatively small compared to the Qwen1.5-72B-Chat model.

In terms of multilingual ability, the team selected 12 languages, including Arabic, Spanish, French, Japanese, Korean, etc., and assessed them in various areas such as exams, comprehension, mathematics and translation. The multilingual capability of Qwen1.5-32B is only slightly inferior to the 72 billion parameter model of Tongyi Qianwen.