On December 1, Alibaba Cloud held a press conference to release version 2.1 of the closed-source model of Tongyi Qianwen, upgrade the visual understanding model Qwen-VL, and open-source the 72 billion parameter model Qwen-72B. Compared with the previous version, the code comprehension and generation ability, mathematical reasoning ability, Chinese and English encyclopedia knowledge, and hallucination-induced resistance ability of Tongyi Qianwen 2.1 are increased by 30%, 10%, nearly 5%, and 14%, respectively. Users can experience the latest models for free in the Tongyi Qianwen APP.
Tongyi Qianwen also open-sourced the 1.8 billion parameter model Qwen-1.8B, and open-sourced the audio understanding model Qwen-Audio for the first time. So far, Tongyi Qianwen has open-sourced four large language models with 1.8 billion, 7 billion, 14 billion, and 72 billion parameters, as well as two multi-modal large models for visual understanding and audio understanding, realizing "full-size, full-modal" open source. The intensity is unparalleled in the industry.
The Tongyi Qianwen pedestal model continues to evolve, and the multimodal exploration is industry-leading
Since its launch in April this year, the Tongyi Qianwen pedestal model has been evolving. At the end of October, Alibaba Cloud released Tongyi Qianwen 2.0 at the Apsara Conference, and in just one month, Tongyi Qianwen was upgraded again, with code comprehension and generation capabilities increased by 30%, mathematical reasoning capabilities increased by 10%, Chinese and English encyclopedia knowledge increased by nearly 5%, and hallucination-induced resistance capabilities increased by 14%. At the same time, the context window length has been extended to 32k to better handle long text inputs.
In addition to LLMs, the Tongyi Qianwen team also has industry-leading explorations in the field of multimodality. In August, Tongyi Qianwen open-sourced Qwen-VL, a large visual understanding model, which quickly became one of the best practices in the international open source community. The conference also announced a major update to Qwen-VL, which greatly improves the basic capabilities of general OCR, visual reasoning, and Chinese text understanding, and can also process images of various resolutions and specifications, and even "look at pictures to do problems". Whether in terms of authoritative assessment results or the effect of real experience, Qwen-VL's Chinese text comprehension ability surpasses GPT-4V.
Qwen-VL can be "Picture Programming"
The Tongyi model can "see" and "hear". On the same day, Alibaba Cloud open-sourced Qwen-Audio, a large audio understanding model, for the first time. Qwen-Audio is able to perceive and understand all kinds of speech signals such as human voices, nature sounds, animal voices, music sounds, etc. Users can input a piece of audio and ask the model to give an understanding of the audio, and even use the audio to create literature, logical reasoning, story continuation, and so on. Audio comprehension enables large models to be close to human hearing.
The industry's strongest open source model, filling the gap in China's LLM open source field
Qwen-72B has achieved the best results in 10 authoritative benchmarks, becoming the industry's most powerful open source model, surpassing the open source benchmark Llama 2-70B and most commercial closed-source models. In the future, enterprise-level and research-level high-performance applications will also have the option of open source large models.
Qwen-72B is trained based on high-quality data of 3T tokens, which continues the strong performance of the Tongyi Qianwen pre-trained model. In English tasks, Qwen-72B achieved the highest score in the MMLU benchmark test of open source models; in Chinese tasks, Qwen-72B dominated the C-Eval, CMMLU, GaokaoBench and other benchmarks, and scored better than GPT-4; in terms of mathematical reasoning, Qwen-72B was ahead of other open source models in GSM8K and MATH evaluation interrupt layers; in terms of code understanding, Qwen-72B was in HumanEval, The performance in MBPP and other evaluations has been greatly improved, and the code ability has made a qualitative leap.
In the 10 authoritative evaluations, the 72 billion parameter model of Tongyi Qianwen won the best score of the open source model
Tongyi Qianwen's 72 billion open-source model has surpassed the closed-source GPT-3.5 and GPT-4 in part
The Qwen-72B can handle up to 32k long text inputs, surpassing ChatGPT-3.5-16k on the long text comprehension test set LEval. The R&D team optimized Qwen-72B's skills in instruction compliance and tool usage to make it better integrated with downstream applications. For example, the Qwen-72B is equipped with a powerful System Prompt capability, which allows users to customize the AI assistant with a single prompt word, requiring a large model to play a certain role or perform a specific response task.
Previously, there was no high-quality open-source model in China's large-scale model market that could benchmark Llama 2-70B. Qwen-72B fills the gap in China, with the advantages of high performance, high controllability and high cost performance, providing an option that is no less than that of commercial closed-source large models. Based on Qwen-72B, large and medium-sized enterprises can develop commercial applications, and universities and research institutes can carry out scientific research such as AI for Science.
From 1.8B to 72B, Tongyi Qianwen took the lead in realizing full-scale open source
If the Qwen-72B "touches the height" and raises the size and performance ceiling of the open source model, the Qwen-1.8B, another open source model at the press conference, "bottoms down" and becomes the smallest Chinese open source model, inferring 2K length text content only needs 3G video memory, and can be deployed on consumer-grade terminals.
From 1.8 billion, 7 billion, 14 billion to 72 billion parameters, Tongyi Qianwen has become the industry's first "full-scale open source" large model. Users can directly experience the effects of the Qwen series models in the Moda community, call the model API through the Alibaba Cloud Lingji platform, or customize large model applications based on the Alibaba Cloud Bailian platform. Alibaba Cloud's AI platform PAI provides in-depth adaptation to the full range of Tongyi Qianwen models, and provides services such as lightweight fine-tuning, full-parameter fine-tuning, distributed training, offline inference verification, and online service deployment.
Alibaba Cloud is the first technology company in China to open-source self-developed large models, and has open-sourced Qwen-7B, Qwen-14B, and Qwen-VL, a visual understanding model, since August. Several models have been listed on HuggingFace and Github, and have been favored by small and medium-sized enterprises and individual developers, with a cumulative number of downloads of more than 1.5 million and more than 150 new models and applications. At the press conference, a number of developer partners shared their practices of using Qwen to develop proprietary models and specific applications.
Zhou Jingren, CTO of Alibaba Cloud, said that the open source ecosystem is crucial to promoting the technological progress and application of China's large models, and Tongyi Qianwen will continue to invest in open source, hoping to become "the most open large model in the AI era" and work with partners to promote the ecological construction of large models.