Two weeks ago, Facing the Wall released the leading open source model "Eurux-8x22B". Compared with the word-of-mouth work Llama3-70B, it was released earlier and has comparable comprehensive performance, especially with stronger inference performance - refreshing the inference performance of open source large model SOTA, which can be called the "science champion" in open source large models. With only 39B activation parameters and support for 64k contexts, it's faster and can process longer texts than Llama3.
Caption: The face-to-wall Eurux-8x22B model refreshes the SOTA for the inference performance of open-source large models in two challenging benchmarks, LeetCode and TheoremQA.
图注：面壁Eurux-8x22B 模型综合性能比肩 LlaMa3-70B，超越开源模型 WizardLM-2-8x22b, Mistral-8x22b-Instruct,DeepSeek-67b，以及闭源模型 GPT-3.5-turbo。
The Eurux-8x22B is aligned from the Mistral-8x22B. Powerful from the new UltraInterat large-scale, high-quality alignment dataset on the Face Wall Ultra Alignment technology. Previously, the Face Wall Ultra Alignment series dataset has "robust" more than 200 large models around the world, which can be called a distraction on large models.
Eurux-8x22B model + alignment dataset, open source for the whole family bucket:
https://github.com/OpenBMB/Eurus
https://huggingface.co/openbmb/Eurux-8x22b-nca
Open Source Model "Science Champion"
Open Source Model "Science Champion"
Complex inference capability is one of the core capabilities that reflect the performance differences of large models, and it is also the key capability required for the real application of large models.
Eurux-8x22B surpasses Llama3-70B in terms of comprehensive performance of complex reasoning such as code and mathematics, and refreshes the open-source large model SOTA, which can be called the "top science". In particular, LeetCode (180 LeetCode questions) and TheoremQA (a college-level STEM question) are two challenging benchmarks that surpass existing open source models.
Caption: Eurux-8x22B surpasses Llama3-70B in terms of comprehensive performance of complex inference such as code and mathematics, and refreshes the open-source large model SOTA.
How does the open-source model "Science Champion" Eurux-8x22B perform in practical applications?
In the recent LeetCode Weekly Competition, a real arena that tests the programming ability of human programmers: Eurux-8x22B has achieved excellent results in Python programming with a comprehensive ranking of more than 80% of human contestants, successfully solved three of the four algorithm questions, and can preliminarily pass the programming interview of programmers in major Internet companies.
Here's Eurux-8x22B's real solution to a moderately difficult algorithm problem in this weekly competition:
In addition to its excellent coding skills, the Eurux-8x22B is also a breeze to solve math problems.
For example, if you give it a high school permutation and combination problem, Eurux-8x22B first gives a clear idea of how to solve the problem, then disassembles and executes it step by step, and then summarizes the results to get the correct answer.
Examining it again, Eurux-8x22B hits the nail on the head, using the binomial theorem to give the correct solution clearly and concisely.
Then give it a vector algebra problem, and Eurux-8x22B can easily do it.
The college entrance examination function questions may be a headache for many people when they recall them, and Eurux-8x22B can also be answered correctly.
(It should be noted that Eurux-8x22B does not have additional fine-tuning and alignment for Chinese corpus!)
Face Wall Ultra alignment technology, distraction on large models!
The excellent performance of the large-scale model Eurux-8x22B is due to the new large-scale, high-quality alignment dataset UltraInteract from the face-to-wall Ultra alignment technology.
Good data leads to good models. Previously, the Face Wall Ultra alignment technology has "strengthened" more than 200 large models around the world, which can be called a distraction on large models.
➤ UltraInterat Aligned Dataset Address:
? https://github.com/OpenBMB/Eurus
UltraInteract is a large-scale, high-quality aligned dataset specifically designed to improve the inference capabilities of large models, containing 86K instructions and 220K preference pairs from 12 open-source datasets covering mathematical, code, and logical reasoning problems, with a total of about 500,000 (pieces) of data. In contrast, the LLaMA 3-70B model uses tens of millions of aligned data, which proves the quality of the UltraInteract dataset – data quality trumps data quantity. The UltraInteract dataset has been widely praised by the community since it was open-sourced.
From the leading end-side model "small steel cannon" MiniCPM to the new SOTA of open-source model inference Eurux-8x22B, why can facewall always launch "efficient large models" with the same parameters and better performance? The answer is that large models are a systematic project, and facewall, as one of the very few teams in China with both large model algorithms and infra capabilities, has a self-developed full-process efficient production line: facewall Ultra alignment technology, Infra process, and exclusive "model sandbox" Experimental and modern data factories, from data, training to tuning processes, are interlocked, and an excellent large model Scaling Law growth curve is born.
In terms of the Infra process, ModelForce has built a full-process optimization acceleration tool suite platform, ModelForce, which can achieve 10x inference acceleration and 90% cost reduction.
In terms of algorithms, through more than 1,000 "model sandbox" experiments, more scientific model training methods are explored. Look for efficient model training configurations to achieve rapid formation of model capabilities.
➤ Eurux- 8x22B3GitHuber:
https://github.com/OpenBMB/Eurus
➤ Eurux-8x2B50HuggingFace:https://huggingface.co/openbmb/Eurux-8x22b-nc
➤ UltraInterat Aligned Dataset Address:
https://github.com/OpenBMB/Eurus