Text: Wang Yining
Edited by Tang Yongyi
In early December, French AI startup Mistral AI officially closed its highly anticipated Series A funding round. According to Bloomberg, the company has raised 385 million euros (about $415 million), and the company is valued at about $2 billion. Mistral AI also opened up its commercial platform today.
On December 8th, Mistral AI's official Twitter released the third tweet since the opening of their account, and there was no lively and grand press conference, no eye-catching promotional video, just a magnetic link that was so ordinary that it could not be more ordinary, which broke the calm of the AI developer circle.
△ Source: Twitter
This download link leads to the open-source MoE model that has recently shaken up the AI circle, Mixtral8x7B.
As the name suggests, Mixtral 8x7B reveals its architecture: a combination of eight small models with 7 billion parameters, also known as the Mixture of Experts (MoE) architecture, which refers to the division of complex tasks into a series of smaller and easier to handle subtasks, each of which is handled by an "expert" in a specific domain. The MoE architecture makes the overall model more functional and powerful. This is also the architecture adopted by GPT-4.
Judging from the official data, Mixtral8x7B performed very well in the benchmark test, surpassing GPT-3.5 in terms of running scores, and surpassing Llama2 70B in many running scores, and the reasoning speed is still 6 times that of the latter.
△图源:Mistral AI
When generating models of the same quality, the cost of the inference process of Mixtral8x7B is also significantly lower than that of Mistral AI, and the energy efficiency advantage is significant.
△图源:Mistral AI
In addition, Mixtral8x7B also supports five languages: English, French, Italian, German and Spanish, with natural coding capabilities.
Open source, excellent running score, efficient ...... The combination of this series of advantages gives Mixtral8x7B the momentum to catch up with Chat-GPT, which is really enough to make people feel excited.
A developer has fine-tuned the Mistral MoE and released the dolphin-2.5-mixtral-8x7 model. It's a completely uncensored, open-source model, which means it's not influenced by the ethics of the developer, and it doesn't reply "As an AI assistant, I can't ......"
△ Source: Twitter
And with Mixtral8x7B into the public eye, there is also its developer, Mistral AI, a large model company from France, who went from being unknown to becoming famous in one fell swoop, and they only took half a year.
The "European version of OpenAI" has only been established for half a year with a valuation of 2 billion US dollars
Bloomberg reported that the company had raised 385 million euros (about $415 million), and the company was valued at about $2 billion.
Just six months ago, Mistral AI closed a €105 million ($113 million) angel round – the largest seed round in European history. In just 7 pages of PowerPoint, Mistral AI has attracted a number of established European venture capital institutions, including Redpoint and Index Ventures. In fact, this AI unicorn company, which is favored by star investment institutions in Europe and the United States, was officially established in Paris, France in May this year.
Although it has not been established for a long time, the three-member founding team of Mistral AI is quite weighty, and has rich experience in the direction of multimodality and RAG. The company's CEO, Arthur Mensch, is a former DeepMind research scientist who led the publication of important papers such as Chinchilla, Retro, and Flamingo, which are Google's most important work in the field of LLM, RAG, and multimodality in the past 20-22 years.
Chief Scientist Guillaume Lample, also a former Meta Research Scientist, and CTO Timothee Lacroix, both from the Llama core team. Currently, MistralAI's team has only 22 employees, maintaining a small but beautiful style.
△图源:Mistral AI
Don't go big and do it, attack the small model
In today's era of large language models, Mistral AI has done the opposite, focusing on the development of "small models" from the beginning of its establishment. In an exclusive interview with Silicon Valley investors Sarah Guo and Elad Gil, co-founder and CEO Arthur mentioned that making models smaller will definitely help the development and application of agents, and that small models can effectively reduce the cost of use and run on more devices, so that more interesting applications have the opportunity to be built.
Mistral AI's recently released chat model is a practical example of this philosophy.
Recently, Mistral AI has just released the open platform La plateforme, and provides three chat models based on instructions to generate text, mistral-tiny, mistral-small and mistral-medium, as well as an embedding model.
Among them, the most popular in the market is the "medium cup" model Mistral-medium, which is still in the testing stage. As the strongest open-source model launched by Mistral AI, Mistral-medium has a score of 8.6 on MT-Bench, which is better than GPT-3.5 in mainstream evaluations.
△图源:Mistral AI
Another special feature of Mistral AI is its firm choice of the open-source route.
In fact, before 2020, most of the research results of large models were shared and transparent, and it was not until some companies began to accelerate commercialization that closed-source models became more widely used. OpenAI, as we know it, was first founded as an open-source non-profit organization, and then turned into a closed-source company, which is also a point that former investor Musk is quite dissatisfied with.
Nowadays, in addition to Meta's LLaMA series, most of the leading large model manufacturers, such as OpenAI, Google, and Microsoft, have chosen closed source, but the open source model is still welcomed for its ability to iterate quickly and be customizable. This is also the reason why Mistral is called the "European version of OpenAI", and at a time when large companies are closing the source, there are also people who call Mistral AI the hope of the open source route.
Interestingly, a recent trend chart produced by the ARK Invest team predicts the development of generative AI by the open source community and proprietary models in 2024. According to its predictions, the performance of open source models is improving, and the gap between them and proprietary models will be getting smaller and smaller. In this regard, Yann LeCun, chief AI scientist at Meta and winner of the Turing Award, also forwarded:
Open-source AI models are on the path to surpassing proprietary models.
△ Source: Twitter
At present, Mixtral8×7B has been launched on many open source model platforms, and it still needs time to verify whether open source can catch up with closed source.
Welcome to the exchange