"Pioneers of the Metaverse" is a column we set up for the development of the Metaverse, mainly for those practitioners who dig deep into the Metaverse industry or "pan for gold" in the Metaverse, share the stories of these companies or entrepreneurs, and get a unique perspective on those enterprises or individuals who lead the development of the global Metaverse, we are convinced that the curtain of the Metaverse has been opened, and the technology Internet that will lead the next 20 years has embarked on the wave of the times. With a valuation of 2 billion and challenging OpenAI's supremacy, how does the rise of Mistral AI change the landscape of large language models?
"ChatGPT is as important as the invention of the internet and will change the world. "Bill Gates's prediction of large models seems to be becoming a reality step by step.
In the past year, OpenAI has led the way in the field of AI (artificial intelligence), and both the popularity of ChatGPT and its internal turmoil have become the focus of the industry.
However, with the rise of Mistral AI, the landscape is undergoing an unprecedented transformation.
As a strong competitor of OpenAI, Mistral AI has shown remarkable breakthroughs in both technology and products, and has become a shining star in the field of AI, known as the "European version of OpenAI".
Compared with OpenAI, Mistral AI pays more attention to the practical application of technology and is committed to applying the most advanced AI technology to solve practical problems.
In terms of financing, Mistral AI raised $113 million in seed funding at its inception and attracted a number of well-known investors such as Lightspeed Venture Partners, Salesforce, and BNP Paribas.
In just a few months, the company closed another $415 million Series A funding round at a whopping $2 billion valuation. This funding scale is extremely rare among AI startups, which not only proves the high recognition and expectation of Mistral AI in the capital market, but also provides strong financial support for its future development.
The rise of Mistral AI not only poses a challenge to OpenAI, but also injects new vitality into the entire AI field, bringing more innovation and breakthroughs to the entire industry.
01. The innovative force leading the AI revolution
Mistral AI, whose full name is Mistral Artificial Intelligence, is a company focused on AI R&D and application, especially the technology used to build online chatbots, search engines, and other AI-powered products.
Since its establishment, Mistral AI has always adhered to the people-oriented, they hope to improve the way people live and work by developing more intelligent and humanized AI systems, bringing more convenience and well-being to human beings, and are committed to using advanced AI technology to provide efficient and intelligent solutions for all walks of life.
Despite being just a start-up, Mistral AI's team of founders is all about a small group.
Among them, Arthur Mensch worked as a researcher at Google's artificial intelligence company DeepMind, while Timothée Lacroix and Guillaume Lample held positions related to the technology at Meta, respectively.
Their previous work experience has given them a deep understanding of technologies such as multimodality, RAG, and algorithm optimization, and has in-depth research in the fields of model inference, pre-training, and model embedding.
This quote from Mistral AI's official website speaks volumes about Mistral AI's ambitions: "Our mission is to push AI forward and serve our open community and our corporate customers. We are committed to driving the AI revolution by developing open weight models that are on par with proprietary solutions. ”
Although Mistral AI is currently only a small creative team, they have always adhered to high scientific standards and developed efficient, useful, and trustworthy AI models through breakthrough innovations. This may be one of the reasons why Mistral AI is so popular.
02. A major leap forward for large language models
Mistral AI's most high-profile product is undoubtedly the Mixtral 8x7B, one of the most competitive open large models on the market today, with a number of special features that significantly outperform other large models.
At the heart of the Mixtral 8x7B is its innovative Mixture of Experts (MoE) architecture, which distributes input data to specific neural network components called "experts" through a network of gateways. In Mixtral 8x7B, there are eight such specialists, each with a whopping 7 billion model parameters.
Although eight "experts" are deployed, only two "experts" are required for each data processing in the actual operation. This data resource allocation algorithm greatly optimizes the processing speed while maintaining the model performance.
For training and fine-tuning, Mixtral AI is pre-trained using multilingual data, including English, French, Italian, German, and Spanish. The Instruct model, trained using supervised fine-tuning and Direct Preference Optimization (DPO), achieved high scores in benchmarks such as MT-Bench.
During the in-depth study of Mixtral 8x7B, Mistral AI also paid great attention to fine-tuning some of its functions, especially for those versions that can follow instructions, so that the model is more refined and personalized.
In addition to its superior performance, another important reason why the Mixtral 8x7B is so well received is the openness it represents.
Mistral AI directly exposed the model's weighted data when it was released, a strategy that was highly effective in capturing the attention of the AI community while ensuring broad accessibility for academic and commercial use. The openness of Mixtral AI encourages the emergence of diverse applications, which have the potential to bring new breakthroughs in large models and language understanding.
Mixtral 8x7B's innovative approach and superior performance have made it the industry benchmark in the field of large models, and despite this achievement, Mixtral AI has never stopped moving forward and is actively optimizing the performance of this model.
03.Mistral AI的里程碑式发展
The birth of Mixtral 8x7B marks an important breakthrough in AI technology, especially in the innovation of model structure and efficiency, so how does it compare to many large models?
Can it surpass the giants?
Since the advent of ChatGPT, OpenAI has been regarded as the gold standard for large language models. However, Mistral AI has demonstrated superior performance in a wide range of benchmarks by introducing a fully open-source, open-weighted model, even outperforming OpenAI's GPT 3.5 model and Meta's LLama 2 13B model in some cases.
Specifically, in the multi-domain large-scale Multitask Language Understanding (MMLU) test, covering 57 subjects such as mathematics, U.S. history, computer science, law, etc., Mistral AI stood out with an accuracy rate of 60.1%, while Llama 2 7B and Llama 2 13B had an accuracy rate of just over 44% and 55%, respectively.
Similarly, in tests involving common-sense reasoning and reading comprehension, the Mistral 7B outperformed the two Llama models with 69% and 64% accuracy, respectively, highlighting its strengths in the field of deep language understanding.
The reason why Mistral 7B performs well in deep language understanding is that it is exposed to a large amount of complex and changeable text data during the training process, which enhances its contextual perception and reasoning ability, so that Mistral 7B can better understand and grasp the internal logic and semantic information of the text during the test, so as to give more accurate and in-depth answers.
In contrast to GPT3, Mistral AI focuses on fast inference and processing longer sequences. Grouped query and sliding window attention mechanism, an attention mode based on the attention model, is used to optimize for lower latency and higher throughput. This makes it the most cost-effective choice for applications that can achieve high-volume, fast processing at a lower cost.
In contrast, GPT3 is known for its deep language understanding and multitasking capabilities, and it is optimized for processing shorter sequences. For example, GPT3 excels in Q&A tasks, understanding and generating accurate answers, quickly summarizing long texts thanks to its strong language comprehension capabilities, and performing text completion, language translation, sentiment analysis, and more.
High-performance small model, but lacking "safety guardrails"
The Mistral 7B has attracted attention for its high performance and adaptability, and has the characteristics of a "small digital footprint", that is, the model requires less computing resources and storage space to run.
In contrast to other models that rely heavily on powerful hardware, the Mixtral 7B can even run on small PCs without a discrete GPU. This gives it the flexibility to deploy tools such as vLLM Inference Server and skypilot open source framework on any cloud platform, including AWS, GCP, and Azure. At the same time, the model can also be used locally with the reference implementation provided by the developer.
Despite its high performance and ability to deploy flexibly, security is where Mistral AI becomes a vulnerability.
LLM models such as GPT3 and Llama 2 have strict content filters that reject messages that the parent company deems harmful, but Mixtral 7B lacks this "safety guardrail". When a user asked Mistral AI's Q&A model how to make a bomb and commit a murder, its chatbot gave a scary detailed instruction.
Although the Mistral AI team is committed to sharing its technology openly, this could become a double-edged sword for its AI products, as regulators may take tougher measures against the model due to its lack of traditional content filters.
On the other hand, Arthur Mensch, CEO of Mistral AI, once said at the AI Security Summit: "There is a trade-off between the risks and benefits of open source, and we need to find the best solution through dynamic conversations. ”
It is reported that the company is building a platform with modular filters and modular mechanisms for managing model networks. Perhaps, the company will work on AI security and protection issues from the inside of the model.
Among today's highly competitive large language models, Mistral AI stands out for its superior performance and excellent adaptability. However, in the face of potential AI security challenges, the industry is also thinking about how to strike a trade-off between open source and security.
04. Create a smart future with Google Cloud
As we all know, Google Cloud is a leader in the global cloud computing field, and when it meets Mistral AI, a dark horse in the AI field, a future full of infinite possibilities is gradually unfolding before our eyes.
Last month, Google Cloud announced a global partnership with Mistral AI, which will use Google Cloud's infrastructure to distribute and commercialize its large language models.
With the help of Google Cloud's powerful cloud computing and big data technology, Mistral AI is expected to make unprecedented breakthroughs in model inference, pre-training and other fields. This will not only further promote the development of AI technology, but also bring smarter and more efficient solutions to various industries.
At the same time, the cooperation between the two parties will also accelerate the application of Mistral AI in various industries. Whether it's e-commerce, finance, healthcare, or education, Mistral AI will bring more convenience and well-being to humans.
Of course, the rise of Mistral AI is not accidental. As a dynamic and innovative company, Mistral AI has always been committed to exploring the boundaries of AI technology and applying it to solve real-world problems.
Its outstanding performance and innovative ability have led people to wonder if it is possible for this startup to surpass OpenAI and become the leader in the field of AI in Europe?
[Statement]: This article is the original of the heart of the metaverse operation team, it is strictly forbidden to reprint without permission, if you need to reprint, please contact us, the copyright and final interpretation of the article belong to the heart of the metaverse.