Compile | giraffe
Edit | vanilla
Zhidong reported on April 24 that on April 23, Microsoft open-sourced a small language model (SLM) - Phi-3-mini on its official website. This is the first model launched in the Microsoft Phi-3 family of models.
As the fourth generation of Microsoft's Phi series, Phi-3-mini surpasses many models with tens of billions of parameters in language, reasoning, coding, and math benchmarks with a huge training data volume of 3.8 billion parameters and 3.3T tokens.
In addition, Phi-3-mini has two context-length variants, 4k and 128k tokens, which are pre-trained and instructionally tuned to better understand human language, expressions, logic, and execute different types of instructions.
Compared with large language models, Phi-3-mini models are easier to operate. The small size of the model allows it to be deployed and run on the device side. The model is said to be accessible offline as a chatbot and has performance that is not inferior to GPT-3.5.
The cost of the Phi-3-mini was significantly reduced. Sébastien Bubeck, Microsoft's vice president of generative AI research, said the Phi-3-mini could cost a tenth of what other models with similar features do.
According to the Microsoft research team, the innovation of Phi-3-mini's small size and high performance is due to a dataset composed of synthetic data. The dataset consists of a large amount of filtered web data and synthetic data from other AI-made children's books.
微软计划在未来几周内推出Phi-3系列的另外两个模型，分别是Phi-3-small（7B）和Phi-3-medium（14B）。
There are currently three platforms where the Phi-3-mini is available:
Microsoft Azure AI Studio：https://ai.azure.com/explore/models/Phi-3-mini-128k-instruct/version/2/registry/azureml
Hugging Face：https://huggingface.co/collections/microsoft/phi-3-6626e15e9585a200d2d761e3
Call:https://ollama.com/library/phi3
1. For the "small but beautiful" model, the Phi-3-mini is better than twice its size
According to Microsoft's official website, the Phi-3 model is currently the most powerful and cost-effective small language model (SLM), and it has demonstrated the ability to outperform models of the same size or even larger in multiple languages, reasoning, coding, and math benchmarks.
Phi-3-mini includes two context-length scales, 4k and 128k tokens, with the context window support for 128k tokens being implemented for the first time in a similar model with minimal impact on model quality. In addition, the model is fine-tuned to respond to and execute multiple instructions in a natural language manner, ensuring its immediate usability.
The Phi-3 model has breakthrough performance in a small form factor. According to tests conducted by Microsoft's R&D team, the Phi-3 model outperforms language models of the same size and larger size in benchmark tests. Among them, the Phi-3-mini outperforms models twice its size, while the Phi-3-small and Phi-3-medium outperform larger models, including the GPT-3.5 Turbo.
According to the performance comparison table, the two variants of the Phi-3-mini outperformed the Gemma-7b model in 17 out of 19 benchmarks, the Mistral-7b model in 18 ways, and the Llama-3-8B model in 11 ways. However, there are also 11 entries inferior to the GPT3.5-Turbo model and 17 entries inferior to the Claude-3 Sonnet model.
▲Phi-3 series benchmark comparison table (source: Microsoft)
In addition, Microsoft also said in the academic report that the overall performance of the Phi-3 model is comparable to that of models such as Mixtral 8x7B and GPT-3.5.
The Phi family of models originated from Microsoft Research and has been widely used, with Phi-2 being downloaded more than 2 million times. Starting with Phi-1 for Python coding, to Phi-1.5 for enhanced reasoning and understanding, to Phi-2 with 2.7 billion parameters, the Phi series outperforms its 25x parameter size models in terms of language understanding.
Microsoft plans to launch two more models in the Phi-3 family in the coming weeks, the Phi-3-small (7B) and the Phi-3-medium (14B), designed to provide more flexible options for Azure AI and other model libraries.
2. Phi-3 follows safety and high quality, and Phi-3-mini is specially optimized for multiple platforms
The Phi-3 model is designed to adhere to Microsoft's Responsible AI Standards, including accountability, transparency, fairness, reliability and security, privacy and security, and inclusivity. Phi-3 models undergo rigorous security metrics and assessments, red team testing, sensitive use reviews, and follow security guidelines to ensure that these models are developed, tested, and deployed in accordance with Microsoft standards and best practices.
The Phi-3 model uses high-quality data during training, which is a continuation of the previous Phi model work. They have also undergone extensive post-safety training, including reinforcement learning (RLHF) from human feedback, automated testing on dozens of injury classes, and manual red team testing.
In addition, Phi-3-mini comes with a complete set of deployment, evaluation, and fine-tuning toolchains on Azure AI, and enables developers to run on local laptops through the Ollama platform. The model is also optimized for ONNX Runtime, supports Windows DirectML, and enables cross-platform support, including GPU, CPU, and mobile hardware.
Phi-3-mini is also available as an NVIDIA Inference Microservice (NVIDIA NIM) with a standard API interface that can be widely deployed and is specifically optimized for NVIDIA GPUs.
3. Phi-3 provides offline inference scenarios, and AI solutions have landed in India
Microsoft helps customers transform their businesses with generative AI by offering Copilots. They believe that there is a growing demand for different scale models on the cost-of-quality curve for different tasks.
Small language models, such as Phi-3, are particularly useful in resource-constrained environments, including on-device and offline inference scenarios, latency-constrained scenarios where fast response times are critical, and cost-constrained use cases, especially those with simpler tasks.
Due to its small size, the Phi-3 model can be used in computationally constrained inference environments, especially the Phi-3-mini can be used on the device side. The smaller size of the Phi-3 model also makes fine-tuning or customization easier and more cost-effective. In addition, lower computing requirements make it less costly and with better latency. The long contextual window allows it to process and reason about large amounts of textual content such as documents, web pages, code, and more.
Some of Microsoft's customers are already building solutions with Phi-3. In agriculture, for example, the Phi-3 model provides farmers with a more convenient and cost-effective solution and helps them use AI technology without a stable internet connection.
ITC, a leading business group in India, is using the Phi-3 model in Krishi Mitra, an app for farmers, to help Indian farmers access better agricultural solutions through technology. It is reported that the app has reached more than one million farmers.
Conclusion: Large and small models complement each other, and offline AI chat has become a reality
Microsoft's Phi-3 model was designed to be compatible with a wide range of devices. Like smaller models such as Stability AI's Zephyr, Google's Gemini Nano, and Anthropic's Claude 3 Haiku, Phi-3 is able to run on the device side without relying on an internet connection.
While large models in the cloud outperform small models in terms of performance across the board, they have some limitations, including higher costs, speeds, and dependence on internet connectivity. In contrast, Phi-3 allows users to interact with virtual assistants without an internet connection, and is able to summarize content without uploading data, which solves some of the drawbacks in the AI field.
In the future, such models are expected to be integrated with smartphones, and may even be built into commonly used home appliances to provide personalized recommendations for users' lives. We look forward to AI becoming more and more life-like, and we also look forward to the progress of small language models in the future.