Microsoft has unveiled its latest lightweight AI model, the Phi-3 Mini, the first of three small models the company plans to release, and the smallest to date. It's a 3.8 billion parameter language model that rivals the performance of models like Mixtral 8x7B and GPT-3.5, which are 10 times larger, while being compact enough to run on smartphones.
Interestingly, Microsoft's developers trained the Phi-3 Mini like children. They are inspired by how children learn from bedtime stories, books with simpler words, and sentence structures that talk about larger topics. "There weren't enough children's books on the market, so we made a list of more than 3,000 words and asked AI large language models to make 'children's books' to teach Phi," said Eric Boyd, corporate vice president of Microsoft's Azure AI platform.
Microsoft researchers set out to create a discrete dataset starting with 3,000 words that included roughly equal numbers of nouns, verbs, and adjectives. They then asked a large language model to create a children's story using a noun, a verb, and an adjective from a list—a prompt they repeated millions of times over several days, generating millions of small children's stories.
The Phi-3 Mini, while small, doesn't score badly on the test, achieving a score of 69% on the MMLU benchmark and an 8.38 on the MT benchmark, demonstrating comparable inference capabilities to the larger model. The model is also tuned for robustness, security, and chat interaction during training.
One of the most striking aspects of small language models like the Phi-3 Mini is their ability to run natively on smartphones. By quantifying the model to 4 bits, the researchers were able to reduce its memory footprint to just 1.8GB. This allowed them to deploy the model on the iPhone 14, run locally on the device and completely offline. Despite the limitations of mobile hardware, the Phi-3 Mini managed to generate more than 12 tokens per second.
The 4-bit quantized phi-3-mini runs natively on an iPhone with an A16 Bionic chip, generating more than 12 tokens per second
While the Phi-3 Mini may lack the huge knowledge storage capacity of its larger counterparts, its ability to search makes up for this weakness. The developers demonstrated the potential of the model by integrating it with a search engine, giving it instant access to relevant information. Phi-3 Mini is already available on Azure, Hugging Face, and Ollama.
The development of the Phi-3 Mini is part of a broader trend in the AI industry to create smaller, more efficient models that can be deployed on a wider range of devices. Rumor has it that Apple is working on an on-device model for its next-generation iPhone. Microsoft's competitors also have their own small AI models, most of which target simpler tasks such as document summarization or coding assistance. Google's Gemma 2B and 7B are great for simple chatbot and language-related jobs. Anthropic's Claude 3 Haiku can read intensive research papers with charts and quickly summarize them, and Google has previously released the Gemma 2B and Gemini Nano.
Microsoft plans to release two more models in the Phi-3 series: the Phi-3 Small (7B parameter) and the Phi-3 Medium (14B parameter). Early results suggest that these models will further push the boundaries of what is possible for smaller language models, with Phi-3 Medium reaching 78% on MMLU and 8.9 on MT-bench.
Microsoft calls the current AI model suitable for small terminal deployment SLM (small language model). "The proposition here is not that SLM will replace or replace large language models," said Ece Kamar, Microsoft's vice president who leads the AI Frontiers Lab at Microsoft Research. Instead, SLM "is uniquely positioned for edge computing, on-device computing, and computing that gets work done without having to move to the cloud." That's why it's important that we understand the pros and cons of this model combination. "It's also about choosing the right size language model for the right task.
Reference Links:
news.microsoft.com/source/features/ai/the-phi-3-small-language-models-with-big-potential
huggingface.co/microsoft/Phi-3-mini-128k-instruct
arxiv.org/abs/2404.14219