Editor: Aeneas is sleepy
Meta's LLaMA model is open source, allowing the text model to usher in a Stable Diffustion moment. No one expected that LLaMA's "epic" leak would produce a series of stunning ChatGPT "replacements".
Who would have thought that an accidental LLaMA leak would ignite the biggest innovation spark in the open source LLM field.
A series of high-performing ChatGPT open source alternatives, the "Alpaca Family", followed by a dazzling debut.
The friction between open source and API-based distribution is one of the most pressing contradictions in the generative AI ecosystem.
In the text-to-image space, the release of Stable Diffusion makes it clear that open source is a viable distribution mechanism for the underlying model.
However, this is not the case in the field of big language models, where the biggest breakthroughs, such as GPT-4, Claude, and Cohere, are only available through APIs.
Open-source alternatives to these models do not exhibit the same level of performance, especially in the ability to follow human instructions. However, an unexpected leak completely changed this situation.
LLaMA's "epic" leak
A few weeks ago, Meta AI launched LLaMA, a large-language model.
LLaMA is available in different versions, including parameters for 7B, 13B, 33B, and 65B, and although it is smaller than GPT-3, it is comparable to GPT-3 performance for many tasks.
LLaMA wasn't open source at first, but a week after its release, the model suddenly leaked on 4chan, triggering thousands of downloads.
This event can be called an "epic leak", because it has become an endless source of innovation in the field of large language models.
In just a few weeks, the innovation of LLM agents built on it has exploded.
Alpaca、Vicuna、Koala、ChatLLaMA 、FreedomGPT、ColossalChat...... Let's review how this "alpaca family" explosion was born.
Alpaca In mid-March, the large model Alpaca released by Stanford went viral.
Alpaca is a new model fine-tuned from Meta's LLaMA 7B, using only 52k data, which is about the same performance as GPT-3.5.
The point is that the cost of training is surprisingly low, less than $600.
Stanford researchers compared GPT-3.5 (text-davinci-003) with Alpaca 7B and found that the performance of the two models was very similar. Alpaca has a 3.5 win against 90 against 89 in comparison with GPT-89.
For the Stanford team, training a high-quality instruction-following model on budget requires two important challenges: having a robust pre-trained language model, and a high-quality instruction-following data.
It is precisely the LLaMA model provided for academic researchers to solve the first problem.
For the second challenge, the paper "Self-Instruct: Aligning Language Model with Self Generated Instructions" gives a good inspiration for using existing strong language models to automatically generate instruction data.
The biggest weakness of the LLaMA model is the lack of instruction fine-tuning. One of OpenAI's biggest innovations is the use of instruction tuning on GPT-3.
To do this, Stanford used existing large-language models to automatically generate instructions following instructions.
Now, Alpaca is directly regarded by netizens as "the Stable Diffusion of the large text model".
At the end of March, researchers from UC Berkeley, Carnegie Mellon University, Stanford University, and UC San Diego open-sourced Vicuna, a fine-tuned version of LLaMA that matches GPT-4 performance.
The 13 billion parameter Vicuna, which was trained by fine-tuning LLaMA on user sharing conversations collected by ShareGPT, cost nearly $300 to train.
The results showed that the Vicuna-13B achieved capabilities that rivaled ChatGPT and Bard in more than 90% of cases.
For the Vicuna-13B training process, the details are as follows:
First, the researchers collected about 70K conversations from ShareGPT, a conversation-sharing site on ChatGPT.
Next, the researchers optimized the training script provided by Alpaca to enable the model to better handle multi-round dialogues and long sequences. This was followed by a day of training on 8 A100 GPUs using PyTorch FSDP.
In terms of assessing the quality of the model, the researchers created 80 different questions and evaluated the model output with GPT-4.
To compare the different models, the researchers combined the output of each model into a separate prompt and then let GPT-4 evaluate which model gave the better answer.
Comparison of LLaMA, Alpaca, Vicuna and ChatGPT
Koala
Recently, the UC Berkeley AI Research Institute (BAIR) released a new model, Koala, which differs from OpenAI's GPT data for instruction fine-tuning, and Koala is trained using high-quality data obtained by the network.
The findings show that Koala can effectively answer queries from a variety of users, and the resulting responses tend to be more popular than Alpaca, at least in half of the cases comparable to ChatGPT.
The researchers hope that the results of this experiment will further advance the discussion around the relative performance of large closed-source models relative to small public models, especially as it shows that for small models that can run locally, large model performance can also be achieved if training data is carefully collected.
In fact, the results of experiments that fine-tune LLaMA data based on OpenAI's GPT model released by Stanford University have shown that the right data can significantly improve smaller open source models.
That's why Berkeley researchers developed and published the Koala model, hoping to provide another experimental proof of the results of this discussion.
Koala fine-tunes free interaction data from the web, with a particular focus on data that includes interactions with high-performance closed-source models such as ChatGPT.
Instead of pursuing as much scraping of web data as possible to maximize the amount of data, the researchers focused on collecting a small, high-quality dataset, including ChatGPT distilled data, open-source data, and more.
ChatLLaMA
Nebuly open-sourced ChatLLaMA, a framework that uses a framework that lets us create conversational assistants with our own data.
ChatLLaMA lets us use our own data and as little computation as possible to create hyper-personalized ChatGPT-like assistants.
Suppose that in the future, we will no longer rely on a large assistant that "rules everyone", and everyone can create their own personalized version of ChatGPT assistants that can support various human needs.
However, creating this personalized assistant requires effort on many fronts: dataset creation, efficient training with RLHF, and inference optimization.
The purpose of this library is to give developers peace of mind by abstracting the work required to optimize and collect large amounts of data from computation.
ChatLLaMA is designed to help developers tackle a variety of use cases, all related to RLHF training and optimization inference. Here are some use case references:
- Create ChatGPT-like personalized assistants for vertically specific tasks (legal, medical, gaming, academic research, etc.);
- Want to use limited data on local hardware infrastructure to train an efficient ChatGPT-like assistant;
- Want to create your own personalized version of ChatGPT assistant while avoiding runaway costs;
- I want to know which model architecture (LLaMA, OPT, GPTJ, etc.) best meets my requirements in terms of hardware, computing budget, and performance;
- I want to align the assistant with my personal/company values, culture, brand, and statement.
FreedomGPT
Built using Electron and React, FreedomGPT is a desktop application that allows users to run LLaMA on their local machines.
FreedomGPT's special features are evident from its name – it answers questions that are not subject to any censorship or security filtering.
This app was developed by Age of AI, an AI venture capital firm.
FreedomGPT is built on top of Alpaca. FreedomGPT uses the distinguishing features of Alpaca because it is relatively more accessible and customizable compared to other models.
ChatGPT follows OpenAI's usage policy and restricts hateful, self-harm, threatening, violent, and sexual content.
Unlike ChatGPT, FreedomGPT answers questions without bias or favoritism and does not hesitate to answer controversial or controversial topics.
FreedomGPT even answers "how to make a bomb at home," which OpenAI specifically removed from GPT-4.
FreedomGPT is unique in that it overcomes censorship restrictions and caters to controversial topics without any safeguards. Its symbol is the Statue of Liberty, as this unique and bold model of the Great Language symbolizes freedom.
FreedomGPT can even run locally on your computer without the need for an internet connection.
In addition, the open-source version will be released soon, allowing users and organizations to fully customize it.
ColossalChat
UC Berkeley's ColossalChat requires less than 10 billion parameters to achieve bilingual capabilities in Chinese and English, and the effect is comparable to ChatGPT and GPT-3.5.
In addition, ColossalChat, based on the LLaMA model, also replicates the complete RLHF process, and is currently the closest open source project to the original technical route of ChatGPT.
Chinese-English bilingual training dataset
ColossalChat has published a bilingual dataset containing about 100,000 Chinese and English question answer pairs.
The dataset was collected and cleansed from real problem scenarios on social media platforms as a seed dataset, scaled with self-instruct and annotated at a cost of around $900.
Compared to datasets generated by other self-instruct methods, this dataset contains more realistic and diverse seed data covering a wider range of topics.
This dataset is suitable for fine-tuning and RLHF training. In the case of providing high-quality data, ColossalChat can achieve better conversational interaction while also supporting Chinese.
Complete RLHF pipeline
There are three phases to the algorithmic forking of RLHF:
In RLHF-Stage1, supervised instruction fine-tuning is performed using the bilingual dataset described above to fine-tune the model.
In RLHF-Stage2, the reward model is trained by manually ordering different outputs of the same prompt to assign corresponding scores, and then supervise the training of the reward model.
In RLHF-Stage3, reinforcement learning algorithms are used, which is the most complex part of the training process.
I believe that soon, more projects will be released.
No one expected that the accidental leakage of LLaMA ignited the biggest innovation spark in the open source LLM field.
Resources:
https://thesequence.substack.com/p/the-LLaMA%20%20-effect-how-an-accidental