Editor: Mindy
Automatically combine different open-source models to generate new models with new capabilities, and the new method developed by Sakana AI has done it!
The basis of human intelligence does not lie in individual intelligent individuals, but in collective intelligence.
For example, our social and economic systems are based on many institutions made up of different individuals with different specialties and expertise.
The Sakana AI team believes that the development of AI will follow a similar collective path.
The future of AI will not be a single, gigantic, omniscient AI system that requires enormous energy to train, run, and maintain;
Rather, it will be a broad ensemble of many small AI systems, each with its own expertise and interconnected, and new AI systems being developed to fill specific specialties.
In fact, open-source base models can be easily extended and fine-tuned in hundreds of different directions to produce new models that excel in their own specialty.
Most of the top-performing models on the Open LLM rankings are no longer original open-source base models, such as LLaMA or Mistral, but models that fine-tune or merge existing models.
At the same time, the computational resources are much smaller than traditional LLM training.
Currently, Hugging Face has more than 500,000 models, covering dozens of different modalities, which can be combined into new models with new capabilities in principle.
Sakana AI has turned this idea into reality. They developed a method of evolutionary model merging, a general approach that uses evolutionary techniques to effectively discover the best way to combine different open-source models.
Address: https://arxiv.org/abs/2403.13187
Surprisingly, the method of merging evolutionary models is able to automatically discover new ways to merge different models from very different domains (e.g., non-English language and mathematics, or non-English language and vision) that may be difficult for human experts to discover on their own.
Use evolutionary algorithms to find the optimal combination
Human intuition is limited. As the diversity of open models and tasks continues to grow, a more systematic approach unlocks a more effective merging solution: evolutionary algorithms inspired by natural selection.
Here's an example of a simple evolutionary algorithm for automatically designing a 2D car that travels a long distance.
At first, many designs are randomly generated, but only a few that can move forward will pass on their successful traits to the next generation.
Through multiple generations of natural selection, these genetic cars can efficiently advance through the environment, traversing challenging terrain.
Sakana AI's evolutionary model merging method combines two different evolutionary approaches.
1. Merge models in the data flow space (layer).
This approach uses evolutionary discovery to optimize how to combine layers of different models into new models.
Intuition and heuristics are used to determine how and which layers combine the layers of one model with those of another.
This problem has a large combinable search space and is best suited for searching with optimization algorithms such as evolution.
2. Merge models in parameter space (weights).
The second approach is to evolve a new way of mixing the weights of multiple models.
There are endless ways to mix weights from different models, not to mention the blending scale of each layer. This is where evolutionary approaches can be applied to effectively find novel hybrid strategies that combine the weights of multiple models.
When these two methods were used together, the Sakana AI team found that the model performed well even in two domains that were relatively far apart, such as mathematics and non-English languages, or visual and non-English languages.
The new model of the fusion performs well
- Large Language Model (EvoLLM-JP)
The first model generated by the Sakana AI team using the method of merging evolutionary models is a large language model that can both Japanese and mathematics.
To build such a model, they used an evolutionary algorithm that merged the Japanese LLM (Shisa-Gamma) with the math-specific LLM (WizardMath and Abel).
LLM performance comparison, the MGSM-JA column shows the percentage of correct answers. Models 1-3 are the original models, and models 4-6 are the optimized merge models. Models 7-10 are the scores of existing high-performance LLMs used for comparison.
The table above reports the results of the evolved LLM model. Model 4 is optimized in the parameter space, and Model 6 is further optimized in the data flow space using Model 4.
The correct response rates for these models were significantly higher than those for the three source models.
This model can also produce some interesting examples, such as being able to solve mathematical problems that require knowledge of specific Japanese cultures.
- Visual Language Model (EvoVLM-JP)
Evolutionary algorithms are able not only to discover novel ways to merge text-only LLMs, but also to evolve models with different architectures created for different purposes. For example, a large model that combines Japanese and visual.
Both the JA-VG-VQA-500 and JA-VLM-Bench-In-the-Wild are benchmarks for questions and answers about images. The higher the score, the more accurate the description will be.
Interestingly, the combined model was able to achieve higher scores not only than the English VLM of its base LLaVa-1.6-Mistral-7B, but also higher scores than the existing Japanese VLM JSVLM.
The following is an example of an EvoVLM-JP response. Most other models would normally output green as the answer, but the EvoVLM-JP's answer is blue.
Green is not technically wrong, but in Japanese customs, the "permission to go" for traffic lights is described as a "blue light". If you're learning Japanese, this is an interesting example of Japanese culture.
- Image Generation Model (EvoSDXL-JP)
Evolutionary algorithms can also automatically discover ways to merge different diffusion models, and they are particularly fast to generate.
For example, isn't it amazing that EvoSDXL-JP, which is suitable for Japanese, can quickly generate the following images with four keywords: "Miso ramen, high-quality ukiyo-e, Hokusai, and Edo period"?
Heralding a new trend in AI development?
At the end of the paper, the authors say that these are only preliminary results and the beginning of a long-term development in applying evolutionary principles to the development of fundamental models.
The idea behind this project is not to create better models, but to get more metrics to help develop a more principled approach to model merging.
Jim Fan, a senior scientist at Nvidia, commented,
"The base model community is almost entirely focused on learning, but not on searching. I believe the latter has great potential in both the training (evolutionary algorithm) and inference (AlphaGo-style search) phases. 」
In addition to model selection, the Sakana AI team is also exploring the use of evolution to generate a diverse set of foundational models with their own unique domains and behaviors.
Does this indicate that the collective intelligence of self-improving models is not far off?