Heart of the Machine column
Heart of the Machine Editorial Department
The ability to Xi in an open, highly dynamic and evolutionary environment is one of the core elements of biological intelligence and an important advantage developed by humans and most animals in the natural selection process of "survival of the fittest". At present, the traditional machine Xi paradigm is to learn Xi a model on static and closed datasets, and assume that its application environment has the same attributes as the previous training data, so it cannot adapt to the challenges of dynamic open environments.
In order to solve this problem, continuous learning Xi simulates the learning Xi process and learning Xi ability of biological intelligence, develops new machine Xi theories and methods, and improves the adaptability of agents to open and highly dynamic environments through the process of continuous learning and Xi. However, the current mainstream machine Xi model Xi by adjusting the network parameters, and when the data distribution of the learning Xi task changes, the previously learned network parameters may be overwritten, resulting in catastrophic forgetting of prior knowledge.
As an important bottleneck in the development of artificial intelligence, especially deep learning Xi, continuous learning Xi has received extensive attention in the field of artificial intelligence in recent years. Most continuous learning Xi methods focus on improving the memory stability of what has been learned to overcome catastrophic forgetting, such as fixing network parameters that execute old tasks when learning Xi new tasks. However, these methods can often only work in specific scenarios, and it is difficult to have universal adaptability to the complex environments and tasks of the real world like biological intelligence.
Therefore, whether we can learn from the continuous learning Xi mechanism of the biological brain and develop new continuous learning Xi methods has always been a common concern in the field of artificial intelligence.
针对该问题，近期清华大学计算机系朱军教授 TSAIL 课题组联合生命学院钟毅教授课题组在自然机器智能（Nature Machine Intelligence）期刊上发表了题为「融入神经启发适应性的人工智能方法」（Incorporating neuro-inspired adaptability for continual learning in artificial intelligence）的研究论文，并被选作12月的封面文章。
This study uses Bayesian methods to deeply analyze and model the adaptive mechanism of biological Xi memory systems, which significantly improves the continuous learning and Xi ability of deep neural networks, and provides interdisciplinary insights for the adaptive development of intelligent systems in dynamic open environments.
Paper link: https://www.nature.com/articles/s42256-023-00747
introduction
With the advent of large-scale annotated data and the enhancement of the computing power of hardware devices, artificial intelligence with deep learning Xi as the core has made a series of breakthroughs in the fields of computer vision, natural language processing, autonomous driving, and intelligent robots. However, deep learning Xi highly dependent on static data distribution, and it is difficult to continuously study and Xi dynamically changing data distribution.
From a theoretical point of view, the optimization goal of continuous learning Xi can be further refined into core elements such as memory stability, learning Xi plasticity, and generalization compatibility. In order to adapt to changing environments and tasks, intelligent systems need to strike the right balance between memorizing old knowledge and learning Xi new knowledge, and have sufficient generalization capabilities to accommodate differences in data distribution.
Figure 1. Schematic diagram of a biointelligence-inspired continuous Xi method (Source: NMI original)
As a natural template, humans, as well as most animals, are born to learn Xi in a continuous way. Even simple living organisms such as fruit flies have evolved a variety of adaptive mechanisms to enable effective continuous learning Xi. In the learning Xi memory system of Drosophila, the dynamically changing sensory information can be selectively protected and forgotten in multiple parallel continuous learning Xi modules, which provides important implications for artificial intelligence.
Figure 2. Selective Protection of Memory and Amnesia Mechanisms (Source: NMI original)
Study overview
At the methodological level, the researchers propose a biologically inspired method of memory regulation to selectively preserve and forget what has been learned. When Xi learning new tasks, this method promotes memory stability by optimizing the old task information in the parameter distribution, and introduces a certain degree of forgetting rate to promote learning Xi plasticity. The researchers further deduced an optimization algorithm for synaptic expansion-renormalization, which enabled the neural network to make a clear trade-off between the optimal solution of the old and new tasks, and analyzed the role of the forgetting rate in reducing the generalization error of continuous learning Xi, echoing biological intelligence from the two levels of functional goals and implementation mechanisms.
Figure 3. Dynamically Regulated Parallel Multi-Module Architecture (Source: NMI original)
At the same time, the researchers constructed a parallel multi-module structure similar to the Drosophila Xi memory system, corresponding to multiple persistent Xi experts. By implementing the proposed memory regulation mechanism in each module, the memory is selectively protected and forgotten, so that each module can differentiate the appropriate task expertise and fully adapt to the differences in data distribution of different tasks. The researchers also delved into the interaction of the randomness factor of neural networks with the rules of learning and forgetting rates in Xi, demonstrating that the adaptive mechanisms of the nervous system do not operate in isolation, but are highly synergistic.
Figure 4. Experimental results of multiple continuous learning Xi benchmarks (Source: NMI original article)
Among a variety of continuous learning Xi benchmarks, including visual tasks and reinforcement tasks, the proposed adaptive mechanism can significantly improve the continuous learning and Xi ability of deep neural networks. In addition, from the perspectives of the biological significance and realization mechanism of forgetting, this paper also deeply explores the relationship between intelligent systems in continuous learning and Xi, as a new paradigm to promote the collaborative development of artificial intelligence and biological intelligence.
Figure 5. A comprehensive review of the Xi of continuing learning (Source: Arxiv paper https://arxiv.org/abs/2302.00487 by the team)
Introduction of the author and research group
Prof. Jun Zhu and Prof. Yi Zhong from Tsinghua University are the co-corresponding authors of this paper, and Liyuan Wang and Assistant Researcher Xingxing Zhang are the co-first authors of this paper. Assistant researcher Li Qian and Associate Researcher Su Hang from Tsinghua University, and Dr. Mingtian Zhang from University College London are the co-authors of this paper.
The TSAIL group at Tsinghua University has long been committed to the theoretical and algorithmic research of Bayesian machine Xi. In recent years, the research group has published a series of important results in the field of continuous Xi based on the cutting-edge progress of machine Xi and neuroscience. At the beginning of this year, he completed a review paper in the field of continuous learning Xi, "A comprehensive survey of continual learning: theory, method and application", which systematically sorted out the research progress in the basic setting, theoretical basis, representative methods and practical applications of continuous learning Xi, and put forward the future development direction, which has received extensive attention from the artificial intelligence community at home and abroad.
In addition, in order to solve the common technical difficulties in the field of continuous learning Xi, semi-supervised continuous Xi learning (CVPR'21) for generating models, a weight regularization method with selective forgetting (NeurIPS'21), a memory playback method with adaptive data compression (ICLR'22), and a continuous learning Xi architecture for dynamic parallel modules (ECCV'22) were proposed.
Recently, the research paper "Hierarchical decomposition of prompt-based continual learning: rethinking obscured sub-optimality" on the theory and method of continuous Xi learning of pre-trained models was awarded by NeurIPS'23 spotlight, this paper proposes a general framework suitable for various fine-tuning techniques (such as prompt, adapter, LoRA, etc.) by hierarchically decomposing the continuous Xi learning optimization objectives in the context of pre-training, which significantly improves the adaptability of the pre-trained model in a dynamic open environment.