Reports from the Heart of the Machine
Machine Heart Editorial Department
Recently published in Nature · A new study in Communications suggests that advanced visual cognitive functions can arise spontaneously in untrained neural networks, and that the visual selectivity of facial images can even be produced in completely untrained deep neural networks.
For animals' social behavior (the behavior of different members of a group to work together to sustain group life), the ability to detect and recognize faces is crucial. This ability is thought to originate from neuronal tuning at the level of single or multi-neurons (neurons selectively represent a characteristic of information such as sensation, coordination, movement, cognition, etc.).
Scientists have observed neurons that respond selectively to faces in young animals of different species, which has caused a heated debate: Are face-selective neurons innate to the brain, or do they need to rely on visual experience?
Recently, a research team led by Se-Bum Paik, a professor in the Department of Bioencepharmacy at the Korea Advanced Institute of Science and Technology (KAIST), contributed a valuable result to this problem. They found that even deep neural networks that were completely untrained could produce visual selectivity for images of faces. Specifically, in the absence of learning at all, they observed neuronal activity that was selective for face images in a randomly initialized deep neural network that showed those features observed in the biological brain.
The new study was published in the December book Nature · Newsletter magazine. It provides enlightening insights into the underlying mechanisms underlying the development of cognitive function in biological and artificial neural networks, and also has a significant impact on our understanding of the origins of early brain function (before sensory experience).
Using AlexNet45, a model neural network that captures the characteristics of the ventral stream of the visual cortex, the team found that face selectivity can occur robustly under different conditions of randomly initialized DNNs. Moreover, their face selectivity index (FSI) is comparable to those of face selective neurons observed in the brain.
The preferred feature image obtained by means of the reverse correlation (RC) method and the generation of adversarial networks shows that the face selection unit is selective for the configuration of the class face, unlike the unit without selectivity. In addition, the face selection unit enables the network to perform face detection.
Interestingly, the researchers also found that in untrained neural networks, unit selectivity for various non-face objects can also be innately generated, meaning that face selectivity may not be a special type of visual tuning, while selectivity for various object categories can also be innately generated in untrained DNNs, spontaneously generated by random feed-forward connections.
These results suggest a possible scenario where random feed-forward connections developed in early untrained networks may be sufficient to initialize the original visual cognitive function.
Professor Paik said: "Our findings suggest that innate cognitive function can spontaneously arise from the statistical complexity embedded in the hierarchical feed-forward projection circuit, even in the absence of learning at all. The findings provide a wide range of conceptual advances and an in-depth understanding of the mechanisms behind the development of innate functions of biological and artificial neural networks, which help solve the puzzle of intelligence generation and evolution."
Reference link: https://techxplore.com/news/2021-12-untrained-deep-neural-networks.html