Author | Twilight
Edit | End of the bush
On June 22, the Beijing Zhiyuan Conference held a special forum on the foundation of cognitive neuroscience, and Professor Bi Yanchao from the State Key Laboratory of Cognitive Neuroscience and Learning of Beijing Normal University, Professor Fang Fang from the School of Psychology and Cognition of Peking University, Professor Liu Jia from the Department of Psychology of Beijing Normal University, Professor Wu Si from the Department of Computer Science of Peking University, and Professor Yu Shan from the Institute of Automation of the Chinese Academy of Sciences made reports to explore what inspiration cognitive neuroscience can bring to AI.
The third speaker was Liu Jia, a professor at the Department of Psychology at Beijing Normal University, entitled "From Cognition to Computation: The Science of Cognitive Neurointelligence."
In the report, Professor Liu Jia first reviewed the history of cognitive science, explained the significance of opening the black box of the human brain, and then revealed the internal representation and algorithm of deep neural networks through a series of experimental paradigms and research techniques of cognitive neuroscience to open the black box of AI, showing the possible path of fusion between the human brain and brain-like bicerebellum.
The following is the full text of the speech, ai technology comments have been made without changing the original meaning of the collation.
Today's report revolves around how to understand how deep neural networks work from the methodology of cognitive neuroscience to the brain.
1
In AI, we usually have a problem with picture recognition, we type the picture into the trained cnn, and the cnn tells us that it's a horse. This process is what our current mainstream deep neural networks do, using behavioral goal orientation, that is, making associations at the input and output, and treating the intermediate process as a blackbox.
Obviously, as scientists, we are certainly interested in opening it, but the question is is it necessary? Does opening and not opening really help to understand AI and promote AI development?
There have been similar debates in the history of psychology. Research on the relationship between stimuli and behavior was first conducted by Pavlov, who called it conditioned reflexes. That is, when the bell and the food appear at the same time or the bell appears slightly earlier than the food, the connection between stimulation and behavior can be established. That is, when food does not appear, just shake the bell, the dog will also secrete saliva. What's going on in a dog's brain was thought to be unimportant and just a black box; what we need to focus on is the law of the connection between stimuli and behavior.
This idea occupied a major position in psychology from the 1930s to the 1950s and 1960s, called behaviorism. There is a famous black box metaphor for behaviorism, that is, the representative of behaviorism, Watson (Watson), said: "Give me a dozen healthy babies, a special environment at my disposal, let me raise them in this environment, I can guarantee, choose any one, regardless of his parents' talents, tendencies, hobbies, his parents' profession and race, I can train them as I wish to be any character - doctor, lawyer, artist, big businessman, even beggar or robber." ”
The logic behind this statement is the "behavior and goal orientation" of deep neural networks, which translates to psychological terms as "man is a product of the environment" or "intelligence is a product of the environment".
2
But is it enough to understand the relationship between the external environment and behavior? Subsequent studies have shown that this is not enough. Garcia has studied the negative effects of radiation therapy, such as nausea and vomiting. Specifically, he gave the mice radiation therapy and then observed the behavior of the mice after the radiation treatment. Garcia found a very strange phenomenon, some of the rats after radiation treatment began to refuse to drink water, no matter how thirsty they were. Garcia dug deeper and found that the water containers for rats who refused to drink water were plastic bottles, while the water containers for rats who continued to drink water were glass bottles.
What is the difference between glass and plastic? It is very simple, because the glass bottle is tasteless, and the plastic bottle is smelly, that is, the rat associates the symptoms of nausea and vomiting with the taste of the plastic bottle, and the mouse will "think" that his vomiting is brought by the plastic bottle. On the surface, this is a very simple connection between stimuli (the smell of plastic bottles) and behavior (vomiting), which is what we just called the conditioned reflex. But! Garcia further found that when he used scent-like conditions, such as flashes and ringtones, to try to form the conditioned reflex of mice not drinking water, he found that no connection could be formed. That is to say, the mouse can only connect the smell with its vomiting, and cannot connect the flash and the bell with its vomiting. Based on this, Garcia challenges behaviorism with the concept of biological preparedness.
At the heart of biological readiness are two things: first, not all stimuli are connected to responses; second, an organism's learning potential is constrained by its biological basis. That is to say, the contents of the black box restrict the formation of stimulus and response connections.
It was Garcia's experiment that led us to study what was "thinking" in the brains of mice and what was "thinking" in the brains of dogs, and cognitive science was born. Scientists began to gradually open the black box of the brain, and concepts such as knowledge representation and attention are cognitive concepts proposed by cognitive science when studying brain mechanisms. In the past, behaviorism believed that man was only a product of the environment, but now we know that man is not only a product of the environment, but also the creator of the environment, and man has his own internal processing process. Similarly, the internal representations and algorithms of deep neural networks must also affect the connection between stimuli and behavior, and must also determine the form and nature of their intelligence.
After cognitive science and neuroscience make connections, we begin to understand the biological basis of cognitive modules and representations. Based on the work of cognitive neuroscience over the past 20-30 years, we are beginning to understand the mechanisms by which vision is produced. The first is the primary visual process, which makes a preliminary analysis of the characteristics of the object such as lines, colors, contrasts, and movements. Next is the intermediate visual process, we begin to integrate the object from the local information into the shape, surface, depth information, and finally we integrate this information into the advanced visual process, at which time we can achieve object recognition and so on.
Cognitive neuroscience has helped us open up part of the brain's black box. So why don't we use the methodology and tools of cognitive neuroscience to understand the functional modules and internal representations of artificial neural networks, understand the intelligent nature behind artificial intelligence, and obtain explainable and predictable AI? Here, I call this line of thinking cognitive neural analysis of artificial intelligence, that is, using the method of cognitive neuroscience to study AI.
3
1. Whether the human brain and brain-like brains use the same representations to complete the task
Essentially, the Turing test is based on the logic of behaviorism — a machine has the same intelligence as a human as long as it behaves. But from the perspective of cognitive science, a more essential test should be whether an intelligent machine has the same cognitive processes as humans. For example, AI can now achieve tasks such as object recognition and object detection, but is the internal representation used by AI the same as that of humans? In this study, we will specifically answer two questions: What representations do deep neural networks use? Is this representation similar to that of humans?
We present a gender-identifying task here, with women on the left and men on the right in the image below. But if I ask, what characteristics do you judge by? The length of their hair? Are their eyes the size? The outer contours of their faces? Or what? You can reflect on what you're relying on to make judgments.
You'll feel that this task is difficult, it's easy to identify gender, but it's hard to understand which characteristics to use. Because our facial cognitive processing is done unconsciously, it cannot be perceived by our consciousness. Here, we take the cognitive neuroscience approach, the reverse correlation, to extrapolate internal representations back and forth through the results.
First, we averaged female faces and male faces to get the average faces of women and men. When we make a smooth transition from the average female face to the male average face, let's feel the effect.
The animation gives the impression that your judgment of gender is similar to a dichotomy. At the beginning is a female face, followed by a male face, and in the middle is a perceptual boundary, and our psychological feelings do not change linearly with the linear change of the image, but a dichotomy, the first half is all female, and the second half is all male. Here, we find the perceptual boundary and generate a neutral face.
Next, we train a gender-aware vgg-face network. The network is pre-trained, and we only do transfer learning, that is, fine-tuning the last layer and doing recognition training on male and female faces. Soon, the accuracy of gender recognition reached 100 percent. We take the neutral faces out and add random noise, and then we enter the photo into vgg-face to classify it. Adding noise allows neutral faces to be recognized as male or female faces. We identified 20,000 photos, each with the same basemap and different noise added, so we could get a set of photos identified as female by vgg-face and a set of photos identified as male.
We labeled all the photos, then removed the original basemap, leaving only noise and superimposing them on gender labels. The following figure is the vgg-face facial features that identify faces as women. The original random noise looks irregular, but the structure information can be extracted from the noise through the reversecorrelation. We roughly see that this information is mainly concentrated in places such as eyes, nose and mouth, and these characteristics are the key information for vgg-face to judge women by their faces.
Similarly, we can overlay the noise that is judged to be male and get a feature map of men. A simple comparison can be found that the feature map judged to be female and the feature map judged to be male are not the same, and the pattern of the two figures is very complicated.
We subtract the female feature map and the male noise feature map to obtain the recognition feature map, which is the internal representation of vgg-face to complete the gender recognition task, which is the key information to separate men and women. We superimpose the basemap, the neutral face, and we can see that the extremum points of the noise characteristic map are roughly distributed on the outside of the eyes and nose, as well as the lower edge of the person and the lips.
We're going to overlay this on the basemap and we get a standard male face. Conversely, if we subtract the basemap from this feature map, we get a standard female face. So through this series of operations, we get what characteristics vgg-face uses to make gender judgments.
What if you replace vgg-face with a human? We looked for someone to see these 20,000 pictures. In most cases, participants would say, "How do I know if he's male or female?" We said, "It's okay, you guessed it, follow the feeling, you think it's a woman press f, think it's a man press m." So the subjects were confused, confused and tired, and completed the experiment. This is the feature map they use to distinguish between men and women. We get the standard face of a man and a standard face of a woman according to the same calculation.
We found that the feature diagram in vgg-face is very similar to that of humans. In fact, if we calculate the correlation between these two feature maps, we can get a correlation of 0.73. From this perspective, humans and vgg-faces use similar representations to accomplish the task of gender recognition.
Further, let's see what spatial frequencies this similarity occurs at. In the study, the random noise added to the neutral face is structured, composed of graphs of different spatial frequencies, the leftmost figure below is low frequency, the rightmost is high frequency, we superimpose the information of low frequency and high frequency, to give you a noise map for experimentation.
Now look at what the feature map of the person and vggface looks like under different spatial frequencies. These feature maps are also very similar, and the similarity is highest at low frequencies, with people and vgg-faces becoming less and less similar as the spatial frequency increases. Therefore, vgg-face and humans rely more on low-frequency information when completing face gender recognition tasks.
To summarize briefly, David Marr, one of the founders of computer vision, proposed that we should understand intelligence on three levels:
The first level is the goal achieved or the task completed, for example, this experiment does the gender recognition task, which is the highest level;
The lowest level is the level of the physical implementation, that is, the hardware used to implement it. In this study there are two implementations of hardware, one is vgg-face, one is the human brain, which are two completely different physical layers;
To achieve the goal with physical hardware, there is also a software level in the middle, called representation and algorithm. Representations and algorithms establish a transformation between input and output, and this transformation is intelligence. The essence of intelligence is representation. In the above study, representation is a feature map that distinguishes males from females.
2) Similar mission experience is important for forming similar representations
What are the prerequisites for vgg-face and humans to use similar representations to complete gender recognition tasks?
Faces are more special for humans, we see a face, usually need to identify the identity, that is, directly identify the individual, that is, this is Zhang San. But for non-face objects, our recognition is usually at the category level, for example, when we see a cat, we will only say that it is a cat, not that it is Zhang San's cat.
The second is that the recognition of faces relies more on low-frequency information, such as the negative effect of psychology, flipping the black and white value of the photo, and finding it very difficult to identify, and also filtering the low-frequency information, recognition is also very difficult.
Because vgg-face is a pre-trained task for face recognition; vgg-face has a similar representation to a person, probably for the above two reasons, namely: (1) vgg-face and both people recognize objects at the individual level; (2) vgg-face and people are sensitive to unique features of faces (such as low-frequency information) because they have processed a large number of faces.
First, let's test the first possibility: a common mission experience. Here, we choose Alexnet. Alexnet is also a pre-trained network, it does not do face recognition but to do object classification, we fine-tuned the last layer, let it do the classification task of identifying men and women, the accuracy rate is 93%. That is, although alexnet is used to train object classification, it can also distinguish between male and female, and the accuracy rate is also quite high.
Now an interesting question to ask, alexnet can achieve the same accuracy as people in gender recognition, but does Alexnet use similar representations to people? Let's look at the alexnet feature map to distinguish between men and women, as shown in the following figure, the naked eye can distinguish that there is a very big difference between the two, basically uncorrelated, and the correlation degree is equal to -0.04. We overlay it on top of the original basemap, and the resulting face has no obvious gender characteristics. So from this point of view, we find that although Alexnet can distinguish between male and female, the representations it uses are completely different.
We do further spatial frequency analysis, the noise feature map is divided into different spatial frequencies, you can see that basically the alexnet and the noise feature map of the various frequencies of humans are not related.
Going back to the conclusion of the first part of the experiment, we found that the pre-training task is very important. Why are vgg-faces similar to the representations humans use to distinguish between males and females? Because they are all trained to process at the individual level, and Alexnet is processed at the class level, from this point of view, they use different representations.
We can understand this from an evolutionary perspective. The reason why we have changed from single-cell to multi-celled animals is because we are constantly completing the tasks that nature has entrusted us; once we can't do it, there is only one result, that is, the gene is eliminated. That is, we are what we do. Our intelligence is determined by the tasks we have accomplished in the past.
To sum up: Cognitive neuroscience has developed a range of useful tools and methodologies as well as experimental paradigms that help us understand the internal features and modules of deep neural networks and derive explainable, predictable deep neural networks.
Further, cognitive neurointelligence, formed by the deep intersection of cognitive science, neuroscience, and intelligence science, will provide a new perspective on the nature of intelligence. Specifically, an ideal model for studying intelligence is to discover a brain inspiration through neuroscience, cognitivemodeling the mechanism according to cognitive science, and then use computational science to develop an algorithm with moderate computational complexity (physicalimplementation) to solve a real-world problem.
Lei Feng network Lei Feng network Lei Feng net