If you are a student who is engaged in AI algorithms, I believe you have seen the full connection layer in many places.
Whether it's a convolutional neural network (CNN) or a natural language processing (NLP) network, you can see the fully connected layer.
So what exactly is a fully connected layer, what does this layer do in neural networks, and how does it relate to matrix multiplication and convolutional operations?
1. What is a fully connected layer?
全连接层（Fully Connected Layer），有时也被叫作密集层（Dense Layer）。
It is so called because each neuron in this layer is connected to each neuron in the previous layer, forming a fully connected network structure.
Fully connected structure: the source network
This fully connected approach differs from convolution and pooling, which move across the input via a fixed-size convolution kernel or pooling window in order to efficiently capture local features.
Thus, "full connection" emphasizes the connections between the neurons in this layer and all the neurons in the previous layer, in contrast to the local connections of the convolutional and pooling layers.
2. What is the function of full connection?
Since it is all connected by all neurons, it means that it can capture all the previous features, and after calculation, all the features can be fused more effectively and completely.
This makes the feature that the neural network ends up seeing is a global feature (a cat) rather than a local feature (eyes or nose).
There is a metaphor of ants meeting to compare the fully connected layer, which is more vivid.
Let's say you're an ant and you're tasked with finding loafs. At this time, your field of vision is narrow, and you can only see a small area, so you can only see a part of a large loaf of bread.
When you find a slice of bread, you don't know if you're finding all the bread, so all you ants have a meeting.
You share information with each other about the bread you found, and through the meeting, you share it, and eventually you confirm, oh, you've found a big loaf.
The process of ant meeting mentioned above is full connection, which is why full connection needs to connect all nodes and complete the information sharing of all nodes as much as possible.
Speaking of which, you can probably understand the role of full connection, that is, it can complete the fusion of all features.
3. The relationship between full connection and convolution and matrix multiplication
From a purely mathematical point of view, full connection is matrix operation, and the essence of matrix operation is clearer here: understand the essence of matrix multiplication in 5 minutes.
The difference between fully connected and convolutional is that in addition to the above mentioned that the former is a global field of view and convolution is a local field of view, there is also a very important connection.
That is, if the convolution kernel of the convolution becomes 1x1, then every pixel equivalent to the convolution kernel is associated with every pixel of the input image, and the convolution of 1x1 becomes matrix multiplication.
Fully connected is usually placed at the end of a neural network and is used for classification.
Suppose that the first layer of the neural network learns 1000 features, then the last layer of full connection can fuse all 1000 features, and after fusion, you can know that the image is a dog.
To borrow an explanation of this algorithm from "Turing's Cat": the fully connected layer acts as a "classifier" in the entire convolutional neural network.
If operations such as the convolutional layer, pooling layer, and activation function layer map the raw data to the hidden layer feature space, the fully connected layer plays the role of mapping the learned "distributed feature representation" to the sample label space.
I will share some of the artificial intelligence learning materials I have compiled for free to you, which has been sorted out for a long time and is very comprehensive. Including artificial intelligence basic introductory video + AI common framework actual video, machine learning, deep learning and neural network and other videos, courseware source code, completed projects, AI popular papers, etc.
The following is a screenshot, scan the code to enter the group to receive it for free: scan the code to enter the group to receive information
I will regularly share the development of artificial intelligence, employment and related information with my friends in the group.
Finally, I wish you all progress every day!!