Why is ChatGPT so powerful? What is the root cause behind this? It is important to emphasize that ChatGPT is just a machine, and they are fundamentally different from humans. While they can produce human-like output, it is not wise to anthropomorphize them.
Why is ChatGPT so good? While ChatGPT still has various issues, the capabilities it has demonstrated have far exceeded initial design expectations. Initially, it was designed to handle NLP tasks, i.e. using machines to process natural language. The so-called natural language refers to human language such as Chinese or English.
Scientists hope that such machines will be able to process human language in the same way they process programming languages. So, they studied NLP as an academic field and developed many technical routes. However, ChatGPT's route is relatively radical, adopting an end-to-end learning approach, in which neural networks can learn on their own from input to output without the need for humans to analyze grammar and knowledge structure. This means that humans no longer need to help machines analyze grammar and sentence patterns, or deal with complex knowledge structures. Since machines can learn deeply, let the machines solve their own problems from start to finish. However, this seemingly absurd method eventually became a breakthrough for NLP.
ChatGPT has performed brilliantly from the beginning, when it was just an experimental network with only 4,000 neurons, but it has been able to react to the positive and negative emotions hidden in a piece of text. This means that it has fully understood the shallow features in the text and begins to recognize the deeper hidden features in the text.
The so-called characteristics are actually the statistical patterns mentioned earlier. In fact, humans are very good at finding statistical patterns in all kinds of data. For example, babies identify patterns in the nature of language they see and hear, build their own knowledge of semantics, word order, grammar, and emotions, and gradually expand to more complex patterns.
While ChatGPT doesn't learn from humans and can learn patterns from a tiny amount of data like a baby does, it can still count and recognize patterns as long as it has seen enough text. However, the information in human text is hierarchical. Grammar and semantics must be understood before understanding the emotions in a text. This hierarchy is crucial for both humans and ChatGPT, both of which must be learned hierarchically. In fact, the entire pre-training process of ChatGPT is the process of gradually deepening the feature level, which is called hierarchical learning in terminology. This is one of the core capabilities of neural networks and the source of the term deep learning.
This hierarchical learning is very similar to the process by which individual humans accumulate knowledge. For example, as a person grows, his reading and social practice become more and more, the information and insight he can read from language becomes richer, and his judgment and prediction of things become more and more accurate. Although ChatGPT lacks a human perspective and can only learn from language, as its network expands, the level of features it can identify is getting deeper and deeper, from the most basic grammar and sentence patterns to mood style emotions, and then to the hidden intentions and preferences in the text.
A further analysis of cultural psychology reveals deeper sociological features such as values and ideologies. Just like humans, ChatGPT can also dig into the underlying information in the text of the language, from the most basic grammatical sentences to tone style sentiment, and then down to the implicit intentions and preferences in the text.
From a statistical point of view, learning these characteristics is not difficult. At least in everyday communication, ChatGPT has been able to accurately identify subtle features in text. However, as a text predictor, what ChatGPT needs to predict is a random piece of text written by anyone. Logically, in order to accurately predict a passage, the author of the passage must have knowledge.
For example, if ChatGPT doesn't understand the law at all, it can't accurately predict the defense written by a lawyer, and if it doesn't know programming at all, it can't accurately predict the code written by a programmer. Therefore, it requires digging deeper into the knowledge in the text, including all the scientific knowledge in human civilization. As a result, ChatGPT has gone far beyond NLP.
Initially, it was designed to predict the next word, but the potential it carries is enormous. What's more, what ChatGPT learns is not just rote memorization of knowledge items, but knowledge structure formed through continuous learning. Each layer builds on understanding the previous layer. Therefore, its body of knowledge is very solid. Although its knowledge structure is not as refined as that of humans, and there are some redundancies and flaws, it has exceeded all expectations, and even surpassed the knowledge map and knowledge base carefully drawn by scientists.
The knowledge it acquires is not just rote memorized items, but a knowledge structure formed through continuous learning, each layer based on understanding the previous layer. Therefore, its body of knowledge is very solid. Although its knowledge structure is not as refined as that of humans, and there are some redundancies and flaws, it has exceeded everyone's expectations, even beyond the knowledge map and knowledge base carefully drawn by scientists.
Although the entire body of human knowledge is too complex for ChatGPT to fully learn, it has far exceeded the expectations of everyone, including designers. No one imagined that this neural network would surpass the knowledge map and knowledge base carefully drawn by scientists just by relying on its humble statistical power, and truly touch the concept of general artificial intelligence. For the first time in the history of human engineering, the emergence of the micro level has been harnessed to go beyond the traditional top-down approach to design.
Although ChatGPT's network size and examples are huge, the key to its success lies in the magic of its neural network. It was able to exceed initial design expectations and ran so well that people had to use the word "faith" to express their awe and admiration. With continuous iteration and improvement, ChatGPT will eventually penetrate into our lives, not only being able to process and provide information, but also assisting us in organizing it more efficiently. This will allow us to truly appreciate the meaning of "everything is information".
This is the essence of ChatGPT, learn something new with me every day, follow me, see you in the next issue.