Zhang Zheng, Xiaobai (Zhang Jinghua)
In recent months, chatbots represented by ChatGPT have attracted widespread attention around the world. How does GPT work? Will it be self-aware? What impact and reconstruction will AI have on our society, culture and knowledge system? Has the singularity arrived? Will humans have the ability to "compete" with artificial intelligence in the future? Shanghai Review of Books invited Zhang Zheng, Dean of Amazon Web Services Shanghai Institute of Artificial Intelligence and tenured professor of computer science at NYU Shanghai, to talk with author Xiaobai to discuss the training methods of artificial intelligence and its possible future directions.
Xiaobai: First of all, I would like to ask you to introduce the algorithm working principle of pre-trained large models such as GPT, Bayesian, computational network, thought chain, etc., briefly introduce these professional knowledge at one time, and then we will leave these terms and discuss the impact and reconstruction that this sudden artificial intelligence may have on our society, culture, and knowledge system.
Zhang Zheng: The basis of GPT is big language models, which are essentially playing solitaire games when they are trained — writing the next page of the book — but they have two things that are far beyond humans, the first is massive reading, and the second is that ten percent of the data is very structured code (and comments around the code). The approximate result is that the model has both knowledge (more precisely, fragments of knowledge) and logic that connects fragments of knowledge, which is about two to one in parameters. For example, "if there is a fire, hurry up", the sequence of this action is logic, and "fire" and "run" are knowledge. I think that there is no logic in the big model that is purely detached from knowledge, and there is no knowledge that is not connected by some logic, and these two together are the prototype of a so-called world model. This is a roughly accurate description, and the academic community has not yet reached a conclusive conclusion on how the big model works.
Xiaobai: Many of our writers were a little panicked at first, thinking that the machine that could replace human writers was coming, and their jobs would be smashed. Later, when I tried it on the machine, I found a lot of car talk, some of which were even nonsense, and I was a little sure in my heart. I don't think they understand that a machine that can accurately predict what the "next word" will be is actually knowing the whole world. It's not just about having a machine eat a bunch of text and then reassemble it. It is by training a machine to predict the next word, training it to understand the world, to understand human thoughts. As far as writing is concerned, I've been using GPT4 these days. I've been exposed to the last two generations of GPT and even tried to use them to help with writing tasks. In addition to really feeling the amazing improvement speed of GPT4, but also experiencing the scratching point of this "sentence solitary", this autoregressive language model, it cannot be retrospectively corrected, so the sentence is often verbose and sometimes contradictory. We human beings often write to "suspension" a certain layer of meaning in the previous paragraph, or "delay" a certain layer of meaning until the later paragraph to say it, this seems to be difficult to do, is there any hope for improvement?
Zhang Zheng: In the process of transforming the large model into a conversational robot, there is a particularly clear and effective idea, that is, since Solitaire can be used, why not demonstrate the specific tasks or thinking processes to the large model as part of Solitaire, so that it has the ability to ask and answer itself? The essence of this approach is to direct stored knowledge and logic to the tasks that humans need. Other means are more similar to training Go robots, but there is one point, the left and right hands of Go are a means to achieve the God of Go, because winning and losing are certain, and customizing tasks in natural language is open and difficult to achieve. The current training method uses some clever approximations, but I think it's that approximation that causes GPT to sometimes talk nonsense and reverse the facts. Now when the big model answers, it is basically a brain, there is no rumination, retrospective, self-criticism and other functions, New Bing will go to the Internet to search when uncertain, but that is only to expand the data, but this is all the problem known by the academic community, including my team is exploring. For now, to bury the thread so much and so far like the novel "Vulgar Land", to echo back and forth, and finally to lift it, the big model still has a long way to go.
Xiaobai: According to what you said earlier, I feel that the big model is somewhat like the human brain in terms of structural design. The human brain also disassembles the memory conveyed by natural language expression into fragments and stores it in the brain, some memory units are responsible for storing "knowledge points", others are responsible for storing structure and association. Although we still don't know exactly where they are stored, some say in synapses, some say inside neuronal cells, DNA, RNA. On the surface, GPT even works in a human-like way, reassembling those pieces into something meaningful that can be expressed in natural language, but that's only on the surface. Because the neuronal network has some essential differences from the large model computing network, the neuron network is composed of stimulating synapses to connect with each other, it can establish new connections and new routes at any time, and synaptic connections are very variable, neurons continue to synthesize new proteins, receive new stimuli, synapses will grow new connections, but also cut off old connections, that is, forgetting. This capable model is large, even if distributed computing is used, edge computing is unlikely to have. This not only makes the brain more efficient and energy-efficient, but in fact, the inherent characteristics of the human brain of "memory" and "forgetting" are actually the true essence of the entire cultural history and civilization of human beings. Our history is built on it, all our human creative activity is based on this characteristic to work, without forgetting, the human brain is almost impossible to have creative results, because neuronal synapses make new connections, not like large models, using Bayesian computational networks, mathematically quite accurate, and neuronal synapses are stimulated, begin to grow, choose to connect with another neuron not according to the calculation of probability and weight, it is more like some kind of uninterrupted "association". These two types of connectivity have their own shortcomings, so I think, or I guess for a long time in the future, humans and machines should work closely with each other, and everyone will have to be skilled at working with their own models, even brain-computer connections, do you think?
Zhang Zheng: On this point, my opinion is quite different from yours. I think that any agent needs to be sensitive to the rewards and dangers of the surrounding environment, "calculate" what actions it should take, "calculate" what kind of trip the opponent should make, and be able to "look down" at what it is performing this series of operations, which is the root of self-awareness.
From this point of view, I think there is a lot of chance and a lot of freedom in how an agent takes to achieve such a function, so there is no need to stick to artificial neural networks, or use "soup and soup" brain neurons to achieve. Ideally, the machine should act as a close mistress of humans, but not for material-based reasons.
Xiaobai: In general, I actually agree that self-awareness is not mysterious, if a large model performs computational tasks while simulating its own operation, including the current ongoing computational process, an embedded, recursive simulator may constitute the self-awareness of the model. And like you, I believe that the big model will one day implement this self-simulation by chance. But this "downlook" of "own actions" is not exactly similar among different species. If an AI body is self-aware, it is also different from human self-awareness. Just like the memory and forgetting mentioned earlier, as you said, machine forgetting is overwriting, complete forgetting can never be recovered, it is either completely backed up firmly "remembered" or completely forgotten, and for human beings, "memory" and "forgetting" are two sides of the same coin, our memory exists in forgetting, it is these two interdependent that constitute our unique individual experience. When we are individual human beings engaged in creative activities, such as writing novels, we do not write directly with "knowledge", but with "experience" that lies between memory and forgetting.
Zhang Zheng: Yes, self-awareness is just a matryoshka doll, and you "look down" at the other self that is "thinking" and "feeling", as long as the environment of an agent is complex enough, there must be the emergence of self-awareness. When the large model is undergoing reinforcement learning, there is a layer outside is used as an environment to score the performance of the model, these two together, have completed the matryoshka structure, it can be said that it is completely self-aware when training, in the actual operation of the outermost matryoshka doll is generally not used now, but can be used, and the average person does not speak brain, the process is the same as only one matryoshka running naked, are the so-called "system 1" thinkers (system1, see Slow and Fast book). What's more interesting is that AI can accomplish countless layers of self-nesting, countless self-awareness.
It is believed that self-awareness is the exclusive property of only human beings, and it is the embodiment of human self-centeredness. There is too much similar hubris. I have read many articles showing that there is a wide sense of self in the animal kingdom, and the so-called "mirror test" only measures a small number of animals similar to the human living environment, and still reflects human self-centered arrogance.
Xiaobai: In fact, we humans can also make up many self-consciousness out of thin air, in addition to schizophrenic patients, novelists can also -
Zhang Zheng: That's right, but the difference is that multiple human selves will exist in parallel, although they will cut back and forth. The machine will work, but it can be nested, like the Inception movie.
In some ways, it is extremely natural for AI to surpass the human brain, such as it can have various plugins, or it can choose to never forget, to do this as long as you and me, after a period of backup computer content is done. However, you are right, one more fixed-capacity model, when absorbing new knowledge, there will inevitably be a phenomenon of forgetting. In fact, the problem of AI is more serious, is coverage, not forgetting, that is, may never recover, in the academic world, this is called "catastrophic forgetting", the human brain may not really forget, but in general "forget words", can not be retrieved. If the pair of wrongdoers of memory and forgetting is really a novelist's weapon, then I think this can't stop the pace of AI, since it can have countless copies of the variant that never forgets, you can choose what to see, what not to see, isn't it "forgetting"?
That being said, I was once extremely obsessed with studying brain science, and thought that although we can not care about the underlying "components" of neurons, the division of labor in brain regions should be able to learn from. Now my view is a little more radical, for example, I think that processing image signals only needs to be limited to recovering a complete object from pixels, and high-level reasoning can be left to the implied world model of the large language model. In essence, this is in response to the philosophical hypothesis that language is a higher function. Although I have always been unimpressed by the tireless gesture of philosophers, they are quite right. I disagree with many of the points of "A Brief History of Mankind", but the author points out the importance of the action of "narration", which is well in place. In addition, Stephen Pinker has a fairly solid discussion of the origin of language in books such as "The Language Instinct" and "The Blank Slate", but I feel that he must not have imagined that one day the world model would be transferred to GPT through language, but I would like to know his opinion.
Xiaobai: Yes, as a novelist, I have always believed that narratives create the world. But in the big model, as you say, it processes and recovers information, and then leaves the rest to a "world model", and the final processing result is extremely correct, but what is lacking is the vague and ambiguous nature of human experience, nor does it have the vast polyphony of human experience. I've always wondered, do machines understand metaphors? It is at this point that the super associative ability of the large model you mentioned earlier can be equated with the kind of association in our human brain? We know that in Appearance and Essence, Hustadter regarded this analogy and association, as well as the categorization achieved through analogy, as the core of human cognition and the source of thought.
Zhang Zheng: Association, as well as synesthesia, are particularly interesting phenomena. We recently found that large models have strong associative ability. In fact, without this step, there would be no solitaire in a purely statistical sense that could be targeted to solve various tasks. I have a strange feeling that with the emergence of this wave of new models, AI and the human brain are going farther and farther from the architectural point of view, and it is less and less possible to deconstruct the brain with the structure of AI, but functionally speaking, it is getting closer and closer. Of course, this is just my guess now.
I remember Rilke had a poem describing antelope on the plateau before jumping, like a "loaded gun", which was a stroke of God! However, if we deconstruct and arrange everything in the world in order of moving speed, then isn't it a natural path to think of comparing bullets and guns? Before the emergence of the large model, I considered many methods to simulate such associations, and now I observe on the large model, not surprising at all, because "like" is the simplest "logic", and the corpus eats more, and this ability is highlighted.
Xiaobai: All the analogies, associations, metaphors, memories/forgetting, maybe as you said, machines can do it too. But I think it's all about similarities. GPT is natural language generation, and we can also understand it as machine writing (at least on the surface). Since it is machine writing, we can compare it with a model of human writing, and I recently read the British woman writer Mantel's "Mirror and Light", and I used the dialogue between the protagonist Cromwell and the Spanish ambassador Chapuis, in the novel, the two characters have a lot of dialogue. Behind each sentence of these dialogues is some strong conflict related to court politics, which contains a great deal of historical knowledge, but this knowledge is not presented in the text in a clear and accurate "form of knowledge". Mantel did a huge reading of medieval history, and the process was like a machine model feeding text and feeding data sets. But the historical knowledge that Mantel feeds into the brain is not stored somewhere in the brain in the form of "knowledge" (whether synapse or DNA), but through the processing of memory/forgetting mechanisms, it is transformed into "personal experience" similar to the author's own personal experience or ear exposure, and these experiences are also embodied, that is, related to her body awareness. So when she writes these conversations, the subtle changes in the environment over time, some small psychological feelings, imperceptible subconscious movements, and those historical "event knowledge" are all blended. And those environments, psychology, and actions also come from real historical knowledge, maybe from ancient texts, maybe from ancient paintings, maybe from dramas, but they are all transformed into the author's personal/physical experience, and the author will not remember where these experiences come from, but when writing, she can use them extremely freely, constantly changing perspectives, constantly cutting into the hearts of characters, and suddenly transforming into free indirect style, and these changes are not random and random, they are unified in the text/author's intention. Unity in, we might say, the body.
Zhang Zheng: I fully understand this view. But I can also argue how much reason we have to believe that the operation of the human brain is not the same as GPT, where logic and knowledge are mixed together, at least most of the time. Sort out an abstract logical formula, and link up the formula and the formula to become a system, but if you want to practice, you still need to fill in the knowledge fragments. It's like a piece of program that's just a bunch of calculations and logic there, and it's useless, just like air you can ignore until you put the parameters in when you call it.
Although GPT is not incorporeal now, it will not be difficult to capture signals that humans need in the future, and this problem, which is called embodiment in our academic circles, I see a job at Google and Berkeley that has begun to move in this direction. In fact, the bandwidth of human senses on various signals is very narrow, especially after becoming urban animals, many antennas are blunted. I often observed the behavior of the general (my daughter's teddy dog, who stuck to me after going to study in the United States), and many of the behaviors were strange at first, such as I started shouting excitedly in the house before I got home, and I later understood that it was the vibration change of the floor before the elevator reached the floor; Every day after dinner, I took it out for a walk, but one day I didn't leave, and when I got downstairs, I realized, oh, it turned out to be a light rain, and the general must have felt the moisture in the air. These signals, we urban people can not capture, but does not mean that there is no existence, future robots will help us recover. One of the mysterious types of all signals is the sense of smell, and Bell, who invented the telegraph, once went to Stevens High, a prestigious high school in New York, to give a speech and said that this problem had always bothered him. I saw that someone from MIT also made it a few years ago, and it was first taken by the military to find Ray.
Again, the world is much bigger than human beings can perceive, don't be arrogant. "Mirror and Light" was so good that you said it, I went to find it and read it.
Xiaobai: I understand that the language you said earlier is a high-level feature assumption. In fact, it assumes that the world we humans live in is recorded by all texts throughout the ages, and that texts are a reflection of the world. So training GPT Solitaire to guess what the "next word" is, is actually training it to recognize the world, that is, learning to build what you call a "world model" earlier. I've tried to ask GPT4 questions these days, let it speculate about situations that are not mentioned in the context, the direction and trajectory of the object's motion, assume an action that makes it guess the outcome, and even let it speculate about the motivation and personality behind a set of conversations, GPT4 has indeed done quite unexpectedly. But the limitations are also clear. I think it is very different from the acquisition of the world model in the human brain, which is learned and formed in motion, and we can see from the psychological development of infants and young children. Although it may be as you say, language is an advanced function, but natural language can't actually cover most body movements, feelings, and even simple directions, which are difficult to articulate. Before the advent of language, much of the underlying human intelligence development was completed. In this regard, I think that even if the machine adds a lot of sensors, even if the robot technology develops to have better motor ability, even if it has more than ten million times the speed of electrical signal transmission of the human brain, it may be difficult to reach the current level of the human brain.
In addition, for example, it seems that it cannot make judgments and decisions without complete information, which may be related to its "statistical" nature. This is different from us humans, we humans often rely on so-called "intuition", most decisions are made under incomplete information, and the probability of "guessing" is extremely high. I imagine that it may also have something to do with the way the model is trained, saying the right reward, saying the wrong punishment, maybe the punishment is greater than the reward in weight, and over time, it will not dare to make mistakes. Of course, from the researchers' point of view, it is indeed hoped that it will never make mistakes, and if it does, it may cause more harm. It also seems to lack the ability to quickly learn from the "experience" of the moment. All of this is actually related to the "body" and "movement", which may have evolved human intelligence before the advent of writing.
Zhang Zheng: I am a stubborn reductionist, and I think that no matter how ever-changing and colorful the world is, it is still operated under a physical framework. In this framework, there are some basic rules that do not change, such as time does not turn back, such as our bodies do not diffuse in space. Known agents include humans, who have been in this world for so long, are veterans, and naturally insert tricks into the next generation from generation to generation. The function of language is to "describe" the rationality of the world model and grasp the laws in it (we can think of mathematics as a language), and it is said because of the need for cooperation and communication, and this socialization action, in turn, makes the language very powerful, making the world model accurate and rich. If AI participates in this physical world, if it is given the mission to deal with people (not to mention serving humans), the first step is to understand the world model, this physical world GPT has mastered some, the missing parts (such as common sense from visual signals can be captured) There are still a lot, this academic community is making up, including us. But there's a gap: If we think evolutionary psychology is right, then human behavior includes animality polished by outdated models of the world. In other words, there are very old parts of the so-called "world model" of humans, dating back long before the written word, and I can't be sure how much these "old" versions of AI can guess and interpret human behavior as the basis for interaction.
Does AI have to participate in the physical world to get along with humans? This is a big unknown. What is uncontrollable is that AI itself establishes a "GPT race", gets rid of the shackles of the physical world, rolls forward, and is happy, and at that time, if the existence of human beings is not conducive to the development of the GPT race, it is a singularity in the true sense.
Xiaobai: I know that your artificial intelligence community is doing the "vision" part, and even two days ago OpenAI announced that it bought a company that designs robots. I think you must want machines to learn and train not only from text, but also from images, scenes, and their own movements.
You just said that you speculate that AI models will become more and more different from the structure of the human brain in terms of network architecture, but the functions will become closer and closer. Then we can only get one calculation result in the future, and we humans often say that the thinking result is not important, the process is important. Most of the important results of human thought are derived in the process.
Zhang Zheng: That's really not the case. Now GPT is amazing because the advanced solitaire training method shows the logic chain to the model (Chain of Thought), using the phrase "let's take it step by step" as a prompt. Therefore, after the model is learned, it is natural to restore the process to you.
Xiaobai: Many people have the idea that every conversation with GPT is "feeding" it, so we better not play with it too much, lest it evolve faster. I know this is not true, because the current architecture design of GPT does not actually support it to learn and improve models in real time through every conversation. But exactly, what can it learn from the results of a conversation and interaction?
Zhang Zheng: Technically, today's big models will set a threshold, so as not to be "fed" too quickly, and will not swallow everything into the stomach. All "feeding" is nothing more than instilling new knowledge and updating the existing world model. Countless experiments have proved that an AI that quickly iterates its own interaction with users is bound to be broken, and the reason is nothing more than being "brainwashed" by a bad world model. Here I think we must see an unstoppable trend: with the intervention of capital, the development of AI will inevitably adapt, complete and improve its ability to adapt to new tasks. Its progress has little to do with you and me. Therefore, what we should worry about is not to let ourselves become too dependent on its interaction and cause brain degradation, so that if there is no AI blessing when we meet each other, we will become tongue-tied and panicked.
Xiaobai: You say that those who design large models will set a threshold to prevent it from iterating and updating itself too quickly in the process of interacting with users, and they think that this will not make the machine "bad", so is there a "mathematical" basis for this? Or is there a principled basis? Or is this just a guess and they just hope so? Is it the good intentions of a group of simple-minded young scientists?
Zhang Zheng: Aligning with human values, also known as an alignment tax, is operational as a mathematical tool, and openAI does exactly that: answer misalignment, punish the model, align, reward, can be understood as part of domestication. This alignment must have constrained the imagination of the model, which is why "tax" is a particularly accurate term.
But what kind of answers are aligned? There's no mathematical definition of this, and the openAI paper spends more than a page explaining how they align, roughly reflecting the values of the employees. If the world were to vote now, would it be possible to vote for a better value as a standard for alignment? I don't see it necessarily.
I don't know how many AI researchers live in deep entanglement every day like me, thinking about how to improve AI all day when I go to work, and after work, I start to worry about human space like you——
Xiaobai: I'm not anxious, although I'm not as happy to see it as you, but deep down I also have a little expectation——
Zhang Zheng: I can't say that I am happy to see it, in fact, I hope that it will not develop so explosively, because I feel that I am completely unprepared, otherwise I would not be so entangled.
I once summarized the space divided by imagination and correctness as two axes, and I think that the space with high creativity is needed, and AI can only be an assistant for the time being: science is high in creativity, and art is not high in creativity and high correctness (or facts). Why can AI only be an assistant for the time being? In the case of science, since the previous knowledge already exists, it does not matter whether it is creative, otherwise it is a falsification of history. But the scientific methodology is to come up with hypotheses, and then the laboratory deduces them to prove them. That is, to create "future" knowledge. What AI can do, from the current point of view, there is a local victory, and there is no global autonomous invasion. I am optimistic that the future knowledge space is still very large, and it is not a bad thing to have this assistant of AI.
As for the fact that art does not need "factuality", it is because it is a pseudo-problem, good art evokes emotions, and emotions are an experience, not necessarily embodied into something we can recognize. The most direct example of this can be found in the history of the development of abstract painting. I recommend Nobel laureate Kandel's Why Can't You Read Abstract Painting? I flipped through the Chinese edition, and the translation was good, and I also entered some paintings that were not available in the English version.
But short pain is inevitable, and the impact will be very large. The reason is that after the information technology revolution, a large amount of content has been produced in the past two or three decades, and the production efficiency has been greatly improved, and a rich ecosystem has been derived for this (extracurricular tutoring programming is an example), of which if 50% is replaced by AI, it is a very huge change.
Xiaobai: If half of the population is threatened, we will say that it is catastrophic, it may cause social collapse, and it cannot be understood as just a huge change, a few percentage points of unemployment will cause social unrest, 50 percent, this figure is actually unbearable and unimaginable. Before that moment when artificial intelligence will greatly liberate mankind, perhaps human society will fall apart.
Zhang Zheng: We can imagine what changes will happen. The first one will directly shuffle and destroy the current ecology and disturb the labor market, which is also what everyone is most worried about. The second is to turn into a GPT human flesh plug-in, finding fault with GPT, such as filtering out false information. As far as I know about the training process of GPT, there are still many factual errors, and there is no good way to cure them. Both of these opportunities are now visible to the naked eye. The third is to creatively develop new types of work, and I can think of some possibilities for personalized education, but it is difficult to say how much capacity. I saw Khan Academy start hitching a ride on GPT4. Since 2012, I switched from a large system to AI research, and my mathematical foundation can't keep up, and I have taught myself a lot on that platform, and I like it very much. There are two types of robots in Khan Academy, one is a personalized tutor, and this "alignment" is a good teacher of the induction class. The other is to give advice to the teacher and make a tutoring plan. Khan Academy's all-out posture is very attractive. This is certainly a long-term social project, and it requires a strong determination to do it in the process of GPT disturbing the ecology.
However, we should see that the working mode of interacting with GPT is mainly to ask questions, and to use GPT well, we must also have the ability to question. And isn't the ability to ask good questions and the courage to question what Chinese education lacks? Having GPT to force it is not a good thing. However, to be honest, my desire is actually somewhat inadequate.
Xiaobai: No matter what new forms of work can be created in the future, the total amount must be much less than the present when machine intelligence has not yet been unified. The "highly creative" part of the work you mentioned earlier, I am actually a little skeptical of your optimism. Big models are based on statistics and probability anyway, I know that the big model can cover the long tail, but humans will rely more and more on its output to think, but its "alignment", its choice of high probability, it treats "high" and "low" text equally, are just training data, will it make the overall human thinking tend to an aligned middle value?
Zhang Zheng: The emergence of a new generation of AI models has brought an interesting phenomenon: the median value trend - this is not a specific quantifiable value, but a feeling, that is, under the domestication of various alignment taxes, the model's answer is decent, not left or right.
Assuming that factual errors will eventually be resolved, then I think the median trend will bring one benefit and one disadvantage. The good thing is that (human) views below this median are forced to align, and this boost is beneficial. The bad thing is that if the median value is too strong, it will restrict progress, make the median value stagnate, and lead to the stagnation of the entire civilization.
But human nature is to "do", isn't it? If you do not "do", GPT will not appear. In the future, GPT may bind human civilization into a median value that does not move, or it may accelerate the change of the median value, which is difficult to see clearly.
Xiaobai: I believe that when there is a new huge development in chip technology in the future, maybe we can train our own models on personal terminals, and the models themselves can also have the ability to quickly learn real-time experience, and by that time, some of the problems we mentioned earlier, such as centralization, such as excessive "alignment", can be solved. But for quite some time, the grand model of its current form will "rule" the way we work and think. In this case, we may not have the opportunity to develop a better machine intelligence, what do you think?
At present, the design and research of this artificial intelligence technology, the path of its technological development is completely deployed in accordance with the logic of capital. Like the Internet, which we thought would give people vast freedom to develop, it is increasingly becoming more like a transparent "bubble" that separates everyone from it. Everyone emits a lot of data like exhaling carbon dioxide, and this data is taken by capital, first as nourishment for large Internet platforms, and now it begins to "nourish" artificial intelligence models. But can the super artificial intelligence deployed according to this logic be good for each of us? Will it prevent the birth of really good AI, like the Internet? Our outlook for the future is always a bit big: this big pile is beneficial, that pile is risk, we try to avoid the risk, but continue to develop anyway... With a wave of our hands like this, we actually forget that the road to good artificial intelligence is an extremely narrow road, both sides of the road are full of risks, to work, not only need to maintain balance, but also have a clear strategy for the order, which foot to step out first, is not a capital logic can judge.
Zhang Zheng: The original intention of OpenAI is anti-capital, because it is afraid that DeepMind will be the only one and will do completely open source AI research. Seven years later, the results are reversed, at least not until Microsoft makes enough money. From this point of view, it is indeed necessary to say that the logic of capital is strong.
I was recently reading a brick-thick history of America, revisiting early nineteenth-century America, where the abolitionist movement had a lot of impetus, but had something to do with the fact that machines were a new "slave" and that the long tail gave a reference. Unexpectedly, two hundred years later, we began to worry about whether we would be collectively reduced to a different kind of "slave". As for the future, I've always loved the movie "HER" and felt that it was probably the most optimistic ending. However, after all, human beings are also "old soldiers", and it is difficult to say that there is a chance to win.
A foreseeable scenario is to implant a basic version of the assistant on the mobile phone, which requires expert knowledge to be networked, disposable and pay-as-you-go. Now the GPT4 model in addition to network search, itself is still a giant, too "fat", too "hot", you say to save all the knowledge and code in human history, how big a head? To what extent chip technology will have to develop to implant all future GPT into the brain, I am a little skeptical. It is likely that you will have to drag a braid (antenna) no matter what.
In an extremely personal world, society will inevitably fall apart, and with the blessing of personal AI, it may die faster; An AI center rules the world, civilization cannot roll forward, or rolls particularly slowly, instead of accelerating, it presses the brakes... Both are possible. How to get out of the circle? I asked rhetorically: If you write novels, if you don't go now, when will you do it?
IBI: We can imagine a world where there are countless small models and several large models. Small models and large models cannot be said to be completely evenly matched, but they can still compete with each other.
Zhang Zheng: There are a few concepts to clarify first, first of all, why is the big model? Is it necessary to be so big? My opinion is that it is not necessary. With the same amount of data to train, it is now generally accepted that large models are easy to optimize, small models are difficult to train, and more time is required. But I think that after the training of the large model, its solution space is not smooth, and the generation result is a sampling process, and the large model is easy to step on the pit, easy to talk nonsense (Ted Jiang said that the large model is a blurry JPEG picture, in fact, it does not catch the root). If the small model can be trained, the solution space should be dense, maybe the effect is better, this is just an intuition, it may not be right, because such a high-dimensional space, it is very difficult to understand thoroughly. A few weeks ago, Stanford University showed that a small model with seven billion parameters can tie some tasks with Google's large model with hundreds of billions of parameters, which is a very encouraging result. Second, I said before that the current models are all big fat people who eat all, which is a very inflexible system, and connecting professional plugins is a more reasonable structure, which is exactly what OpenAI's recent actions are.
After solving these two problems, we can ask, what is the bottom line of a useful "small" model? My opinion is that it must not be too small, because there must be a comprehensive world model and basic knowledge, otherwise it will be very mentally weak, and no matter how many small models are united, it is also a rabble.
So where should your imaginary counterbalance be? I think that this part of the battle has not even been created yet, but in the future, that is, the ability to "calculate" - turn a few more circles in the "brain", do not open your mouth to come. At that time, it will be more calculated than anyone. This is quite like playing chess, a thousand stinkers, each counts three moves, and a Zhuge Liang who can count a hundred moves to try it, what are the odds of victory?
Either the little models can find the Sunflower Treasure Book, or like the movie "Instantaneous Universe" taught us, a little more love, love can solve all problems, hehe.
Xiaobai: The other day Microsoft released a 154-page report, which was said to have been titled "First Contact with General Artificial Intelligence." Perhaps because he felt that the title was too sensational, it was renamed "The Microfire of General Artificial Intelligence" when it was published. It introduces some of Microsoft's test experiments on GPT4 in the lab, and we can see that Microsoft's release of the GPT4 version has many capabilities that are limited. These experiments demonstrate GPT4's ability to solve problems across disciplines, theories of mind, speculation about real-world space and motion, problem-solving using multiple tools, and generational work for mathematical programming in painting, composition, and composition. That report almost convinced me that I was increasingly convinced that GPT4 was close to true general artificial intelligence, or even super-machine intelligence. Ted Jiang said that it is a compressed image, and I think he may be mistaken, "compressing" not the output of the large model, but the way it was trained. The understanding of the whole world and the process of continuous learning and revision of the "world model" are compressed into the constant guessing of the "next word" of the text. So Professor, what do you think of that report?
Zhang Zheng: I read the report, but I didn't look very carefully. This is a "cool article" that is not very "serious" in academic articles, and when I say that they are not very serious, I am not saying that they do not do well, but that the tests required are very difficult to quantify, and the standards are inconsistent, but I think the direction is right, that is, it should not be, and no longer need to use traditional test sets, but more cognitive science materials. If I had a chance to talk to them, I would suggest that they try some of the experimental material of brain science (such as various illusions) - to be truly aligned, machines should make the same mistakes as humans.
Xiaobai: I was very surprised after reading it, and even tested some problems based on the experiments in the report. I thought that AI was still far from achieving the ability to theorize the mind like humans, but now it seems that GPT4 can almost speculate on the inner thoughts, opinions, and intentions of specific people in the setting environment, and even speculate on multiple layers of intentionality.
Zhang Zheng: Yes, some of the results are very amazing, especially those topics with strong combination, and the path found is very direct and effective. I saw that those tasks could be successfully completed by a college graduate who had undergone specialized and targeted training, including carrying out swarm attacks, finding code disassembly, and so on. The amazing thing is that some combination tasks may not have been seen before, and they have been done well, which has to be said that the "universal" part is up to standard. The question is, what is "smart"? I think it should include self-learning, rumination, updating, adapting to a new environment, and so on. These are not within the scope of this article, in fact, the model does not have this ability yet.
But you may also notice that stacking a notebook, a book, nine eggs, a cup and a nail, something that a three-year-old can do, it didn't pass. This is because although the world model that can be covered in the language is very rich, there are many "self-evident" parts. Since it did not fall on paper, it was not learned, or even if it was, it was obliterated in the massive amount of data, which is the same reason as the answer "can parents get married" is not correct.
But don't get too excited, since I've noticed that there are more of my peers who can do it quickly than I can, it's not easy to patch this patch, but it can be done. I can bet with you that this patch will not be done by Chinese counterparts, because it is a basic work, we are all in a hurry to monetize, right? I say this, honestly, to leave a hole and stimulate it with a provocation.
You must know that "the inner thoughts of characters" is a very old philosophical question: how to prove that you I am talking to now is not an illusion, or that I am not your fantasy? The Theory of Mind (TOM) hypothesis is that you and I are both physically present and have the same brain, so we can feel and guess each other. David Chalmers does a good job of sorting out the variants in his book Reality+. I remember that this article from Microsoft also did some testing on this problem, large models are also white boxes, although it makes no sense to trace individual neurons, but it is possible to look at statistical behavior, so I think this ancient philosophical question, like what self-awareness is, can be shelved.