This paper analyzes several important textbooks in different stages of artificial intelligence, as well as their co-evolution process with artificial intelligence as an independent discipline, and looks forward to the possible development direction of its textbooks with the maturity of artificial intelligence disciplines.
Keywords: artificial intelligence textbook subject evolution
The author of the earliest systematic textbook on artificial intelligence was probably Nils Nilsson. Already realizing in the early '60s that it was impossible to integrate the two lines of artificial intelligence, he first wrote Learning Machines: Foundations of Trainable Pattern-Classifying Systems, published in 1965. This book is also one of the earliest textbooks on neural networks; In 1971, he wrote the semiotic book Problem-Solving Methods in Artificial Intelligence. Interestingly, the years between the publication of these two books coincided with the turning point when artificial intelligence abandoned neural networks and moved towards heuristic programs. Several of his subsequent books belonged to the semiotics. Among them, Principles of Artificial Intelligence, published in 1980, was the most influential and was a standard textbook that made no mention of machine learning. This book was translated as Chinese by Professor Shi Chunyi of Tsinghua University in 1983 (Principles of Artificial Intelligence, 1984, Science Press), which is probably one of the earliest artificial intelligence textbooks introduced in mainland China. Artificial Intelligence: A New Synthesis, published in 1998, can be considered a revised version of the previous book, with 12 chapters, 4 of which deal with theorem proofs, which are actually boiled down to methods and their variants; Chapter 4 deals with agents; The remaining 4 chapters are about searching; All aspects are taken care of, it can be regarded as moderation.
Nelson left the Stanford Research Institute (SRI), where he had served for more than two decades, in 1985 to join Stanford as chair of the computer science department. In 1988, he collaborated with Genesereth of Stanford University on another, narrower textbook, Logical Foundations of Artificial Intelligence, focusing on logic-related topics. Geneceres has been pushing logic in the field of artificial intelligence, but he is a senior engineer rather than a professional logician, and his interest is to push the Knowledge Interchange Format (KIF) into a standard, and the book has a dedicated chapter "Meta-knowledge & Meta-reasoning", which is the theoretical basis of KIF. It also accumulated some experience for the standardization of the knowledge graph later. This book is a loose and incomplete textbook of first-order logic and theorem proofs, and its quality and impact are inferior to Chin-Liang Chang and Richard Char-Tung Lee's Symbolic Logic and Mechanical Theorem Proving, which unfortunately has some errors in Chapter 7, and the two authors have since stayed away from the field of machine theorem proofs. A very valuable revision has not been published. Then the wind of artificial intelligence changed, and Nelson had been writing a book called "Introduction to Machine Learning," which unfortunately did not finish, and he updated the manuscript on his Stanford University profile as he wrote, the last update in 2005, with two chapters on "reinforcement learning." These books reflect both his career and the evolution of the discipline of artificial intelligence: from machine learning to machine learning. His 2009 monograph on the history of artificial intelligence, The Quest for Artificial Intelligence: A History of Ideas and Achievements, is his reflection on the development of the entire AI discipline over decades, and he is a participant in the ups and downs of AI.
Marvin Minsky, one of the pioneers of artificial intelligence, disliked affairs management and quit his position as director of MIT's AI Lab early to hand over to his student, Patrick Winston. At a young age, Winston ran a heavily funded artificial intelligence laboratory with excellent administrative skills. Winston's Artificial Intelligence (1983, Science Press) was once the standard textbook, with its first edition published in 1977 and its third edition in 1992. After 20 years, but this is the era of the neural network school, this book naturally belongs to the symbolic school, although the content is constantly enriched, but the ideological line has not changed essentially.
Elaine Rich's Artificial Intelligence (1993, Xiaoyuan Press) has been one of the most popular textbooks in the United States since its first edition was published in 1983. New authors were added to the second edition in 1991 and the third edition in 2009. After graduating from Carnegie Mellon University, Ricky joined the faculty of the University of Texas at Austin. In the early 90s, the United States established the Microelectronics and Computer Consortium (MCC) in Austin in response to Japan's rise in semiconductors and artificial intelligence, and Ricky joined MCC as the head of the artificial intelligence project. After the dissolution of MCC in 1998, Ricky returned to Austin as a senior lecturer. The final chapter of the third edition of Artificial Intelligence turned out to be Prolog, and was subtitled "The Natural Language of Artificial Intelligence." One of MCC's missions is to confront Japan's fifth-generation computers, and Prolog is the foundation of fifth-generation computers, which shows the imprint of the times. The first two editions basically belong to the traditional symbolic school; But the third edition has two chapters on neural networks and one chapter on genetic algorithms, which is still balanced.
Eugene Charniak of Brown University and Drew McDermott of Yale University, who were Ph.D. classmates at MIT, co-authored Introduction to Artificial Intelligence, published in 1985, is also the standard textbook for semiotics. Charnik's research interest is natural language processing, and he has been deeply engaged in statistical analysis (parser). In 2019, he kept pace with the times and published his own book Introduction to Deep Learning (Chinese translation of "Introduction to Deep Learning", 2020, People's Posts and Telecommunications Press), which is project-driven, except for the first chapter, each chapter requires a project in Python, including a chapter dedicated to how to use TensorFlow. Although this book is more applicable, it is not rigorous, the mathematics used are clearly explained, and those with a certain programming foundation can be used as a primer, taking into account both theory and practice.
Matt Ginsberg's Essentials of Artificial Intelligence was also influential and was published in 1993. Somewhat strangely, the seventh chapter is titled "Predicate Logic" and the eighth chapter is titled "First-Order Logic". In the dictionary of logicians , predicate logic is first-order logic. But Ginsburg's use of "predicate logic" refers specifically to Prolog-type syntax, that is, first-order logic in which quantifiers have been eliminated, or "clause logic" in Prolog's parlance. In addition, the book has a chapter on "non-monotonic reasoning", an area that is not much of a concern these days.
The intersection of cognitive science and artificial intelligence is large, although the terminology is different. Steven Pinker is not only a cognitive psychologist, but also a public intellectual. He has a best-selling book, How the Mind Works, which translates formulas and algorithms from AI textbooks into psychological descriptions and metaphors. The chapter titles Chinese translations of this book are also translated into artificial intelligence terms, and if you only look at Chinese titles, it is not much different from standard artificial intelligence textbooks.
Russell and Norvig's book Artificial Intelligence: A Modern Approach is the most comprehensive and "modern" standard textbook, used by more than 1,300 schools worldwide. Novig, the second author, went on to run Google's search project for a long time, but never left teaching, claiming to have taught at least 160,000 students with the book. The book is also the thickest of its kind, with more than 1,100 pages, more than double the thickness of Ricky's latest edition. If divided by the number of pages, the traditional content in the book is about 600 pages, probabilistic reasoning is less than 200 pages, while the content related to deep learning and reinforcement learning is about 220 pages, and there are two chapters of philosophy and future prospects of 40 pages. The first edition of the book, published in 1992, contained almost no neural networks. When the second edition came out in 1995, although there were no neural networks, there was a chapter on reinforcement learning, and there was a unique evaluation of reinforcement learning, which reflected the author's taste. The book says: "It can be thought that reinforcement learning encompasses all artificial intelligence. "The fourth edition was published in 2021, almost 30 years after the first edition came out. Compared with the third edition published in 2012, the fourth edition added a chapter on "Deep Learning", and after the chapter on "Natural Language Processing", a chapter on "Deep Learning in Natural Language Processing" was added, focusing on the basics of large models. A reinforcement learning section has been added to the chapter on "Robotics". The chapter "Adversarial Search" includes a section on Monte Carlo trees, which is clearly influenced by AlphaGo. There were originally two chapters on planning, which are now combined into one, while probabilistic reasoning, which originally had two chapters, now has three chapters. Overall, the fourth edition is 9 years away from the third edition, and the structure has not changed significantly, after all, deep learning has blossomed everywhere when the third edition was published. In addition, like Chapter 20 "Knowledge in Learning", although it talks about machine learning, it focuses on explanation-based learning, which was also called induction in the early years, and is more logical content, and has little to do with deep learning and reinforcement learning today.
Table 1 summarizes the AI sub-disciplines, active years, and versions of related textbooks.
Is the textbook as thicker and more comprehensive as possible? The Feynman Lecture Notes on Physics came out in several different editions, taking the Millennium Edition as an example, with a total of more than 1,500 pages, but it was printed in three volumes, each with a different focus, on mechanics, electromagnetism, and quantum mechanics. The most popular textbook for physics is David Halliday and Robert Resnick's Fundamentals of Physics, first published in 1960 and eleventh by 2017, with long-lived authors dying in 2010 (age 94) and 2014 (age 91). The latter editions were all written by Jearl Walker, who is younger than them, and Volcker is also 78 years old this year (2023). The eleventh edition is close to 1500 pages, which can be described as "unbearably light". When the twelfth edition came out in 2021, it was also divided into two volumes, the upper volume covering mechanics and thermodynamics, and the lower volume covering electromagnetism, relativity, and quantum physics. Computer Architecture: A Quantitative Approach, another best-selling computer science textbook, was nearly 1,000 pages old when it came out in its sixth edition in 2017.
Russell and Novig are in the prime of life, and perhaps their book will reach its tenth edition, which will probably exceed 2,000 pages. According to the experience of physics textbooks, in order to become a classic, the discipline must first be in a steady state; Second, books are widely adopted. The most promising artificial intelligence textbook is Russell and Novig's one, but artificial intelligence as a discipline is still developing, and we are not even sure that the content in the book will be in the same discipline in the future. In addition, a mature discipline takes longer to write the content of front-line research into textbooks; Compared with today's physics, artificial intelligence is a rapidly developing discipline, and its textbooks reflect cutting-edge research in a more timely manner, even so, an important progress can take three to five years to appear in textbooks at the earliest. Each author has his own taste. As comprehensive as Russell and Novig's book, there are also trade-offs in the selection of materials. They didn't say a word about the semantic Web — at the 2006 AAAI conference, Novig complained about the progress of the semantic web and disagreed with advocate Tim Berners-Lee. There is also no unity within the logicists.
There is no logical relationship between the sections of artificial intelligence, and whether throwing everything into a vat is a good textbook organization is worth exploring. Will AI, like physics, also split into two or three before textbooks become too thick? "Deep Learning," one of the authors of ACM Turing Award winner Yoshua Bengio, is the standard textbook for deep learning. Barto and Sutton's classic Reinforcement Learning was first published in 1998, and a second edition came out 20 years later. It is worth pointing out that reinforcement learning, as an independent field, only gradually became popular after AlphaGo became famous in 2016, and the indomitable persistence of the two authors made reinforcement learning obsolete.
The writing of artificial intelligence textbooks requires new thinking, and may need to add some theoretical basis as a glue. Turing's seminal 1950 article on artificial intelligence, Computing Machinery and Intelligence, raised questions based on his pioneering work in 1936 in computational theory. Whether layman or insider, the most frequently asked question is is intelligence equivalent to calculation? Neither the semiotical nor the deep learning school has an answer to this, or even a framework for answering. Among the practitioners of deep learning and reinforcement learning, there are no fewer people with electronic engineering backgrounds (including control engineering, systems engineering, etc.) than people with computer science backgrounds, and almost none of the former are familiar with computational theory, or even know the Church-Turing thesis, which certainly does not help to provide a systematic theoretical basis. Russell and Novig's book borrows the vocabulary of economics in the final chapter. Neoclassical economics assumes "perfect rationality", and there is a counterpart in the book "calculative rationality"; In economics, there is Herbert A. Simon's "bounded rationality" and the book counterpart "bounded optimality." From the perspective of computational theory, the so-called "computational rationality" can refer to computability in computational theory; For "bounded optimization", you can refer to computational complexity. Corresponding to the "rational man" in economics, the agent in artificial intelligence can refer to the Turing machine with a basis.
American academics have a tradition of compiling manuals. Chinese manuals are mostly reference books, such as Handbook of Mathematics, while American manuals are mostly collections of long review articles, such as Handbook of Mathematical Logic (published in 1977) and Handbook of Automated Reasoning (published in 2001). In the early 80s, Feigenbaum, one of the leaders of artificial intelligence, led the editing of the three-volume Handbook of Artificial Intelligence, and in 1989 added a fourth volume, totaling more than 2,000 pages. The book does not systematically talk about neural networks, except for 4 pages in the D2 section of Chapter 9 of Volume III, "Learning and Inductive Reasoning," as a historical review, mentioning Rosenblatt and his perceptron, as well as Minsky and Papert's 1969 book Perceptual Machines, which gives "theoretical limitations" of neural networks. Today, at least more than half of the manual is outdated. From the perspective of a more mature discipline, the turn of artificial intelligence textbooks is incredible, and it is impossible to imagine that a physics textbook will focus on relativity or quantum mechanics, sometimes shunning. Recently, the two routes of artificial intelligence have also begun to intersect. For example, the method of reinforcement learning is used to solve the problem of logic, which brings some light to the theorem proof, which has almost stalled. At the same time, theorists began to explore the boundaries of various learning problems.
Textbooks should not only be followers of academic progress, but also the knowledge combers of the latter, providing students with a systematic and economical learning path. The authors of the textbooks should preferably be senior front-line researchers, or what Yau Chengtong calls "newly retired front-line researchers", and their macro taste can help future scholars see the scenery of the discipline.
Nick
He is a CCF professional member and a guest columnist for CCCF. Chairman of Wuzhen Think Tank. In his early years, he worked at Harvard University and Hewlett-Packard, and then started a business, and won the Wu Wenjun Artificial Intelligence Science and Technology Progress Award. His Chinese books include A Brief History of Artificial Intelligence, Anatomy of the UNIX Kernel, and Philosophical Book Reviews.
Special statement: The Chinese Computer Federation (CCF) owns all the copyrights of the content published in the "Newsletter of the Chinese Computer Society (CCCF), and without the permission of CCF, the text and photos of this journal shall not be reproduced, otherwise it will be regarded as infringement. For infringement, CCF will pursue its legal responsibility
CCF recommended
【Featured Articles】
- CCCF column | Moore's Law is dead, long live Moore's Law
- Nick: Lessons from fifth-generation computers
- CCCF column | 50 Years of Computational Complexity: Wang Hao and Computational Theory
- CCCF column | The boundaries of artificial intelligence
- 【CCCF column】The origin of artificial intelligence
- Zhang Zhihua: The development process and enlightenment of machine learning
- Zhou Zhihua: The exploration of Boosting learning theory - a story spanning 30 years