Reports from the Heart of the Machine
Machine Heart Editorial Department
Due to its popularity, the book was published in English, Chinese Simplified, and Chinese Traditional. The breadth of its coverage impressed Kwok Yi Ke, a professor at Imperial College and vice-chancellor of Hong Kong Baptist University. Today, the Chinese simplified version of the book is officially available for download.
Thanks to the success stories of DeepMind AlphaGo and OpenAI Five, deep reinforcement learning has received a lot of attention, and related technologies are widely used in different fields. However, for a learner, there are very few books or tutorials on the market that cover both "0 to 1" and "from 1 to N" deep reinforcement learning content, and the learning material is very fragmented.
To overcome this challenge, Dr. Hao Dong, an assistant professor at Peking University's Center for Frontier Computing Research, and others wrote an English book called Deep Reinforcement Learning: Foundamentals, Research and Applications. Released by Springer in June 2020, the book has sold well in both the print and electronic editions, with more than 80,000 downloads of the electronic version.
Due to the enthusiastic response, the author team set up a reader exchange group, many readers expressed their hope for a Chinese (simplified) version, and students in Hong Kong and Taiwan reported that they hoped to have a traditional chinese version. So they translated the book into Chinese Simplified and Traditional, and released it in June 2021 and January 2022. Among them, the first edition of Chinese Simplified Chinese has been sold out once it has been released, and the second edition has been released.
Why is this book so popular? Let's take a look at its contents first.
The book is divided into three sections that cover everything needed to learn deep reinforcement learning.
The first part (the basic part) introduces the basics of reinforcement learning, commonly used deep reinforcement learning algorithms, and their implementation methods:
The second part introduces a selection of deep reinforcement learning research directions, which is of great significance to readers who wish to carry out related research.
In order to help readers understand the details of deep reinforcement learning more deeply and apply related technologies to practice, the third part carefully describes the implementation details of a large number of applications, such as robot learning to run, robotic arm control, playing Go, multi-agent platform, etc., and provides related open source code.
You can see that whether you're a computer science background, a student who wants to learn deep reinforcement learning from scratch and work on research topics and practical projects, or a software engineer who doesn't have a strong machine learning background but wants to quickly learn deep reinforcement learning and apply it to specific products, this book can help you.
Guo Yike, professor at Imperial College, founding director of the Institute for Data Science and vice-chancellor of Hong Kong Baptist University, said he was impressed by the scope of the book, saying, "This style of book is an excellent learning material for beginners and researchers." He added, "Embracing the open source community is an integral reason for the rapid development of deep learning. I'm glad that this book provides a lot of open source code."
Chen Baoquan, Distinguished Professor of Liberal Arts at Peking University and Executive Director of the Frontier Computing Research Center, also believes that "this book provides a reliable introduction to deep reinforcement learning content, bridging the gap between basic theory and practice, featuring detailed descriptions and algorithm implementations, and providing a large number of skills and quick reference tables."
Of course, such a good book is inseparable from a strong editorial team. The author team of the book is all front-line researchers and members of the open source community, using deep reinforcement learning to solve problems in different fields. Among them, Dong Hao, Ding Zihan and Tong Shanghang are also members of the editorial team.
A team of authors of Deep Reinforcement Learning: Fundamentals, Research, and Applications.
Hao Dong is an assistant professor and doctoral supervisor at the School of Computer Science and Frontier Computing Research Center of Peking University. Received his PhD from Imperial College London in autumn 2019. The research direction mainly involves computer vision and robotics, with the aim of reducing the data required to learn intelligent systems and achieving autonomous learning. Dedicated to promoting ARTIFICIAL intelligence technologies, he is the founder of TensorLayer, an open source framework for deep learning, and received the ACM MM 2017 Best Open Source Software of the Year Award. He holds First Class Postgraduate and First Class Undergraduate degrees from Imperial College London and the University of Central Lancashire, UK.
Ding Zihan is a Ph.D. from Princeton University. He received his master's degree from Imperial College London in 2019 and has worked at Borealis AI and Tencent Robotics X Labs in Canada. He studied at the University of Science and Technology of China with a double degree in physics and computer science. His research interests mainly involve reinforcement learning, robot control, and computer vision. He has published numerous papers in top journals and conferences such as ICRA, IROS, NeurIPS, AAAI, IJCAI, Physical Review, and is a contributor to open source code libraries such as TensorLayer-RLzoo, TensorLet, and Arena.
Shanghang Tong is an assistant professor and doctoral supervisor at the School of Computer Science at Peking University. He received his Ph.D. from Carnegie Mellon University in 2018 and joined the BAIR Lab at the University of California, Berkeley in 2020 as a postdoctoral fellow. His research interests are mainly open environment generalized machine learning theories and systems, and he has rich research experience in computer vision and reinforcement learning. He has published more than 30 papers in top AI journals and conferences, and applied for 5 U.S.-China patents. Winner of the AAAI '21 Best Paper Award, the 2018 "EECS Rising Star" in the United States, the Adobe Academic Partnership Fund, the Qualcomm Innovation Award nomination, etc.
The diversity of the team makes the style of the book more reader-friendly to readers in different fields, and supports and maintains the code base.
Commenting on the writing process of the book, the team said, "The workload of writing this book is huge, the authors are busy with work and study, they are writing this book part-time, and there are several supporting code bases, and it is not easy to complete it as planned." So we explored and adopted an open source model for writing, but it took more than a year."