Large model agent technologies, such as OpenAI's GPTs and Stanford's Generative Agents AI Town, have demonstrated great potential to develop into general artificial intelligence by combining external capabilities and social division of labor on large models. This workshop will introduce the cutting-edge progress, unique characteristics and future trends of large-scale autonomous agents and swarm intelligence.
CCF Frontiers Workshop
CCF Frontiers Workshop
CCF Advanced Disciplines Lectures
CCF ADL Issue 146
Topic: Large Model Autonomous Agents and Swarm Intelligence
May 17-19, 2024 Beijing
In the CCF Frontier Workshop ADL146 "Large Model Autonomous Agents and Swarm Intelligence", we will introduce the basic technologies and latest progress of large model autonomous agents, covering from the basic theory and technical principles of a single agent, to the division of labor and cooperation mechanism between multiple agents, from task-oriented tool learning to simulation research based on multi-agent systems, and related extension problems. Through in-depth explanations, students can not only master the core knowledge and key technologies of large model autonomous agents and swarm intelligence, but also deeply understand the main challenges and many application scenarios they face. We believe that through the study of this ADL, students will be able to greatly broaden their scientific research horizons and significantly enhance their practical ability in the field of artificial intelligence.
In this ADL workshop, 7 experts and scholars from well-known universities at home and abroad who are active in cutting-edge fields were invited to give keynote speeches. On the first day, Liu Zhiyuan, Qin Yujia, and Lin Yankai will respectively explain the introduction of general artificial intelligence, agent autonomous memory, planning and decision-making technology, and tool learning technology. On the second day, Qian Chen and Chen Xu will introduce large-scale model swarm intelligence technology and large-scale model social and economic simulation technology. On the third day, Liang Yitao and Hu Di will introduce topics such as open-domain autonomous agents and robotics based on autonomous agents. Through the three-day teaching, it aims to lead students to realize in-depth learning and thinking about autonomous agents and swarm intelligence from basic technologies to cutting-edge dynamics to innovative application scenarios.
Academic Director: Liu Zhiyuan Associate Professor Tsinghua University/Lin Yankai Associate Assistant Professor Chinese Renmin University
Organizer: China Computer Federation
The theme of this issue of ADL "Large Model Autonomous Agents and Swarm Intelligence" is led by CCF senior member, associate professor Liu Zhiyuan of Tsinghua University and associate professor Lin Yankai of Chinese Renmin University as academic directors, and seven experts (in order of lecture) Liu Zhiyuan (Tsinghua University), Qin Yujia (Tsinghua University), Lin Yankai (Chinese Renmin University), Qian Chen (Tsinghua University), Chen Xu (Chinese Renmin University), Liang Yitao (Peking University), Hu Di (Chinese Renmin University) and other experts were invited to give special lectures.
Agenda:
| Friday, May 17, 2024 | |
| 09:00-09:10 | Opening ceremony |
| 09:10-09:20 | Overall Contrast |
| 09:20-10:00 | Lecture 1: Autonomous Agents and Swarm Intelligence Driven by Large Models: Embracing the Second Emergence of Intelligence Zhiyuan Liu, Associate Professor, Tsinghua University |
| 10:00-12:00 | Lecture 2: Autonomous Memory, Planning and Decision-making Techniques of Agents Yujia Qin, Ph.D. candidate, Tsinghua University |
| 12:00-13:00 | lunch |
| 13:00-16:00 | Lecture 3: Learning Large Model Tools Yankai Lin, Assistant Professor, University of China |
| Saturday, May 18, 2024 | |
| 09:00-12:00 | Lecture 4: Large-scale model swarm intelligence technology Chen Qian, Assistant Researcher, Tsinghua University |
| 12:00-13:00 | lunch |
| 13:00-16:00 | Lecture 5: Social and Economic Simulations Based on Autonomous Agents Xu Chen, Associate Professor, Renmin University of Chinese |
| Sunday, May 19, 2024 | |
| 09:00-12:00 | Workshop 6: General Agents in the Open World Yitao Liang, Assistant Professor, Peking University |
| 12:00-13:00 | lunch |
| 13:00-16:00 | Lecture 7: Autonomous Agent-based Robotics Hu Di, Associate Professor, Chinese Minmin University |
Invited Speakers (in order of lecture)
Liu Zhiyuan
Associate Professor, Tsinghua University
About the speaker: Zhiyuan Liu is an associate professor and doctoral supervisor of the Department of Computer Science at Tsinghua University, and a senior member of the China Computer Federation. His research interests include natural language processing, knowledge graphs, and social computing. He received his Ph.D. degree from Tsinghua University in 2011 and has published more than 100 papers in well-known international journals and conferences in the field of artificial intelligence, such as ACL, EMNLP, IJCAI, AAI, etc., with more than 42,000 citations by Google Scholar. He has won the first prize of Natural Science of the Ministry of Education in 2020 and 2022 (the second completer), the first prize of the Qian Weichang Chinese Information Processing Science and Technology Award of the Chinese Chinese Information Processing Science and Technology Society (the second completer), the Hanwang Youth Innovation Award of the Chinese Chinese Information Processing Society, and has been selected as the National Young Talent, the 2020-2022 Elsevier China Highly Cited Scholar, the MIT Technology Review China 35 Under 35 Science and Technology Innovation List, and the China Association for Science and Technology Young Talent Lifting Project.
Title: Autonomous Agents and Swarm Intelligence Driven by Large Models: Embracing the Second Emergence of Intelligence
This report first introduces the development process of artificial intelligence, clarifies the historical status of large model technology, and then introduces the characteristics of large model intelligence, discusses the development trend and technical dynamics of large models in tool intelligence, autonomous agents, and swarm intelligence, and looks forward to the second emergence of intelligence.
Qin Yujia
Ph.D. candidate, Tsinghua University
About the speaker: Qin Yujia is a 2016 undergraduate degree in the Department of Electronic Engineering and a 2020 Ph.D. in the Department of Computer Science at Tsinghua University. The research directions are: (1) efficient pre-training of large models, that is, using less computing power to train large models with stronger effects, (2) efficient fine-tuning of large models, that is, fine-tuning the performance of fine-tuning with very few parameters to achieve full-parameter fine-tuning, saving a lot of video memory, and (3) tool learning, that is, teaching large models to flexibly use external tools like humans to expand their own capabilities. During his Ph.D., he published 15 one/total AI top conference papers (ICLR, NeurIPS, Nature sub-journal, ACL, NAACL, EMNLP, TMLR, TASLP, etc.). Google Scholar has 1400+ citations, published multiple open source projects (XAgent, ToolBench, BMTools, WebCPM, etc.), and has a total of 20000+ GitHub stars. He has won the Baidu Scholarship, the Tencent Rhino Bird First Prize Scholarship, the Outstanding Youth Paper Award of the World Artificial Intelligence Conference, and the Best Paper of the China Computing Conference.
Title: Autonomous Memory, Planning and Decision-making Techniques for Agents
Abstract: In recent years, large models have shown amazing application value in many fields such as natural language processing, computer vision, and biology, and the agents supported by large models can solve complex problems by invoking tools and making multi-step sequence decisions on the premise of understanding user needs. Among them, long-term memory ability, reasoning planning ability and multi-step decision-making ability have a great impact on the task execution performance of agents. In this report, we will first briefly review the development of large model-based autonomous agents, and focus on the latest cutting-edge research in the above three areas. In addition, based on the above discussions, we will look forward to the future development direction of agent technology.
Lin Yankai
Prospective Assistant Professor at Chinese People's University
About the speaker: Yankai Lin, an associate assistant professor at the Hillhouse School of Artificial Intelligence, Renmin University of Chinese, whose main research direction is pre-trained models and large model agents, has published more than 50 papers in CCFA/B international top academic conferences, with 11,938 Google Scholar citations (until February 2024) and an H-index of 41, and has been selected as an Elsevier China Highly Cited Researcher for three consecutive years from 2020 to 2022. His achievements were awarded the first prize of the Natural Science Award of the Ministry of Education (the third person to complete it) and the leading scientific and technological achievements of the 2022 World Internet Conference (a total of 15 projects in the world). In terms of knowledge representation, his TransR paper was listed as a representative method of knowledge representation by Yoshua Bengio in his "Deep Learning" textbook, and the related work results OpenKE and OpenNRE have received more than 7,800 stars on Github, the world's most influential open source platform. In terms of large-model agents, he presided over the release of the world's first large-scale tool learning large-scale model ToolLLM, presided over the release of the large-scale model autonomous agent system XAgent, which has obtained more than 6,900 stars on the open-source platform Github, and built RecAgent, a multi-agent simulation platform for simulating user behavior, which is the first multi-agent platform for simulating user behavior at home and abroad.
Title: Learning from Large Model Tools
Abstract: In recent years, large models have shown amazing application value in many fields such as natural language processing, computer vision, and biology. The extraordinary understanding, reasoning, planning, and decision-making capabilities of large models in complex interactive environments obtained in large-scale pre-training show great potential for invoking tools to solve complex tasks in complex real-world scenarios. This report is about learning about the Big Model Tool, how Big Models can understand and use various tools to accomplish their tasks, including their unified framework, key challenges, key work, and future directions.
Qian Chen
Assistant Professor, Tsinghua University
About the speaker: Ph.D. from the School of Software, Tsinghua University, currently working as a postdoctoral fellow at the Natural Language Processing Laboratory (THUNLP) of Tsinghua University, a Mizuki Scholar of Tsinghua University, whose main research interests are pre-trained models, autonomous agents, and swarm intelligence, and his co-supervisors are Prof. Maosong Sun and Zhiyuan Liu, who have published several papers as the first author in international academic conferences or journals related to artificial intelligence, information management, and software engineering such as ACL, SIGIR, AAAI, and CIKM. In terms of swarm intelligence, he led the release of ChatDev, a multi-agent software development framework driven by large language models, which is the first batch of task-completion-oriented multi-agent systems at home and abroad, and its open-source system has received more than 22,000 stars in March 2024, and has topped the Github Trending list for many consecutive days. The powerful group collaboration model behind it was interpreted by Bailey, the head of Google's DeepMind large model product, and Sanyam Bhutani, a senior data scientist, and Andrew Ng, a well-known scholar of artificial intelligence, used ChatDev as an example at the Sequoia US AI Summit to emphasize that multi-agent collaboration is a powerful design pattern.
Title: Large Model Swarm Intelligence Technology
Abstract: ChatDev (Chat-powered Software Development), a large-scale model-driven full-process automation software development framework, is proposed to be a virtual software company operated by multi-agent collaboration, after a human user specifies a specific task requirement, agents of different roles will interact and collaborate to produce a complete software (including source code, environment dependency manual, user manual, etc.). This technology opens up new possibilities for software development automation, supports fast, efficient and affordable software production, and effectively frees some of the manpower from the heavy labor of traditional software development in the future. Based on the key ideas of ChatDev, this report will share and exchange relevant technologies and practices around the construction, collaboration, and evolution of large language model agents.
陈旭
Associate Professor, Chinese University
About the speaker: Xu Chen, Ph.D. from Tsinghua University, then joined Renmin University Chinese of London (UCL) as a postdoctoral researcher at University College London (UCL) in 2020. His research interests include large language models, causal inference, and recommender systems. He has published more than 80 papers in well-known international conferences/journals such as TheWebConf, NeurIPS, AIJ, KDD, ICLR, etc., and has been cited more than 5,500 times by Google Scholar, and has been selected as one of the top 2% scientists in the world at Stanford University. His research results have won the second prize of CCF Natural Science Award (ranked second), the ACM-Beijing Rising Star Award (three people in Beijing), the Best Paper Nomination Award of TheWebConf 2018 of CCF Class A Conference, the Runner Up Award of CIKM 2022 of CCF Class B Conference, the Best Paper Nomination Award of SIGIR-AP 2023 of the prestigious Asian Information Retrieval Conference, and the Best Paper Award of the famous Asian Information Retrieval Conference AIRS 2017. He led the team to write the world's first review of large language model agents, and built the first user behavior simulation environment "RecAgent" based on LLM Agent. He has presided over or participated in more than 10 projects of the National Natural Science Foundation of China and enterprise cooperation projects, and the relevant achievements have been implemented in many enterprises, including Huawei's "Innovation Pioneer President Award" and Huawei's excellent school-enterprise cooperation projects.
Title: Social and Economic Simulations Based on Autonomous Agents
Abstract:Human behavior simulation has always been the focus of academic attention, but traditional simulation methods are often limited in accuracy and application scope due to the difficulty of accurately describing the complex decision-making process of human beings. In recent years, the development of large language models has given people hope for the construction of accurate human-like agents. The virtual town of Stanford University has brought a huge imagination to many related practitioners. This report will systematically explain the various explorations and efforts made by researchers in the past year in the field of "human behavior simulation based on large language model agents", and analyze the current core problems and potential solutions in this field.
Liang Yitao
Assistant Professor, Peking University
About the speaker: Dr. Yitao Liang is an assistant professor and doctoral supervisor at the Institute of Artificial Intelligence, Peking University. He received his Ph.D. from the University of California, Los Angeles in June 2021 and has been working on neural symbol fusion, where he has been researching how to inject knowledge into machine learning to improve its performance and versatility. He has been nominated for the best paper in the AAMAS2016 of the top conference on reinforcement learning, the best paper in the Reinforcement Learning for Real Life Workshop held at ICML19 and the second best paper in the Learning from Limited Labeled Data (LLD) Workshop held in NeurIPS 2017, and the TEACH held in ICML2023 Workshop Best Papers. In terms of academic services, he has served as a field chair (senior reviewer) for many top journals and conferences all year round.
Title: General Agents in the Open World
Report Summary: With the advent of large language models, there has been a resurgence of debate about whether general-purpose agents will emerge. However, the general capabilities of GPT seem to be difficult to reproduce in the field of words other than text. In this talk, we will cover the various efforts of our group, as well as some other well-known research labs related to it, that are using open-world environments such as Minecraft to develop general-purpose agents. Due to its high degree of freedom, the traditional multitasking, data-driven approach is untenable (we can't train thousands of tasks at scale at the same time, which is too expensive). One possible direction is to use some general common sense to obtain high training efficiency and model generalization. I will focus on how to use large language models to use environmental knowledge to disassemble tasks with systematic generalization capabilities, and how to use unsupervised learning to obtain a general policy expression that can be controlled with prompts, and show what kind of task completion capabilities the agents at the forefront of Minecraft are now capable of.
Woody
Associate Professor, Chinese University
About the speaker: Hu Di is currently an associate professor and doctoral supervisor of the Hillhouse School of Artificial Intelligence, Chinese University, and is funded by the Young Talent Lifting Project of the China Association for Science and Technology. His main research direction is machine multimodal perception and learning, and he has published more than 30 papers in top international conferences and journals in the field as the main author, such as TPAMI, NeurIPS, CVPR, ICCV, ECCV, etc. His representative work includes the visual and audio multimodal scene understanding algorithm DMC and the scene question answering task MUSIC-AVQA, the theory, mechanism and method of balanced multimodal learning, and the kinematic-guided generalizable hinged object manipulation. During his PhD, he was selected into the CVPR Doctoral Consortium, won the 2020 Chinese Society of Artificial Intelligence Youbo Award, the 2021 Shaanxi Youbo Award, won the 2022 Wu Wenjun Artificial Intelligence Outstanding Youth Award, and Baidu Global Top Artificial Intelligence Talent Program. He has been invited to review papers for many international high-level conferences and journals, served as AAAI, IJCAI SPC/Session Chair, etc., and hosted/co-organized a number of multimodal learning workshops (Tutorials) for top international conferences.
Title: Autonomous Agent-based Robotics
Abstract: In recent years, the ability of perception, understanding, reasoning, and decision-making demonstrated by multi-modal large models pre-trained by massive data in complex interactive environments provides reliable basic capabilities for the realization of robot interaction with complex real-world interactions. The content of this report is based on autonomous agent-based robotics technology, mainly introducing how large models can support the generalizable manipulation of robots in complex scenarios, including semantic-level task planning, control-level policy generation, related important work and future development directions.
Academic Director
Liu Zhiyuan
Associate Professor, Tsinghua University
Introduction: Liu Zhiyuan is an associate professor and doctoral supervisor of the Department of Computer Science of Tsinghua University, and a senior member of the China Computer Federation. His research interests include natural language processing, knowledge graphs, and social computing. He received his Ph.D. degree from Tsinghua University in 2011 and has published more than 100 papers in well-known international journals and conferences in the field of artificial intelligence, such as ACL, EMNLP, IJCAI, AAI, etc., with more than 42,000 citations by Google Scholar. He has won the first prize of Natural Science of the Ministry of Education in 2020 and 2022 (the second completer), the first prize of the Qian Weichang Chinese Information Processing Science and Technology Award of the Chinese Chinese Information Processing Science and Technology Society (the second completer), the Hanwang Youth Innovation Award of the Chinese Chinese Information Processing Society, and has been selected as the National Young Talent, the 2020-2022 Elsevier China Highly Cited Scholar, the MIT Technology Review China 35 Under 35 Science and Technology Innovation List, and the China Association for Science and Technology Young Talent Lifting Project.
Lin Yankai
Prospective Assistant Professor at Chinese People's University
Introduction:Yankai Lin, an associate professor at the Hillhouse School of Artificial Intelligence, Renmin University of Chinese, whose main research direction is pre-trained models and large model agents, has published more than 50 papers in CCFA/B international top academic conferences, and has 11,938 Google Scholar statistical citations (until February 2024), with an H-index of 41, and has been selected as an Elsevier China Highly Cited Researcher for three consecutive years from 2020 to 2022. His achievements were awarded the first prize of the Natural Science Award of the Ministry of Education (the third person to complete it) and the leading scientific and technological achievements of the 2022 World Internet Conference (a total of 15 projects in the world). In terms of knowledge representation, his TransR paper was listed as a representative method of knowledge representation by Yoshua Bengio in his "Deep Learning" textbook, and the related work results OpenKE and OpenNRE have received more than 7,800 stars on Github, the world's most influential open source platform. In terms of large model agents, he presided over the release of the world's first large-scale tool learning large model ToolLLM, presided over the release of the large model autonomous agent system XAgent, which received more than 6,900 stars on the open source platform Github, and built RecAgent, a multi-agent simulation platform for simulating user behavior, which is the first multi-agent platform for simulating user behavior at home and abroad.
When: May 17-19, 2024
Address: Lecture Hall, 1st Floor, Institute of Computing, Chinese Academy of Sciences, Beijing (No. 6, South Road, Zhongguancun Academy of Sciences, Haidian District, Beijing)
Take Beijing Metro Line 10 to "Zhichunli Station" and get off at Exit A, and walk for 10 minutes.
Notes:
1. Application fee: 2,800 yuan for CCF members and 3,600 yuan for non-members. Accommodation, transportation (expenses) are at your own expense. According to the order of payment, members will be admitted on the principle of priority until the quota is full. At the request of some students, this ADL will be held online simultaneously, and the online and offline registration fees will be the same. The online meeting room number and password will be sent by email 1 day before the meeting.
2. Registration deadline: May 15, 2024. Please reserve an email address that will not intercept external emails, such as QQ mailbox. One day before the meeting, the meeting notes and WeChat group QR code will be sent by email.
3. E-mail: [email protected]
Payment Method:
Pay online or by bank transfer in the registration system:
Bank transfer (support online banking, Alipay):
Bank: China Merchants Bank Beijing Haidian Branch
Account name: China Computer Federation
Account Number: 110943026510701
Please be sure to indicate: ADL146+ name
After the registration and payment, the registration system shows that the payment is completed, which means that the registration is successful and will not be notified separately.
How to Register:
Please choose one of the following two ways to register:
1. Scan (identify) the following QR code to register:
2. Click the registration link to register:
Hatps://conf.ccf.org.cn/adl146
CCF Recommended
【Articles】
- ADL145 Explores Sora's Past and Present: Basic Models and Key Technologies for Video Generation