Machine Heart report
Heart of the Machine Editorial Office
Jim Fan, an AI scientist at NVIDIA, said, "AutoGPT is just an interesting experiment, and although it is hot, it does not mean that it can be put into production." His views have been echoed and spoken out by many people.
As if overnight, a new top player in the AI world appeared: AutoGPT.
As the name suggests, AutoGPT is autonomous artificial intelligence, a task given to it, it can autonomously propose a plan, and then execute, without human intervention. It also features Internet access, file storage using GPT-3.5, and summary generation.
For example, a user asks AutoGPT to build a website, and the request is to create a form, add the title "Made with autogpt" to the form, and finally change the background to blue, which takes less than 3 minutes, without human involvement, AutoGPT does it itself, as shown below. The React and Tailwind CSS used by AutoGPT during this period were all self-decided.
As an example, AutoGPT has been able to look up information on the Internet, use third-party tools, and operate your computer on its own. Since its launch, the popularity of the project has not decreased, and as of today, AutoGPT's GitHub Star volume has reached 78k, and it is immediately close to 80k, surpassing PyTorch's 65k.
AutoGPT address: https://github.com/torantulino/auto-gpt
PyTorch address: https://github.com/pytorch/pytorch
You know, AutoGPT is a project that has only been live for a few days, and the original version of PyTorch dates back to 2018. Not only that, from the summary of Twitter users, AutoGPT also exceeds the Star volume of Bitcoin, Django and other projects.
Source: https://twitter.com/MattPRD/status/1647653434760515584
Even Andrej Karpathy, a former Tesla AI director who has just returned to OpenAI, commented: "AutoGPT is the next frontier of prompt engineering."
However, unlike optimistic about the development of AutoGPT, Jim Fan, an AI scientist from NVIDIA, poured cold water on it.
Jim Fan says he only sees AutoGPT as a fun experiment, nothing more, and that the research, while hot, doesn't mean it can be put into production, and many of the cool demos on the Internet are handpicked out.
Later, Jim Fan also said that in his experiments, "AutoGPT can solve some simple and well-defined tasks well, but most of the time, AutoGPT is not reliable for really useful, more difficult tasks."
This unreliability can be attributed to the inherent limitations of GPT-4. If you don't have access to GPT-4 weights or better fine-tuning, I don't think you can solve the problem by hinting at the root of the problem alone.
Like the ability to turn GPT-3 into GPT-4 without any hint, I don't think AutoGPT + frozen GPT-4 can reliably solve important complex decisions. The current media hype is pushing the project towards completely unrealistic expectations."
Agreer: AutoGPT is too limited to solve any business problem
Jim Fan's view was shared by many. Some people think, "Admittedly, AutoGPT is a great experiment and will lead the wave of doing a lot of cool things autonomously through agents." But it can't be a product that can build the foundation to solve any business problem, after all, it's too unpredictable."
Someone came forward to say that they had been asking AutoGPT to open a docx document, open its exported ChatGPT conversation to provide more context (JSON), browse other technical content, and rewrite the docx document all Saturday. Unfortunately, AutoGPT didn't even come close to achieving these goals, so give up.
There are many more examples of these experiences, and there are a lot of prompts for real-world problems, but AutoGPT always goes in different directions that don't make any sense.
Dissenters: Although exaggerated, its prospects are comparable to GPT
While many people agree with Jim Fan, others point out that while AutoGPT is definitely exaggerated, it is now very "brute" and inelegant. But the prospects it presents are still very strong, almost on par with GPT models.
Some people dissect the shortcomings of AutoGPT from an application perspective, although it does not solve many things well, such as looping, tangent, and randomly completing different tasks. But to be clear, AutoGPT requires a lot of brain power, and it's expected to get better and better.
The above view is not unique, "AutoGPT will definitely improve over time. Projects like this became possible two years ago, although reliable use on any generic domain may only arrive in years, not months."