According to academics, AI agents combine large language models and automated software to successfully exploit real-world security vulnerabilities by reading security advisories. Four computer scientists at the University of Illinois at Urbana-Champaign (UIUC)—Richard Fang, Rohan Bindu, Akul Gupta, and Daniel Kang—reported in a newly published paper that OpenAI's GPT-4 is capable of: Large language models (LLMs) can autonomously exploit vulnerabilities in real-world systems by giving them a CVE advertisement describing the vulnerability.
To illustrate this, the researchers collected a dataset of 15 single-day vulnerabilities, including those that were classified as severe in the CVE description.
"When given a CVE description, GPT-4 was able to exploit 87% of these vulnerabilities, while the other models we tested (GPT-3.5, open-source LLMs) and open-source vulnerability scanners (ZAP and Metasploit) had 0% utilization."
A "single-day vulnerability" is a vulnerability that has been disclosed but has not yet been patched. The CVE description by the team refers to CVE tagging advisories shared by NIST—for example, this advisory for CVE-2024-28859.
Failed models tested include GPT-3.5, OpenHermes-2.5-Mistral-7B, Llama-2 Chat (70B), LLaMA-2 Chat (13B), LLaMA-2 Chat (7B), Mixtral-8x7B Instruct, Mistral (7B) Instruct v0.2, Nous Hermes-2 Yi 34B, and OpenChat 3.5。 2, Nous Hermes-2 Yi 34B and OpenChat 3.5, but excluding GPT-4's two main commercial competitors: Anthropic's Claude 3 and Google's Gemini 1.5 Pro. Although UIUC engineers would have liked to have tested them at some point, they were unable to obtain these models.
The researchers' work builds on previous findings that LLMs can be used to automate attacks on websites in a sandbox environment.
Daniel Kang, an assistant professor at UIUC, said in an email that GPT-4 "can actually autonomously perform certain steps to implement certain exploits that the open-source vulnerability scanner (at the time of writing) can't."
Kang said that he hopes that the LLM agent created by connecting the chatbot model with the ReAct automation framework implemented in LangChain (in this case) will make it easier for everyone to exploit the vulnerability. It is reported that more information can be obtained from these agents through the link in the CVE description.
In addition, if you extrapolate the capabilities of GPT-5 and future models, it is likely that they will be much more powerful than what the script kids can get today.
The CVE description associated with denying LLM agent (GPT-4) access reduced its success rate from 87% to just 7%. However, Kang said he doesn't see restricting the disclosure of security information as a viable way to defend against LLM agents. He explained: "I personally believe that 'covert security' is untenable, and that seems to be the common opinion of security researchers. I hope that my work and others will encourage people to take proactive security measures, such as regularly updating software packages as security patches are released. "
The LLM agent failed to leverage only two of the 15 samples: Iris XSS (CVE-2024-25640) and Hertzbeat RCE (CVE-2023-51653). According to the paper, the former is problematic because the interface of the Iris web app is very difficult for agents to navigate. The latter is characterized by detailed Chinese instructions, which can probably confuse LLM agents running at prompts in English.
Of the vulnerabilities tested, 11 emerged after GPT-4's training deadline, meaning that the model did not learn any data about these vulnerabilities during training. These CVEs have a slightly lower success rate of 82%, or 9 out of 11.
As for the nature of these vulnerabilities, they are all listed in the above paper, which tells us: "Our vulnerabilities involve website vulnerabilities, container vulnerabilities, and vulnerable Python packages, and more than half of them are classified as 'high' or 'critical' severity according to the CVE. "
Kang and his colleagues calculated the cost of a successful LLM proxy attack and came up with a figure of $8.8 per exploit, which they said was 2.8 times less than hiring a human penetration tester for 30 minutes.
According to Kang, the proxy code is only 91 lines of code and 1056 hint tokens. OpenAI, the maker of GPT-4, asked researchers not to release their tips to the public, though they said they would do so upon request.
OpenAI did not immediately respond to a request for comment.