At the beginning of last year, ChatGPT detonated a new wave of AI. While people expect large models to bring disruptive changes to all walks of life, they also have to admit that large models create new opportunities, but also bring new security risks.
For example, based on the efficient content generation characteristics of large models, hackers' attack costs and thresholds have become lower, making it easier to launch more intensive attacks and promote hacking behaviors; based on the advantages of generative AI in multiple languages and information integration, the cost and time of counterfeiting an extremely realistic scam have been reduced, and the scale and scope of phishing activities and the implementation efficiency have been greatly improved; and the age-old problem of data leakage, which puts the security and privacy of enterprises at great risk.
It's hard to imagine how serious the consequences would be if the security of the large model was not guarded against and allowed to grow wildly.
Now, after the "100 model war" that lasted for more than a year last year, the large model has entered the stage of landing and application, and has begun to enter thousands of industries, deeply integrating with various scenarios. Based on this, in the face of the security threats and challenges brought about by the continuous development of large models, how to take active measures to innovate technologies and methods under the existing and traditional security tools and policy systems, and create a safe and credible large model has become the focus of attention of major large model manufacturers and practitioners around the world.
WDTA has released international standards for large model security, and AI security assessment and testing have entered a new benchmark
From April 15th to 19th, the 27th United Nations Conference on Science and Technology was held in Geneva, Switzerland. Among them, on April 16, at the AI side event with the theme of "Shaping the Future of AI", the World Digital Technology Institute (WDTA) released a series of breakthrough achievements, including two international standards, "Security Testing Standards for Generative AI Applications" and "Security Testing Methods for Large Language Models".
WDTA released international standards for large-scale model security, with OpenAI, Ant Group, iFLYTEK and others co-compiling
According to Leifeng.com, the World Digital Technology Institute (WDTA) is an international non-governmental organization registered in Geneva, which follows the United Nations guidance framework and is committed to promoting digital technology and promoting international cooperation on a global scale.
Among them, the AI STR (Safe, Trusted, Responsible) program is a core initiative of WDTA, which aims to ensure the safety, trustworthiness and accountability of AI systems. Ant Group, Huawei, iFLYTEK, International Data Space Association (IDSA), Fraunhofer Institute, and China Electronics are all members.
The two international standards released at the conference, the "Security Testing Standard for Generative AI Applications" and the "Security Testing Methods for Large Language Models", are the first international standards issued by an international organization in the field of large model security, representing a new benchmark for global AI security assessment and testing.
It is reported that these two international standards are jointly compiled by a number of experts and scholars from dozens of units such as OpenAI, Ant Group, iFLYTEK, Google, Microsoft, NVIDIA, Baidu, and Tencent.
It is understood that the "Generative AI Application Security Testing Standard" (hereinafter referred to as the "Standard") is led by WDTA, and the "Standard" outlines a comprehensive framework for testing or verifying the security of downstream AI applications, especially those built using large language models (LLMs).
Overall, the Standard defines the scope of testing and validation for each layer of AI application architecture, including base model selection, embedding and vector databases, RAG or retrieval enhancement generation, APP runtime security, etc., to ensure that all aspects of AI applications are rigorously assessed for security and compliance to protect them from various threats and vulnerabilities throughout their lifecycle.
The "Large Language Model Security Test Method" (hereinafter referred to as the "Test Method") was compiled by Ant Group.
Compared with the Standard, the Test Method provides a comprehensive, rigorous and practical structural scheme for the security assessment of the large model itself. Attack classification and classification methods and testing methods, and the first to give four types of attack intensity of attack methods classification standards, provide rigorous evaluation indicators and test procedures, etc., can effectively solve the inherent complexity of large language models, test their ability to resist hostile attacks, enable developers and organizations to identify and mitigate potential vulnerabilities, and ultimately improve the security and reliability of AI systems built with large language models.
It is urgent to focus on the security risks of large models and build a strong line of defense
At the meeting, Huang Lianjin, head of the WDTA Working Group on Artificial Intelligence Security and Trustworthiness, said that these two standards bring together the wisdom of experts in the field of global AI security, fill the gap in the field of security testing of large language models and generative AI applications, and provide a unified testing framework and clear testing methods for the industry, which will help improve the security of AI systems, promote the responsible development of AI technology, and enhance public trust.
As Huang Lianjin said, the formulation of these two standards has gathered the wisdom of many experts and scholars at home and abroad, and manufacturers in the field of large models such as OpenAI, Ant Group, iFLYTEK, Google, Microsoft, NVIDIA, Baidu, Tencent, etc., as well as security vendors such as 360, Qianxin, and Sangfor, have also carried out long-term exploration on the road to creating safe and credible large models.
For example, OpenAI, the company behind the most cutting-edge large model technologies such as ChatGPT and GPT-4, announced the establishment of a Superalignment team in July last year, co-led by co-founders Ilya Sutskever and Jan Leike, with the aim of creating an "AI researcher" who is close to human level and responsible for model alignment, that is, using AI to supervise AI.
As the computing power provider behind the rapid development of large models, NVIDIA launched a software called NeMo Guardrails in April last year, which can set security "guardrails" for AI models to avoid outputting some wrong facts, involving harmful topics or causing security risks, so as to solve the "illusion" problem of large models.
In March this year, the domestic security vendor 360 released version 3.0 of the 360 security model, and security vendors such as Qianxin and Sangfor have also released AI+ security products one after another, and have conducted in-depth exploration on the road of exploring the security model.
Ant Group, which is the lead compiler of the "Large Language Model Security Testing Method", has rich experience in the field of security.
As we all know, Ant Group was originally born from the gadget of Alipay, and has developed to the present, and the importance of safety and reliability is self-evident when it comes to people's "money bags", so since 2015, Ant Group has been actively investing in the research of trusted AI technology, and has now established a large-scale comprehensive security governance system.
According to Leifeng.com, in September last year, Ant Group self-developed the industry's first large-model security integration solution "Ant Tianjian", including two major products, the large model security detection platform "Ant Jian" and the large model risk defense platform "Tianjian", which can be used for AIGC security and authenticity evaluation, large model intelligent risk control, AI robustness and explainability detection, etc.
The "Evaluation Method" released this time is based on the application practice of Ant Group's "Ant Tianjian" AI security detection system and exchanges with global ecological partners.
In addition, Ant Group has set up a technology ethics committee and a dedicated team within the company to assess and manage the risks of generative AI, and all of the company's AI products are required to pass a technology ethics evaluation mechanism to ensure that AI is safe and trustworthy.
Not only that, in February last year, Ant Group also established a Technology Ethics Advisory Committee on the basis of the company's internal Technology Ethics Committee, and regularly invited experts and scholars in the field of AI to conduct in-depth discussions on topics such as the construction of science and technology ethics governance system, generative AI governance, and large model risk management, in an effort to build a "safe, compliant, controllable, and reliable" large model.
Wang Weiqiang of Ant Group, as a representative of the standard participating units, spoke at the meeting
As Wang Weiqiang, general manager of Ant Group's machine intelligence department and chief scientist of Ant Security Lab, said at the conference, "Generative AI will unleash tremendous productivity, but we must also be highly vigilant about the new risks it brings." ”
Big tech companies should play a key role in promoting the safe and responsible development of generative AI, using their resources, expertise, and influence to drive best practices and build an ecosystem that prioritizes security, privacy, reliability, and ethics. For example, by developing industry standards and guidelines to provide clear guidance for developers and institutions developing and deploying generative AI systems, and investing in R&D and opening up tools to ensure the security of generative AI, industry co-governance is formed.
Nowadays, when large models begin to enter all walks of life and are implemented in thousands of scenarios, creating safe, credible, and reliable large models is no longer a problem that a large model manufacturer needs to solve, but should be a joint effort from all walks of life in the industry to face and meet the challenges of the new era of AI and create a better future.