Deep synthesis or deep forgery? The prelude to the "metacosm" governance "offensive and defensive game" has begun

As a new practice in the field of artificial intelligence, in recent years, the deep synthesis technology that uses deep learning, virtual reality and other generative synthesis algorithms to produce images, audio, video, virtual scenes and other information has been widely applied in many fields. With the emerging demand for scenes, the number and attention of deeply synthesized content have surged, but on the other hand, there are also huge security risks for the audio and video generated by malicious use of this technology.

The number of synthesized videos increased by more than 10 times

As the "deep synthesis" technology gradually matures and enters the stage of commercial application, its huge economic value has gradually been revealed. According to a "Report on Ten Trends in Deep Synthesis (2022)" (hereinafter referred to as the "Report")," jointly released by the Artificial Intelligence Research Institute of Tsinghua University, Beijing Ruilai Smart Technology, national industrial information security development research center and other units on Monday, the number of newly released deep synthesis videos in 2021 has increased by more than 10 times compared with 2017, and the number of likes has exceeded 300 million.

According to the report, deep synthesis has developed diversified commercial applications in the fields of film and television production, advertising and marketing, and social entertainment, such as AI synthesis anchors, virtual idols, repairing old photos of history, localized dubbing of film and television dramas, "digital resurrection", etc.; and the proposal of new commercial thinking such as "meta-universe" provides a broader application scenario for deep synthesis.

"For example, virtual people and digital people are the main applications of deep synthesis, and they are also an important part of the metaverse." Xue Hui, head of Alibaba's security perception and cognitive intelligence department, said. Chen Changfeng, executive vice dean of the School of Journalism and Communication at Tsinghua University, also said: "Deep synthesis will redefine the virtual digital space, and in the sense of communication sociology, a new human survival scenario will be developed based on deep synthesis technology." ”

In the immersive shared virtual world represented by the metaverse, with the help of AR, VR and 3D technology, the boundary between virtual and reality is gradually blurring, and it is difficult to distinguish between true and false.

Tian Tian, CEO of Ruilai Wisdom, told the first financial reporter that the continuous maturity of technology is an important reason for the explosive growth of deep synthetic content. "The continuous increase in research papers, the emergence of open source technology tools and a large number of representative methods have made the effect of deep synthetic content more realistic and more efficient, especially the emergence of algorithms such as generative adversarial networks, which has made synthetic content difficult to distinguish between true and false." He said.

NVIDIA, the top technology company in the field of graphics computing, last year used superb deep synthesis technology to synthesize a digital twin version of Huang Jenxun, which almost fooled the eyes of the world. The leather jacket worn by "Digital Jen-hoon" and the kitchen where he is located are all simulated by computer scientists through 3D simulation technology, showing the powerful creativity and possibilities of digital technology.

"Fun" hides risks behind it

However, while deep synthesis stimulates innovative content, it also brings new threats. The "Report" pointed out: "With the gradual popularization of technology, criminals can easily forge audio and video, carry out illegal acts such as framing, defaming, fraud, and extortion, and disrupt social order." ”

In October 2021, police in Hefei, Anhui Province, seized a case of illegally using deep synthesis technology to forge mobile phone users' faces and dynamic videos to crack identity verification, providing technical support such as registered virtual mobile phone cards for the black and gray industry. In recent years, similar incidents have begun to enter the public eye more.

Ren Kui, dean of the School of Cyberspace Security at Zhejiang University, said: "At present, the detection of deep synthesis mainly relies on the completeness of the artificial intelligence model and training data, including the relatively low versatility of the detector, the applicability of the public data set, and the sensitivity of the data. ”

Since the advent of Deepfake (deep forgery) in 2017, the ability of AI technology in counterfeiting has aroused the world's attention. The rapid development of algorithms can not only achieve AI face change, but also automatically generate various digital content such as text, artificial voice, and images. Previously, the State Internet Information Office and the Ministry of Public Security instructed local network information departments and public security organs to strengthen the security assessment of new Internet technologies and new applications involving voice social software and "deep forgery" technology, and interviewed relevant enterprises in accordance with the law.

At present, academia and industry have invested a lot of research on anti-"deep forgery" detection, and technology giants such as Meta, Google, and Microsoft have launched methods or products for deep synthetic video certification. In China, universities and enterprises such as Tsinghua University, University of Science and Technology of China, Ruilai Smart Technology, Baidu, and iFLYTEK have achieved remarkable results in in-depth content detection.

Zhu Jun, director of the Basic Theory Research Center of the Institute of Artificial Intelligence of Tsinghua University, believes that deep synthesis detection faces continuous attack and defense and game, and in the future, it is necessary to integrate multi-modal content forensic analysis, traceability technology based on digital watermarking and other aspects of capabilities to achieve accurate identification.

Tian Tian also told the first financial reporter: "New forgery methods emerge in an endless stream, the network communication environment is becoming increasingly complex, coupled with the existence of vulnerabilities and defects based on detection algorithms, anti-deep pseudo detection technology faces strong confrontation and needs continuous update and iteration." ”

Explore deep synthetic governance paths

In addition to the development of deep-depth forged content detection technology, in recent years, in response to the challenges brought about by the malicious use of deep synthesis technology, countries around the world have issued relevant laws and regulations to explore the governance path of deep synthesis. Internationally, the United States has made specific legislation at the federal and state levels, and the European Union has incorporated deep synthesis into existing legal frameworks such as the General Data Protection Regulation (GDPR).

On January 28, 2022, the Cyberspace Administration of China (CAC) recently published the Provisions on the Administration of Deep Synthesis of Internet Information Services (Draft for Solicitation of Comments) (hereinafter referred to as the "Deep Synthesis Consultation Draft"), which makes a series of clearer provisions and guidelines for deep synthesis technology as the cornerstone of the metacosmum.

According to the definition, AI voice, NFT generation art, virtual concerts, holographic portrait projection, virtual people digital people, AR shopping and other important components of the metaverse belong to the specific application of deep synthesis technology, all within the scope of the "Deep Synthesis Consultation Draft", which will have a profound impact on the supervision of deep synthesis and even the artificial intelligence industry, especially the prevention of "deep forgery" technology.

Wu Hequan, an academician of the Chinese Academy of Engineering, believes that there are two main principles for the governance of deep synthesis, one is to continue to develop technology, can not be "one-size-fits-all" prohibition, to avoid hindering positive application and innovation; the other is to solve the derived safety problems from the source, the use of technological innovation, technological confrontation and other ways, continue to improve and iterative detection technology capabilities.

Shi Lin, deputy director of the Artificial Intelligence Department of the Yunda Institute of the Chinese Academy of Information and Communications Technology, believes that it is necessary to make a clear distinction between "deep forgery" and deep synthesis, and cannot use the "stigmatization" term of "deep forgery" to summarize the "deep synthesis" technology. "There is no distinction between good and evil in deep synthesis technology itself, and deep forgery occurs when the technology is abused and crosses the boundaries of morality and law." Shi Lin said.

Cui Baoqiu, vice president of Xiaomi Group and chairman of Xiaomi Security and Privacy Committee, told the first financial reporter: "Technical supervision is an eternal game of offensive and defensive confrontation. While providing services, the service provider of deep synthesis technology will generate many risks, in addition to the risk of deep forgery, but also the risk of copyright and copyright infringement brought about by automatically generating content text, images or videos, the risk of information security and privacy leakage, and the risk of sensitive content. ”

Cui Baoqiu suggested that in the relevant regulations issued in the future, it should be mandatory for service providers to label which content is deeply synthesized, and for technology providers, they should start from the underlying technology to promote the establishment of relevant standards and ensure the fairness of the calculation model in the algorithm.