"AI face swapping" is no stranger to everyone. With just one photo, through AI deep synthesis technology, anyone can become the protagonist of the video, and even the expressions and movements can be fake. However, this new technology has made some people have bad ideas, and Yu in Hangzhou used the "AI face swap" software to generate a large number of obscene videos for dissemination, and there were many female celebrities and other public figures who were swapped, and Yu even provided face swap customization services.
"AI face swapping" fakes pornography videos of female celebrities
In August 2022, the Xiaoshan police in Hangzhou City obtained a clue during an online inspection that someone in the jurisdiction used AI technology to make obscene videos for a large number of online dissemination, and the police filed a case for investigation in accordance with the law, and quickly arrested the suspect Yu, and then the case was transferred to the People's Procuratorate of Xiaoshan District, Hangzhou City for review and prosecution.
Zhang Lei, full-time member of the Procuratorial Committee of the People's Procuratorate of Xiaoshan District, Hangzhou City: Yu discovered that there is an AI face-swapping technology software on the Internet, and he himself mastered the skills of using this software, and since then he has begun to use AI face-swapping technology to make profits.
The procuratorate's investigation found that Yu began to contact the "AI face swap" software in 2020, at first, he wanted to make a profit by promoting the technology and related software online, but after a period of time, there were very few people who paid attention to him, and by chance, Yu saw someone posting obscene information in social groups to attract the attention of others, so he also started similar thoughts.
Chen Yuyao, assistant prosecutor of the Seventh Procuratorate Department of the People's Procuratorate of Xiaoshan District, Hangzhou: He may find that it may be difficult to get the attention of ordinary people just by selling software. Because he saw that some online groups were spreading some obscene pornographic videos on the Internet, he thought of making some obscene pornographic videos through face-swapping software, and then putting them on the Internet group first, so as to attract traffic and attract everyone to pay attention to this software or face-swapping videos.
According to Yu's confession, in order to attract more attention and traffic, he created a "browsing group" and a "private custom group" on social software, and the "browsing group" is mainly used to spread obscene videos of "AI face swapping" to attract and solicit viewers, including some videos that have undergone Yu's face swap, and when someone puts forward the demand for "customized face swap", Yu will introduce these people into the so-called "private custom group".
According to the Xiaoshan Procuratorate
Chen Yuyao, assistant procurator of the Seventh Procuratorate Department of the People's Procuratorate of Xiaoshan District, Hangzhou City: By charging fees for joining (browsing) groups, one is 198 yuan, 298 yuan or 398 yuan. The other is that Yu also provides face video customization services, which is equivalent to that if a customer needs to customize a face-swapping video, he can charge a corresponding fee according to the duration and the difficulty of the video.
According to the Xiaoshan procuratorate, as of the time of the case, the public security organs had seized more than 1,200 obscene videos and more than 1,600 pictures in the public group set up by Yu.
Yu XX formed the chat group and disseminated the synthetic obscene video in the group
Private customization → "as long as you can see the face, you can change it"
According to the investigation of the procuratorate, Yu made a profit of tens of thousands of yuan in less than half a year by selling "AI face-swapping" software and providing so-called "AI face-swapping" private customization services. The victims of the "AI face swapping" infringement of sensitive information on faces not only include many well-known public figures, but also many unspecified ordinary people.
According to the investigation of the procuratorate, Yu charged customized fees according to the difficulty conditions such as the length and clarity of the "AI face swap" video, according to the standards of 300 yuan for 5 minutes and 500 yuan for 10 minutes. Yu claimed, "As long as you can see your face, you can change it."
Chen Yuyao, assistant prosecutor of the Seventh Procuratorate Department of the People's Procuratorate of Xiaoshan District, Hangzhou City: First of all, this price is also set by himself, and then a primary, intermediate, and senior level, there may be a certain difference in the effect of completing the face change at the end of this video. Some people may have purchased a beginner's tutorial at the beginning, some materials and materials for the beginner's level, and found that the final effect could not achieve the desired effect, then they may continue to pay for some advanced videos, including advanced tutorials.
Zhang Lei, full-time member of the Procuratorial Committee of the People's Procuratorate of Xiaoshan District, Hangzhou City: He customized through VIP, that is, the customer asked "whose face I want to change my face to today, you help me customize it, and I will pay the corresponding remuneration", through this way of making profits.
The "customized" face swap targets are both public figures and ordinary people
According to the procuratorate's understanding, Yu's customers asked for "customized" face swaps, including not only some well-known public figures, but also ordinary people in life, among which some people proposed to customize their crush on the girl's face swap video, and for these illegal acts of infringing on other people's sensitive information without consent, Yu also did not refuse.
In addition, in addition to customized face swap services, Yu also divided the "AI face swap" software into three levels: primary, intermediate and advanced for sale and taught tutorials, and also provided other people's face materials illegally collected online.
Assistant Procurator of the Seventh Procuratorate Department of the People's Procuratorate of Xiaoshan District, Hangzhou Chen Yuyao: The threshold for the entry of this software is actually relatively low, its operation is equivalent to through some programs and codes that have been set to run, which is equivalent to me as long as I follow your steps, put the video that needs to be changed into a folder specified in it, or follow the steps to operate, it will automatically extract the face information in the face video you want to replace, or the information in the photo, automatically establish a learning model for learning, and then the next step is to automatically synthesize.
According to the evidence collected by the procuratorate, among the many social groups created by Yu, the minimum number of members was 95, and the maximum number of members reached 2,053, and through illegal acts such as selling face-swapping obscene videos and customizing face-swapping obscene videos, Yu received password red envelopes more than 200 times, with a total amount of more than 60,000 yuan.
In addition to pursuing criminal responsibility
The procuratorate also filed a civil public interest lawsuit
The procuratorate believes that as a user of the "AI face swap" technology, Yu has further expanded the social danger of more "unspecified subjects" having their personal information infringed by disseminating methods and means of infringing on citizens' personal information. Therefore, the People's Procuratorate of Xiaoshan District, Hangzhou City, in addition to pursuing Yu's criminal responsibility in accordance with the law, also filed a civil public interest lawsuit. In July 2023, the Hangzhou Internet Court heard the public interest litigation case, which is also the first public interest litigation case in mainland China on the infringement of citizens' personal information by "AI face swapping".
On April 14, 2023, the People's Procuratorate of Xiaoshan District, Hangzhou decided to file a public interest lawsuit against Yu's "AI face swapping" infringement of citizens' personal information. On June 9, the procuratorate filed a civil public interest lawsuit against Yu in accordance with the law. On July 28, the case was publicly heard in the first instance of the Hangzhou Internet Court.
During the trial, the procuratorate presented detailed evidence of Yu's infringement of citizens' personal information and harm to the public interest, including records of interrogations and interrogations, inquests, inspection records, and appraisal opinions made by the public security organs against Yu, as well as more than 1,200 obscene videos, more than 1,600 pictures, and other electronic data.
Xiao Peng, President of the First Division of the Internet Trial of the Hangzhou Internet Court: In the past, we often thought that seeing was believing and hearing was believing, but with the application of deep synthesis technology and new technologies, what we saw was not necessarily believing, and the improper application or abuse of new technologies actually impacted the trust relationship between the main bodies of the public in our society, especially in the Internet space. Then a kind of infringement on this kind of trust relationship and social relationship is actually a kind of damage to social order.
After trial, the Hangzhou Internet Court held that the illegal facts of Yu's use of "AI face swapping" technology to infringe on citizens' personal information were conclusive, and therefore supported all the litigation claims of the procuratorate as the plaintiff in public interest litigation: order the defendant Yu to delete the illegally obtained personal information, publicly apologize, eliminate the impact and compensate for losses totaling 60,000 yuan, and these funds will be used exclusively for public welfare matters such as personal information protection and governance of improper application of face deep synthesis technology.
Zhu Minming, Executive Vice President of Hangzhou Internet Court: Any new technology, first of all, it is always neutral, so we have an attitude of acceptance and support in the technology itself, but the purpose of technology development and application, it must be beneficial to the public welfare of all mankind, and it cannot be used for personal illegal profit-making for the sake of this technology, and then infringe on the legitimate rights and interests of other citizens, collectives, countries and society, once it crosses the line, it must bear the corresponding responsibility. In this case, because the procuratorate will file a public interest lawsuit, and then the court will try it in accordance with the law, it will give full play to the judicial function of the trial to regulate the new technology.
There is no good or evil in technology, but there are boundaries
It must be used within the scope of the law
In November 2023, the Xiaoshan District People's Court of Hangzhou City convicted the defendant Yu of the crime of producing and disseminating obscene materials for profit, and sentenced him to seven years and three months in prison and a fine of 60,000 yuan.
"With just a few photos, anyone can be a victim. From the live broadcast of female celebrities who change the face of online anchors to bring goods, to the face swap being grafted into obscene videos, the abuse of "AI face swap" technology to carry out various acts of infringing on personal information occurs from time to time, and even abuses in the gray field and even in the field of illegal crimes. So, where is the legal boundary of "AI face-swapping" technology, and how should it be regulated and governed?
Xiao Qian, President of the First Division of the Internet Trial of the Hangzhou Internet Court: Our country's Civil Code has clear provisions on infringing on the personal rights and interests of others through deepfakes, and prohibiting the infringement of citizens' personal and property rights and interests through deep synthesis, which is a provision of the Civil Code. With the development of technology, the PIPL also has relevant provisions on deep synthesis, because whether it is a replacement of the face or a replacement of the voice, it actually needs to collect the personal information of citizens, so these are sensitive biometric information that is specially protected by the PIPL.
In addition to the Civil Code and the Personal Information Protection Law, the Provisions on the Administration of Deep Synthesis of Internet Information Services and the Interim Measures for the Administration of Generative Artificial Intelligence Services, which will be officially implemented in 2023, also clearly stipulate that users shall not use deep synthesis services to engage in activities prohibited by laws and administrative regulations, and require deep synthesis service providers to implement the main responsibility for information security.
Caution!
There are also legal risks associated with the use of "AI face swapping" to make jokes
The judge held that even if the "AI face swap" technology is only used to make funny or spoof photos, it may involve infringement of portrait rights and reputation rights, and if it causes serious consequences, it may also constitute the crime of insult or defamation.
The abuse of "AI face swapping" technology seems to be an infringement of the personality rights, portrait rights, reputation rights, and privacy rights of the infringed individuals, but in fact, it is a trampling on social order and good customs, and a challenge to a stable social environment and a clear cyberspace.
Zhu Minming, Executive Vice President of Hangzhou Internet Court: The production of a new technology often lags behind in the laws and regulations specifically for this new technology, but in general, the entire rule of law system in our country has been quite perfect. I think normal people can judge whether you use this new technology to benefit the whole human race, whether it is to improve people's living standards, improve their well-being, or seek improper benefits for themselves. Then I want to grasp this boundary, these new technologies can serve our better life.
Source: CCTV News