On July 3, Chen Lan, an Internet celebrity female anchor with more than 6.7 million followers, posted a video saying that she had encountered malicious PS and AI face changes, and the photos were made into pornographic photos and videos and sold publicly online.
Chen Lan said that she wanted to collect relevant evidence and find out the root cause, but did not want to be too public, but as more and more people in the comment area "spread rumors", she planned to call the police.
Recently, the reporter's investigation found that many accounts sell pornographic photos and videos of celebrities and Internet celebrities changing faces on social platforms, and more than 100 videos are sold for only a few tens of yuan.
Sellers selling pornographic materials for face swapping are mainly divided into two modes, one is to use public photos of celebrities to "remove clothes"; The other is to swap celebrities' faces into pornographic images and videos.
On many e-commerce platforms, many merchants sell AI drawing software online, claiming that such software can provide "decoating" function, and the software is priced at only a dozen yuan.
In fact, "AI face swapping" can not only produce pornographic photos and videos, but may also be used by people to scam them. On June 25, the Legislative Affairs Commission of the Standing Committee of the National People's Congress held a press conference, and spokesman Zang Tiewei said that the use of "AI face change" technology to commit fraud is a new type of telecommunications network fraud, which requires the general public to improve their awareness of fraud prevention and prevention, and also requires relevant departments to strengthen governance and law enforcement in accordance with the law, and investigate the legal responsibility of relevant personnel.
Face-swapping videos
Being openly sold
In addition to the female anchor Chen Lan, Du Yue, an Internet celebrity female anchor with more than 4 million fans, also encountered a similar situation.
According to Du Yue, half a year ago, many fans left messages in the comment area of her short video, saying that they had all seen Du Yue's "pornographic videos".
"I didn't think much about it at the time, I thought that the Qing Dynasty cleaned himself, after all, this 'rumor' will always pass." Du Yue said that it was not until one day that a friend of hers transferred a pornographic video in which the heroine was exactly what she looked like, that she realized that she had encountered a malicious "face change".
Du Yue said that in order to prove her innocence, she began to collect relevant evidence to find the root cause. "Finally, I found that someone had registered an account with a fraudulent identity on an overseas social platform, and then 'transported' it to a domestic video platform to post it, and when the account had a certain number of fans, they began to sell pornographic videos after changing faces."
A price list provided by Du Yue shows that hundreds of Internet celebrities are involved, and it is also divided according to the size of fans, from less than 500,000 fans to more than 10 million fans, and the corresponding face-changing videos are priced at 30 yuan to 50 yuan.
In addition, there are many female stars in the table. The publisher of the form said, "200 yuan can enter the group, and after entering the group, it can be customized privately, and the permanent half price is permanent."
The reporter's investigation found that on some social platforms, many accounts claimed that they could sell Internet celebrities and celebrities
Face pornographic material, in this way, is led to QQ for trading.
According to the clues provided by Du Yue, the reporter added a QQ friend of a seller, and the other party directly sent a screenshot with dozens of female celebrity pictures pasted on the picture.
The seller said he had more than 80 celebrities and more than 15,000 pictures. If you buy them all, it will only cost 60 yuan, and 23 videos will be given away, and you can enjoy a free update after that. The seller also said that if you buy a single celebrity's face change picture, you will charge according to the number of photos, and 5 yuan can buy 300 photos.
Some sellers said that 60 yuan can buy face-changing pornographic materials of multiple celebrities (reporter screenshot)
In addition, there are sellers who say that according to the buyer's needs, he can specifically produce pornographic material that designates a certain star and influencer.
After paying 60 yuan, a seller sent dozens of compressed packages, all named after the stars. The reporter found that the face-changing pornographic materials sold by the seller are mainly divided into two modes, one is to use the public photos of celebrities to "remove clothes", and the other is to change the faces of these stars into pornographic pictures and videos.
Zhang Qihuai, director of Beijing Lanpeng Law Firm, has represented a number of "AI face change" related cases. In his opinion, "AI face changer" was supposed to help advertising, movies, art, etc. to make proper repairs, which was a good technology, but now many people have begun to use this technology for illegal profit.
Zhang Qihuai said that using other people's portraits to change faces is suspected of infringing on the portrait rights, reputation rights, honor rights and personality rights of the parties concerned. If a person's face is made into a pornographic video and sold, it may constitute a criminal offence.
"AI Drawing Software"
Only 16.66 yuan
Monthly sales exceeded 200
Recently, reporters searched for the keyword "AI face change" on multiple e-commerce platforms, but they could not be displayed. However, reporters searching for keywords such as "intelligent synthesis, star", "real face model training" and "clothing AI" can still search for goods on sale.
The product page of an "AI drawing software" clearly states that "the software can realize the function of converting real people to cartoons, changing heads and changing clothes". The price of this software is only 16.66 yuan, and the monthly sales volume reaches more than 200.
"Can you really 'get dressed'?" Some people raised questions in the store's comment area, and many people replied that they could. In addition, some people asked if it was possible to produce pornographic images, and someone replied, "Just buy the software to enter the buyer communication group, everything is available." ”
In the Q&A section of an AI drawing software, someone replied that the software did have a "stripping" function. (Reporter screenshot)
After the reporter entered the "buyer exchange group", in just two days, the group members quickly grew from 40 to 150 people, most of whom exchanged how to use the software in the group and shared their "works" from time to time.
Some members asked in the group if anyone was willing to take orders and make "custom diagrams". "Isn't this thing illegal?" Some people asked rhetorically in the group, and the other party said "wealth is in danger".
In addition to selling drawing software, there are also merchants to provide "real model training", which is the use of real people photos or videos production, after the training, you can directly use the model to generate the corresponding photos and videos to achieve the effect of face change.
A merchant that provides substitute training services told reporters that only 20 to 30 photos need to be provided, and the face is mainly close-up as much as possible, and the training can be completed in more than 2 hours. The buyer can then import the data into a piece of software and generate the desired footage.
The merchant sent a renderings to reporters, which is a renderings produced using a real photo of a domestic model, and after changing faces, the face is very similar to a domestic star.
The above-mentioned merchants said that after the model training is completed, buyers can generate various photos according to their own appeals, including "de-dressing" materials, but the "de-dressing" operation is prohibited in the cloud, and they need to operate it on the computer.
Demystifying the "AI Face Swap" scam
Behind the merchants selling "crime tools", there are also people who use "AI face changing" technology to defraud.
Zhang Qihuai told reporters that he had represented many transnational extortion cases in northern Myanmar. The suspect in this case used the "AI face change" technology to disguise himself as a woman and induce the victim to "chat naked" with him. Subsequently, the video recording content of both parties was used as evidence to extort the victim.
Recently, China Galaxy Securities issued an announcement that the use of "AI synthesis technology" fraud methods may gradually affect the securities industry.
According to the announcement, a well-known economist said that he was used by "AI face change" and other technologies to contact investors through video, voice and other means to deceive investors into trusting and carry out stock recommendation activities.
Previously, police in many places reported fraud cases related to "AI face change". On May 20, the Baotou police released a case of telecom fraud using AI to change faces. The client, Mr. Guo, suddenly received a WeChat video from a friend, who told Mr. Guo that he had a friend who needed a deposit of 4.3 million yuan to bid abroad, and wanted to borrow Mr. Guo's company account to post the account. Mr. Guo felt that he had opened a video with his friend and confirmed that it was him, so he didn't think much and transferred money directly to the corresponding account. In 10 minutes, Mr. Guo was cheated out of 4.3 million yuan because of a video call.
In February 2022, Mr. Chen reported that he had been defrauded by "friends" of nearly 50,000 yuan. After verification by the police, the fraudsters used the video posted by Mr. Chen's friend "Ah Cheng" on social platforms, intercepted his facial video picture and then synthesized it with "AI face changing" technology, creating the illusion of a video chat between Mr. Chen and his "friend" to deceive his trust, thereby committing fraud.
The legislature is being established
Relevant legal regimes
With the development of artificial intelligence, the technical threshold of "AI face changing" has gradually decreased, and there are more and more application scenarios, while meeting people's needs for curiosity, beauty, socialization, etc., in order to make profits, some criminals break through the bottom line of morality and the red line of the rule of law.
In order to curb the chaos of "AI face changing", on November 25, 2022, the Cyberspace Administration of China, the Ministry of Industry and Information Technology, and the Ministry of Public Security jointly issued Article 17 of the Provisions on the Administration of Deep Synthesis of Internet Information Services, which requires that where deep synthesis service providers provide deep synthesis services such as intelligent dialogue, synthetic human voice, face generation, and face replacement, which may cause confusion or misidentification among the public, they shall be conspicuously marked in the reasonable location and area of the information content generated or edited. Inform the public about deep compositing.
Zhang Qihuai said that many criminals now begin to use the "AI for face" technology to commit crimes, because legislation always lags behind social phenomena, it is difficult for everyone to grasp legal standards in judicial practice of such cases, so it is very necessary to improve relevant legislation as soon as possible.
Zhang Qihuai said that for the "AI face change" technology, it is necessary to strengthen technical supervision and identification, and strengthen the popularization of related technologies. If you encounter infringement in a similar situation, you should report or prosecute it in a timely manner when possible, and you should not condone such phenomena for fear of trouble.
Zang Tiewei said that at present, the Central Propaganda Department and the Ministry of Public Security are carrying out the "National Anti-fraud in Action" concentrated publicity month activities across the country, and will further implement the anti-telecommunications network fraud law, continuously enhance the public's awareness of anti-fraud, and effectively create a strong atmosphere for anti-fraud in the whole society. The legislature is also strengthening research on legal issues in areas such as artificial intelligence, and continuously establishing and improving relevant legal systems.
Zhu Wei, deputy director of the Communication Law Research Center of China University of Political Science and Law, said that the regulatory focus on AI technology,
First, the producers of the content are responsible for the results.
Second, users need to be informed in the process of using AI technology.
If someone encounters the malicious face changing behavior of others, even if the malicious face changing person has not been found, the platform that provides the face changing technology needs to be responsible, because the platform itself also has the responsibility and obligation to supervise what kind of materials users use.
(Chen Lan and Du Yue are pseudonyms in the text)