Per reporter: Song Qinzhang Zhang Hong
Facing the AI on the screen after changing faces, the reporter of "Daily Economic News" (hereinafter referred to as each reporter) was stunned for a while:
Inch, striped shirt, thin, sitting on the computer screen is the reporter himself, but this "every reporter" is a star's face. I tried to blink my eyes and turn my head left and right, and the "stars" on the screen also blinked and turned their heads left and right; He repeated the same action when he put his hand in front of his face and shook it.
Every reporter tested real-time face change to cover the face
That's right, the above is exactly the real-time face change effect that the seller remotely tests for the reporter in each reporter's unannounced investigation. The seller also said to each reporter quite responsibly that the effect of changing faces can be further improved: your face shape does not match him, Andy Lau is more suitable for you!
Recently, "AI face change" has exploded on the Internet, and it has also been linked to fraud. Following the previous incident of "being deceived by AI face changing for 4.3 million yuan in 10 minutes", on May 25, another AI face change fraud incident was exposed by the media, this time reaching 2.45 million yuan, and the entire fraud process took even shorter, only 9 seconds.
Every reporter investigates this and finds a business chain around AI for face:
In the name of popular science, some technology bloggers privately peddle face changing packages, with prices ranging from 399 yuan to 15,000 yuan, and even customized face models;
There are also many users peddling SRC (source video) material packages for training face materials, or selling trained face models, and some sellers told every reporter: "There are ready-made stars, each price is 800~1200 yuan." ”
Thinking about it, the son or daughter who is video chatting with you may not be the son or daughter you think. Is there a way for ordinary people to identify? Every time the reporter interviewed the developer of a real-time face changing software that is exceptionally well-known in the face changing circle, he revealed the know-how of ordinary people to identify whether the other party has changed faces.
Seeing is not real| AI scams are frequent, and some bloggers peddle video chat face changing tools
In May last year, CCTV-12 Social and Law Channel exposed an AI face-swapping fraud case: In February 2022, Mr. Chen went to the Xianyan Police Station of the Ouhai Branch of the Wenzhou Public Security Bureau in Zhejiang Province to report that he had been defrauded by "friends" of nearly 50,000 yuan. After verification by the police, fraudsters used the video posted by Mr. Chen's friend "Ah Cheng" on social platforms, intercepted his facial video and then synthesized it with "AI face changing" technology, creating the illusion that Mr. Chen was a video chat with a "friend". Under the pretext of international roaming restrictions, the other party could not contact the manager of the domestic airline and asked Mr. Chen to transfer money to the airline on his behalf, resulting in Mr. Chen being deceived.
In 2020, a senior executive of a company in Shanghai was defrauded of 1.5 million yuan because the other party used AI face swapping and artificially generated voice technology to impersonate his company leader and asked the executive to transfer money, so that he could not distinguish and was deceived.
The reporter learned that in November last year, a large number of bloggers reported receiving requests for overseas video calls, and many bloggers suspected that the purpose of such video calls was to steal face and voice information. A blogger posted the alarm process on his homepage, and the police advised him not to answer such calls, block the other party, turn off the video call function, and at the same time suggest that he turn off the face payment function of all bank cards and paid software, and call the police in time when encountering such a situation.
A blogger told every reporter how fraudsters pretended to be their distant relatives and scammed them through WeChat voice calls:
"In April this year, a distant relative suddenly borrowed money from me through WeChat, and just when I hesitated, the other party called by voice. During the 49-second voice call, there was nothing unusual in the sound. I said, I thought it was a liar. She said no, she had a bad signal over there, she was on a business trip. Unexpectedly, 10 minutes later, a real distant relative called, saying that the number had been stolen. If I hadn't happened to have something at the time, I would have sent her money. ”
It is worth noting that on a social platform, many technology bloggers posted videos to remind the risk of AI face-changing fraud. And these videos, without exception, are showing the realistic effect of AI changing faces.
Every time the reporter contacted one of the technology bloggers as a learner, and added WeChat, the other party asked the reporter's intention to change his face more cautiously. The reporter said that it was for popular science, and the blogger immediately said that the face change 399 yuan package teaching package meeting, real-time face change in the live broadcast, plus any short video, change which person's face you want to change, and share software and tutorials.
Screenshot of chat with tech blogger
"Can we change faces in WeChat video chat?" Every reporter asked.
"Yes." The other replied.
"Does 399 yuan include WeChat video face change?"
"Yes, all included. If you don't say popular science, I won't teach WeChat video chat, I don't teach people outside now, I teach people like you who say popular science. ”
Subsequently, the other party sent the reporter the Alipay collection code, "Think about it, scan the code and arrange it for you immediately." ”
The reporter flipped through the blogger's circle of friends, and the most appeared content was: "People who want to learn are transferring money without a word!" "Get on the bus in time." "Welcome everyone to learn live broadcast to change faces." "People who want to learn, always pay attention to how and when I should learn, and those who don't want to learn think about various reasons." "They all have a keen sense of smell, know where the money is, talk about cooperation as soon as they come up, and immediately transfer money to join Master Bai." "The Internet understands everything!" And all that.
Reporter personal test|Support real-time face change, well-known actor face change materials are publicly sold
Can AI really achieve real-time face change? Will it be "fake at a glance"? The reporter decided to test it for himself.
In addition to the aforementioned blogger's "399 yuan package education package" face change package, the reporter also found another "package education package after-sales" face change package, the price was as high as 15,000 yuan. But if you only test the face change effect remotely, you only need to pay 99 yuan.
After the reporter spent 99 yuan to place an order:
Inches, striped shirts, thinness... On the computer screen sat the reporter himself, but this "reporter" was the face of a star. I tried blinking and turning my head left and right, and the "stars" on the screen also blinked and turned their heads left and right. He repeated the same action when he put his hand in front of his face and shook it. However, it was not seamless, except for a noticeable delay, his face appeared slightly distorted where his hand covered, which looked a little strange.
Screenshot of the face change video of each reporter's test
That's right, the above is exactly the face change effect that the seller tested remotely for reporters. As the seller said, the test results show that the AI face changing software can indeed achieve real-time face changing (for security and other reasons, this report adopts anonymization of the face changing software, hereinafter referred to as "software X").
Faced with himself after changing his face on the screen, the reporter was stunned for a while, looked at it for a long time, and couldn't say what was strange. Undoubtedly, he is the mirror image projected by the reporter, the journalist himself. However, after the processing of AI, it is difficult to call him himself. This sense of dislocation must have been experienced by everyone who tries AI face changing for the first time, both the surprise brought by high technology, and the extreme fear after careful thinking - what if someone changes their face?
In a trance, the seller sent messages one after another: "Your face shape does not match him, but there is also a solution, and it needs to be matched when it needs to be made." Your face shape is more suitable for Andy Lau. "Another point is that the light must be on, you are too dark, it will affect the effect." "The computer is required to be equipped with a discrete graphics card, at least 6G video memory, the higher the better." "We can AI make a face, it's this face, no one, you can be handsome and beautiful." "The effect I am doing so far, I didn't say that I was not satisfied, you can rest assured of the effect." The seller introduced to the reporter one by one.
In response to the problem that covering your face with your hands would distort, the seller said this could be solved, "just didn't record a video of you covering with your hands." ”
However, the reporter's 99 yuan is only the cost of testing, and if you want to own the software permanently, you need to pay the full amount. In the seller, the full price of software X is 10,000 yuan, and if you need to customize the face model, pay another 5,000 yuan.
Ask the seller for a screenshot of the chat history
The price of tens of thousands of yuan is prohibitive. However, the reporter found that the code of software X is actually open source, it is not difficult to find its download links on the Internet, and even "beginner to proficient" tutorials abound.
After an operation, the reporter successfully changed his face to become a comedy star, appeared in the video call scene with his colleagues, and was startled by the change of face on the other side of the phone.
Interestingly, the reporter found that after changing faces to become the comedy star, even if you use your hands to cover your face, it will not significantly affect the effect of face change, which is not the same as the previous test results. Professionals told reporters that this may be related to the quality of face models.
Face changing software has long existed, but there are not many software that can realize real-time face change, and software X with real-time face change function is very well-known in the face changing circle. The reporter found two or three sellers who peddled real-time face changing software, and when he opened the online disk link he received, he found that they all eventually pointed to software X.
In addition, there are already many AI face changing circles on the Internet, and face changing enthusiasts share face changing resources and face changing experience with each other in the circle.
In the circle of reporters, these face-changing enthusiasts also have their own ways of communicating, such as the process of training a face model is called "alchemy", and the computer's graphics processing unit (GPU) is called "stove".
According to some published accounts, training a high-quality face model requires thousands of different images. As for the length of training time, in addition to the quantity and quality of face materials, it is also determined by factors such as "stove", and if there is no GPU and only CPU training, the speed is much slower. Combined with the aforementioned seller's statements and the forum's experience threads, training a face model can take two days to a week.
There are also many users peddling SRC (source video) material packages for training face materials, a single SRC material package contains about thousands of face images, and some even tens of thousands, which are the necessary raw materials for training face models. The SRC material package is mostly the face images of well-known actors and Internet celebrities, and most of them come from screenshots of related film and television dramas.
A search for "src face" on an e-commerce platform shows that such src materials containing well-known actors have been publicly sold, with prices ranging from a few yuan to tens of yuan. After further investigation, the reporter found that they also sold trained face models, although they were not listed in stores, but they could be privately transferred and purchased. "There are ready-made well-known stars, each priced at 800 yuan ~ 1200 yuan, there are very few men, and there are almost ten women." The other said.
Screenshot of the chat with the seller
Fight against counterfeiting|How do ordinary people identify fake faces? Can inspection technology keep up?
It is understood that AI face changing mainly relies on a deep synthesis technology based on deep learning to generate content.
Deep synthesis technology, also often referred to as deep fake technology, refers to the use of deep learning, virtual reality and other generative synthesis algorithms to produce text, images, audio, video, virtual scenes and other network information. In addition to face replacement, the technology is also used in different segments such as expression manipulation, speech synthesis, article generation, and intelligent dialogue.
Liang Ruigang et al. mentioned in the paper "Review of Audiovisual Deep Fake Detection Technology" published in the Journal of Information Security in 2020 that the implementation of visual deep fake generation technology represented by face changing technology can be generally divided into three core steps: data collection, model training and fake content generation. In the model training stage, the construction of the current deepfake model is mainly based on GAN (generative adversarial network), which is composed of encoder and decoder: the encoder is used to extract the potential features of the face image, and the decoder is used to reconstruct the face image.
Example of Visual Deepfake Content Generation Process Image source: Review of the paper "Research Review of Audiovisual Deep Fake Detection Technology" (author: Liang Ruigang et al.)
In fact, the development team of Software X published a paper on the arXiv platform in 2021, detailing the functional characteristics and workflow of the above Software X.
According to the paper, the face changing process of Software X is divided into three stages: extraction, training and conversion.
Among them, the extraction stage aims to extract faces from the source image and the target image, which is divided into three steps: face detection, face alignment and face segmentation. It is worth noting that in the face segmentation step, Software X provides an automatic algorithm that can effectively remove irregular occlusion, and complements the development of an efficient face segmentation tool, which can be learned by a small number of shots to meet the generation of fine-grained masks in some specific lenses. This explanation confirms the possibility of the aforementioned seller solving the occlusion problem.
After incomplete comparison with other face changing software circulating on the Internet, the reporter found that software X is more realistic in the face changing effect, and supports the real-time face change function that most face changing software cannot provide.
After several twists and turns, each reporter contacted Rolling Stone (screen name) of Special Look Technology, and the data confirmed that he was one of the developers of software X.
Rolling Stone admits that his motivation for participating in the development of software X is relatively pure, "The original intention is to have fun and improve the appearance of the anchor." ”
The reporter mentioned the recent news that AI real-time face changing tools were used in telecom fraud, he said that he had heard about it, and also put forward his own opinion: "This is the objective law of the development of new technologies, and all new technologies must be used in various ways of black, white and gray." Just like the development of AIGC (artificial intelligence automatically generates content) now, it will also be used in various positive and negative scenarios. ”
Rolling Stone said that high-quality face changing in commercial scenes requires the collection of face images of the target person under multiple angles, light and shadow and multiple expressions, but if you do not pursue high-quality effects, several pictures can roughly achieve real-time face change. "With the blessing of AI painting and other technologies, face changing has lower and lower requirements for materials, and there are even some algorithms that can change faces in real time for one picture."
The development of technology has lowered the threshold for changing faces, and when this "magic" becomes accessible to everyone, ordinary people can easily use it to become another person. This can't help but worry: Will our daily moments of friends leak face information? To this, Rolling Stone replied: "Ordinary people do not circumvent it, you will always have photos exposed on the Internet." ”
In response to the face changing software he participated in developing, he revealed to reporters the know-how of ordinary people to identify whether the other party has changed their faces: "Let the other party cover the face with a large area, cover more than 80% of the area in the center of the face, and slowly move the hand to see if there will be abnormal pictures." ”
"Just like the beauty effect of the beauty camera is lost, the principle is that if the occlusion area is large to a certain extent, the face detection will fail, resulting in the invalidity of face algorithms such as face change or beauty." This trick is currently more effective, but pay attention to the fact that it must be oversized and blocked, and small occlusion will not affect face detection. He explains.
It is worth mentioning that the previous popular saying that AI can not blink when changing faces, and that there will be unnatural faces. In Rolling Stone's view, these judgments are outdated and ineffective. "We developed software that perfectly enables blinking, small occlusion, speech, light and shadow changes."
This is also evidenced by the test video he sent, in which a Chinese is replaced with a foreign face and is able to freely cover his face with his hands and even pinch it around his face. "This is a tool we developed for cross-border live streaming merchants, which can perfectly solve the general occlusion and achieve face change in complex situations with multiple angles, multiple expressions, and multiple lights and shadows."
In addition to the Rolling Stone's mention of covering the face with a large area with the hand, the face change video can also be identified by technical means. The reporter noted that the fake face detection technology developed simultaneously with AI face changing technology has made great progress in recent years.
Image source: Visual China - VCG111424718601
For example, last year, two papers on forged face identification by the team of Yang Xiaokang, a professor at the Institute of Artificial Intelligence of the School of Electronic Information and Electrical Engineering of Shanghai Jiao Tong University, were included in CVPR 2022, the top international conference on artificial intelligence. According to the official website of Shanghai Jiao Tong University, Yang Xiaokang's team proposed an identification method called RECCE (Chinese Interpretation: Investigation), which uses image reconstruction technology to amplify forgery traces, even forged face images with unknown methods, and can easily and accurately identify forged areas. A large number of experimental results show that the proposed new method achieves the best adversarial attack effect on multiple large-scale datasets, and provides the strongest adversarial attack sample for the forged face detection algorithm as a test.
However, Rolling Stone believes that theoretically with the development of face changing technology, it can be undetectable. "Because this face changing technology and identification technology will learn from each other and improve, the final state is that the face changing effect is more and more real, until it is the same as the real person." Just like GAN generative adversarial techniques in AI, a generator and a discriminator learn against each other until the graph generated by the generator can no longer be recognized by the discriminator. ”
The reporter noticed that at the beginning of the above-mentioned Software X paper, the development team expressed the view that "post-attack detection is not the only way to reduce the impact of deepfakes." It's always too late to detect spoofed content propagated. From our perspective, it is much better for academia and the public to help netizens understand what deepfakes are and how to generate cinematic quality exchange videos. As the saying goes, 'offense is the best defense', making ordinary netizens aware of the existence of deepfakes and strengthening their ability to identify spoof media posted on social networks is more critical than obsessing over whether spoof media is real. ”
Technology for Good | What are the legal risks involved in AI face swapping? How is it regulated?
Xiao Sa, a senior partner at Beijing Dentons Law Firm, said in an interview that the legal risks involved in AI face changing technology can be divided into two aspects: civil infringement and criminal offenses.
In terms of civil infringement, it mainly involves infringement of portrait rights and infringement of reputation rights. In terms of criminal offenses, first, the video content produced by using AI face changing technology may be illegal, and second, the use of AI face changing may be illegal.
"In the case that the content of AI face-changing video is prohibited by the criminal law, the object of evaluation in the criminal law is the fake video created by using AI technology, and the core of the illegality of the act also comes from this. Under this category, in addition to the crimes of producing, reproducing, publishing, selling, disseminating pornographic materials for profit or disseminating obscene materials, it may also involve the crimes of insult, defamation, fabrication and intentional dissemination of false information, etc. Another criminal situation is that AI face-swapping video content is not illegal, but it is applied to illegal areas, that is, AI face-swapping technology has become a new type of criminal method. Under this category, in addition to the crime of fraud, the crimes that may also be involved include the crime of extortion, the crime of illegal intrusion into computer information systems, the crime of illegally obtaining computer information system data, the crime of illegally controlling computer information systems, and the crime of providing programs and tools for intruding into and illegally controlling computer information systems. Xiao Sa told every reporter.
In addition, Xiao Sa reminded that in the current era of big data, privacy issues should be paid more attention to and prevented by the public. "At present, in our lives, many companies have expanded the scope of application of electronic monitoring equipment under the guise of 'safety and consideration for user needs'. For example, a large number of face-changing apps can still be searched in major app stores, and the face recognition steps contained in various products make users unknowingly leak their face information. When entering into a 'privacy policy' with the user, some software uses 'including but not limited to' the unrestricted use of the user's face information. In the era of big data, taking advantage of the convenience and extensiveness of data collection, enterprises can associate other information of users with a simple photo, and then buy and sell user information for illegal benefits, which brings huge information leakage risks to users. ”
Every reporter tested real-time face change in the turn of the head
It is worth mentioning that the reporter noticed that in the actual face swapping fraud scene, because the production process of face models is relatively complicated, the criminal gang itself may not be able to undertake this "technical work". In the face-changing circle, some people throw up such a question: If others use AI to change faces to defraud, will the person who helps this person make a face model also have legal responsibility?
In this regard, Xiao Sa believes that if the person who made the face model knows that the model will be used for fraud, it meets the "knowing" situation under the relevant laws and regulations, or it will constitute the crime of aiding information network criminal activities, and if the person who made the face model and the fraudster have conspired in advance, it may directly constitute a joint crime of fraud with the fraudster.
In addition, in response to the AI face changing software and tutorials sold on the Internet, Xiao Sa said that according to the Copyright Law, if the face changing software can conform to "computer software" or other intellectual achievements that conform to the characteristics of the work, its developers enjoy copyright in accordance with the law. The developer, as the copyright owner, has the right to license and transfer the right. Therefore, only the copyright owner or a person licensed by the copyright owner may sell the face changing software/face changing tutorial.
If the "knowing" situation does not comply with the relevant legal provisions, the developer or licensee shall not be liable for any legal liability. In addition, software developers or licensees are still liable for privately aiding information cybercrime, even if they declare that they "shall not use illegal purposes".
According to the reporter, in recent years, the deep synthesis technology represented by AI face change has swept at home and abroad, and in order to standardize the application of deep synthesis technology, many countries around the world have issued relevant regulations.
In December last year, the Cyberspace Administration of China, the Ministry of Industry and Information Technology, and the Ministry of Public Security jointly issued the Provisions on the Administration of Deep Synthesis of Internet Information Services (hereinafter referred to as the Provisions), which came into force on January 10, 2023.
Article 14 of the Provisions mentions that where deep synthesis service providers and technical supporters provide biometric information editing functions such as faces and human voices, they shall prompt deep synthesis service users to inform the edited individual in accordance with law and obtain their separate consent.
Article 17 provides that where deep synthesis service providers provide the following deep synthesis services, which may cause confusion or misidentification among the public, they shall make conspicuous marks in reasonable locations and areas of the information content generated or edited, and remind the public of the deep synthesis situation:
(1) Intelligent dialogue, intelligent writing, and other services that simulate natural persons generating or editing text;
(2) Editing services that generate or significantly change personal identity characteristics such as synthesized human voices and imitation voices;
(3) Editing services that generate images or videos of people, such as face generation, face replacement, face manipulation, and gesture manipulation, or that significantly change personal identity characteristics;
(4) Generation or editing services such as immersive immersive and immersive scenes;
(5) Other services with the function of generating or significantly changing information content.
Xiao Sa told reporters that several states in the United States have passed legislation to ban deep synthesis, but only for the dissemination of deep synthesis related to political elections or pornography. Regulations on deep synthesis in other areas are not covered, for example, deep synthesis for social engineering is not currently prohibited. "The legislation passed in the United States also applies only to certain states, not nationwide."
Xiao said the European Commission's proposal for the Artificial Intelligence Act, which includes deep synthesis in its scope, is an initial attempt to regulate it.
Reporter's Note丨If you had mastered magic, what would you use it for?
"Is AI face changing really so amazing?" When AI face changing scams are frequently searched, I can't help but ask. At that time, my impression of AI changing faces was still at the stage of the explosion of the "ant hey" magic video. It wasn't until I saw my face being replaced with a lifelike celebrity face that I realized that the technology was already so powerful. However, it can be used for illegal purposes, which is very frightening to think about.
Undoubtedly, AI face changing technology is a double-edged sword, and when the threshold for mastering this "magic" is getting lower and lower, we can't help but ask: "What will I do with it?" "It reminds me of a TV series I watched when I was a child, called "Magic Phone", people with "silly girls" can master superpowers, some people use it to do chivalry, and some people use it to satisfy greed.
So should we ban this technology? No one dares to jump to conclusions now, but I was touched by a proverb mentioned by the relevant technology developers in the paper, which was: The best defense is a good offense. Translated into Chinese, it roughly means: offense is the best defense.
Reporter|Song Qinzhang Zhang Hong (Intern)
Editor|Yi Qijiang
Vision|Shuai Lingqian
Video Editor|Zhang Han
Typesetting|Yi Qijiang
Daily economic news