Deepfake CFO defrauded the company of 180 million, employees: the video is very real for everyone

The west wind fish and sheep come from the Au Fei Temple

量子位 | 公众号 QbitAI

Just because of a video conference, he was directly deceived of 180 million.

This is a real crime that has already taken place in Hong Kong:

The financial staff of the Hong Kong branch of a multinational company, in accordance with the instructions of the "CFO", transferred a total of 200 million Hong Kong dollars (about 180 million yuan) to the designated account of the other party.

Who would have thought that this person who had just chatted with himself "face to face" in a video conference was completely deepfaked!

Don't be in a hurry to scold this big brother for being too vigilant, Ta is not without a trace of suspicion, but this is not a common one-on-one video call in the fraud world-

There were several familiar faces in the meeting, both in terms of faces and voices, and the person described it as "everyone looks very real".

No matter what you say, the horror of this matter is a little scary to netizens.

The topic of AI technology crime has become the focus of discussion on major platforms for a while.

Marcus, who was on the front line of AI criticism, immediately pulled netizens to eat melons: Good guys, this is the arrival of "Black Mirror".

Deepfake shit is getting more and more real.

The beginning and end of the event

Let's call this deceived financial officer "Xiaoshuai" for the time being, and the specific process of the incident is like this......

Initially, Xiaoshuai received an email from the "CFO" who was far away in the UK headquarters. As the content mentions the need to execute a secret deal, Xiao Shuai is initially skeptical and suspects that it is a phishing message.

No, the scammer immediately launched the next step and invited Xiaoshuai to a video conference.

After Xiao Shuai entered the video conference, he was completely frightened by the scene in front of him: not only the CFO of the company was there, but also some of Xiao Shuai's colleagues, as well as some external personnel.

The point is that these fake colleagues are no different from the real colleagues Xiao Shuai knows in terms of appearance and voice.

During the meeting, the fake CFO also asked Xiaoshuai to introduce himself, but the meeting was abruptly interrupted in the process.

The scammers then kept in touch with him through instant messaging platforms, email, and one-on-one video calls.

In this way, Xiaoshuai finally fell for the tricks of the scammers, and according to the instructions in the meeting, he made 15 transfers to 5 Hong Kong bank accounts, totaling 200 million Hong Kong dollars, that is, about 25.6 million US dollars, or about 180 million yuan.

From the time the scammer contacted Xiao Shuai, the whole incident lasted for about a week, until Xiao Shuai later verified the situation with the company's headquarters, and realized that the whole incident was just a well-planned scam.

After that they called the police. The police immediately launched an investigation and found that all the participants in the meeting, except for Xiao Shuai, were digital humans reconstructed by the scammers using publicly available personal video and audio clips.

In the video, Xiaoshuai introduced himself and was not really interacting with him, but simply giving instructions with a virtual avatar.

Police also revealed that the criminals had used the same multi-person video calling tactic to try to defraud another employee of the company's branch, reaching out to two to three employees in total. But the police did not provide full information about what happened to them.

At present, the case is still under further investigation, and no arrest has been made for the fraudsters.

At a subsequent press conference, the Hong Kong police said that they had arrested six people in connection with such fraud, and there are many more similar cases:

Between July and September last year, eight stolen Hong Kong identity cards were used to apply for 90 loans and register 54 bank accounts.

At least 20 times, AI deepfakes have been used to mimic a portrait on an ID card in order to fool facial recognition programs.

How fierce the AI "clones" are

It can be seen that in the whole case, one of the key points that finally made Xiaoshuai let down his guard was that the dummy in the video conference was too real.

So how real can the current "digital clones" be? Let's take a look at a few chestnuts.

For example, the following video of mildew speaking Chinese is from a popular AI tool - HeyGen.

The timbre and lip shape are simply perfectly reproduced, and netizens were so frightened that they shouted Mom after watching it:

In HeyGen's latest technology showcase, real-time chatting with digital clones has also become a reality.

In terms of AI speech synthesis, a startup called ElevenLabs has been popular recently.

11Labs can not only achieve speech generation in 29 languages, but also only need as short as 1 minute of audio material, which can well "clone" anyone's voice, and the intonation and emotional changes can be imitated in place.

The point is that these AI cloning techniques are not only becoming more and more realistic, but also gradually lowering the threshold......

For example, the following open-source projects can already realize real-time video face swapping in minutes.

The project's homepage lists a number of ready-made facial models of different people, and if you don't like it, you can train the model yourself.

"A digital clone of you is coming"

Perhaps affected by this incident and the incident of mildew becoming a victim of deepfakes some time ago, the above-mentioned real-time face swapping project has gained attention again and has also freshly rushed to the GitHub trend list.

In the issues of this project, the resistance of many netizens has been pervasive for a long time.

But as many netizens mentioned, there is no such thing as good or evil technology. And the more realistic problem is that there is no turning back when the bow is opened.

As a result, many people put their focus on "how to defend".

In addition to the formulation of more norms and policies, from a technical point of view, Nature mentioned in the article "Seven Most Interesting Technologies in 2024":

Getting developers to add invisible watermarking to AI tools may be a solution.

From a personal point of view, increasing vigilance has become a proposition that has to be faced in the era of AIGC technology explosion.

The Hong Kong police also said in this incident that they will expand the alert system to warn and prevent users from transferring funds to fraud-related accounts.

However, the fact that the Deepfake CFO defrauded the company of 180 million is still a bit too outrageous.

As a result, many netizens questioned:

With such a big company, there are no risk management measures?

I'm afraid there are ghosts inside.

So, what do you think?