From face leaks to "mobile phone eavesdropping", in the era of explosive growth in the application of AI technology, our personal data is being comprehensively "encircled and suppressed"; The global cybercrime is large enough to be the world's third-largest economy, while the cybersecurity technologies and regulations that protect us have always lagged behind cyberattack technologies.
Written by | Du Baojie
Recently, the Shanghai Municipal Public Security Bureau's move to prohibit hotels from conducting "mandatory face scanning" verification on travelers who have shown their ID cards has received a lot of praise online. In fact, not only Shanghai, but also Shenzhen, Hangzhou, Suzhou, Xi'an, Sanya and other cities have begun to implement relevant policies.
According to relevant reports, the hotel is not allowed to be forced to "brush face", which is a notice issued by the Ministry of Public Security last year, but the implementation progress is different in various places. As early as August 2021, the Personal Information Protection Law of the People's Republic of China established strict ex-ante information processing rules and administrative, civil and even criminal liability divisions for the protection of facial recognition information, which required information processors to process facial recognition information only if it meets the "specific purpose" and "sufficient necessity" and takes strict protective measures. On August 8, 2023, the Cyberspace Administration of China (CAC) issued the Provisions on the Security Management of the Application of Facial Recognition Technology (Trial) (Draft for Comments), which once again emphasized the relevant aspects, and put forward the principles of minimum use, voluntary, and minimum storage of facial data.
During the two sessions of the National People's Congress in 2024, some representatives submitted the "Proposal on Restricting the Excessive Use of "Face Recognition" in Tourism Scenarios", bluntly saying that there is no relevant legal basis for hotels to install face recognition equipment on their own, and it is recommended that investigation and equipment management be carried out as soon as possible. The Regulations for the Implementation of the Law on the Protection of Consumer Rights and Interests, which will come into force on July 1, 2024, also clearly state that business operators shall not excessively collect consumers' personal information, and shall not use methods such as one-time general authorization or tacit authorization to compel or indirectly compel consumers to consent to the collection and use of personal information that is not directly related to business activities.
The era of "face brushing" has brought a lot of convenience, but it has also made people suffer from "face brushing" for a long time, which is not new in the technology world. Because since the birth of related technologies represented by AI, user privacy issues have been accompanied. In the early stage of the savage growth of technology, economic interests and user experience come first, businesses avoid or ignore privacy, and users are not sensitive enough to privacy protection, which indirectly promotes the rapid development of the entire technology industry. However, when the technology develops to a mature stage and forms a huge industrial scale effect, privacy protection has become a relatively lagging problem.
The update of laws and regulations has been catching up with the pace of technological development, and the potential privacy risks of computer vision technology represented by "face brushing" are only the tip of the iceberg.
The shadow zone of face recognition
At present, there are many types of mainstream biometric technologies, including face, fingerprint, voiceprint, iris, gait recognition, vein recognition, etc., which have been widely used in identity recognition, such as security, access control, attendance, operating system (computer, mobile phone and network software operation), smart door lock and other hardware, financial payment and public safety management (such as aircraft, high-speed rail and various hotels, public venues), etc.
As one of the widely used branches of technology, face recognition is almost ubiquitous in our lives today, from the monitoring network all over the city, to the small smartphones, smart watches and APP, each of us has closely bound the face image and identity and property security, which brings a lot of convenience and also produces a lot of risks. In recent years, there have been frequent data leaks of companies related to face recognition at home and abroad, and the potential hidden dangers should not be underestimated.
In February 2019, SenseNet, a subsidiary of Oriental Netpower, a listed domestic security company, was exposed to a hacker attack on its database, and more than 2.5 million pieces of personal data were leaked; In February 2020, a major data breach occurred at Clearview AI, an American facial recognition app, and even more outrageously, the company provided facial recognition systems to more than 600 law enforcement agencies, including police and other law enforcement agencies, as well as banking systems. Similar data breaches happen almost every year, even some of the most secure tech giants.
Behind face recognition, along with the increasingly powerful gray industry chain, many face information and user privacy data are bought and sold in the underground market. A better use might be to be sold to data companies for AI training, analyzing user attributes, generating virtual identities, etc.; The terrible thing is that the data will also flow into illegal channels to carry out high-tech fraud, or through "photo activation" and other means to crack the identity authentication of financial clients, fraudulently use identity to transfer loans, and even be used for deepfake video calls, rumor spreads and production of pornographic videos, posing a great threat to social security and trust. A lot of face-related data that has been leaked or traded comes from weak links such as hotels, supermarkets, and stores that lack protection capabilities.
European and American countries that attach great importance to the issue of personal privacy data have already begun to carry out relevant technical control. In January 2020, the draft of the EU's "Artificial Intelligence White Paper" banned the use of facial recognition technology in public places for the next 3 to 5 years, and then in 2021, the European Commission first proposed the "Artificial Intelligence Act", which adjusted real-time remote biometrics such as facial recognition from the "high-risk" level to the "prohibited" level, and passed the bill by an overwhelming vote in June 2023, which is regarded by the industry as the world's first comprehensive artificial intelligence legal framework. It provides a reference system for the formulation of AI laws and regulations in many countries.
The United States, which is relatively advanced in AI technology and has stronger monitoring penetration capabilities, is considered to be the first country in the world to enact a biometric privacy protection law, but at present, the country does not have a unified and comprehensive federal AI law to regulate the collection and use of biometric data, but since 2020, many state and municipal regulations have begun to ban facial recognition technology in public places, and they have been implemented in San Francisco, Boston and other places. In March 2024, the Biden administration announced a new regulation on the use of artificial intelligence (AI) by U.S. federal government agencies, urging the federal government to use AI safely, securely, and responsibly, and not to infringe on people's rights and safety.
In addition, the open gun is easy to dodge and the dark arrow is difficult to prevent, in addition to those who can clearly perceive the face collection process, there are also some malware or applications to take a "silent candid photo" way to quietly sneak into the device to steal user information, once the user opens the login interface, the software will immediately detect the device model, and then call the front or rear camera to take silent photos, the whole process will not emit prompt sounds and flashes, you can easily avoid the user's attention, and a large number of user albums, text messages, call records, contacts and other data will be collected.
Whether it is for individuals or countries, the privacy data of citizens' visual identity has had an immeasurable impact on the security and stability of society. In addition to the introduction of relevant laws and regulations, many AI labs and research teams are following up on countermeasures to protect face privacy and security, developing face anonymization technology or anti-face recognition technology, including the IDeudemon method at Shanghai Jiao Tong University, K-Same at Carnegie Mellon University and K-Same-Net optimized by the University of Ljubljana, and the video developed by Facebook's AI Lab De-identification technology. It is also urgent to promote the better industrialization of related technologies.
Covert cell phone eavesdropping technology
In addition to worrying about the risk of face recognition, the smartphone with the highest daily use is likely to be the hardest hit area by privacy leakage. Many people have such an experience: just chatting with their family to discuss what to buy recently, opening the shopping software and being accurately recommended related products, is the mobile app a mind reader? Or is some software "eavesdropping"?
From a technical point of view, although there is no eavesdropping behavior of mainstream compliance apps, mobile phone eavesdropping itself is not a magical technology, as long as there is electricity and network, even if the mobile phone is locked or turned off, powerful hackers can "listen" to the user's speech, and even if you turn off the microphone permission, it is also working - this is technically completely achievable.
Song Yuhao, co-founder of KEEN, a well-known white hat company in China and a security expert in GeekPwn Labs, once said in a program interview that behind accurate recommendations is not voice monitoring, but mainly due to user big data analysis and AI-driven accurate advertising recommendations, but if APP manufacturers really want to monitor you, there are no barriers from a technical point of view.
As long as the APP upgrades some hidden components and tries to open the microphone permission of the mobile phone, it can directly collect and upload audio, or it can take a more hidden way, such as converting the collected voice data recognition into text, and then sending it to the cloud by extracting text keywords, analyzing text features in the cloud and combining user identity, information search and browsing records, etc.
As early as 2014, at the GeekPwn Security Geek Contest, the technical staff of the information security research team KeenTeam demonstrated the whole process of Android phones being eavesdropped by hackers in the shutdown state, hackers can try to make the phone enter a black screen "fake shutdown" state, which looks like it is turned off, but the listening program running in the background is still silently stealing text messages, contacts, call logs, location information, call recordings and even candid images, etc., and uploading them to the server. Unless the phone's battery, SIM card, etc. are removed, and all the functions of the phone's devices are physically cut off, many components can operate in a low-power state, continuing to provide opportunities for hackers.
Today's AI technology can recognize dozens of dialects and multiple Chinese, and a person's words spoken in a day are organized into text, at most a few pieces of paper, for the computing power processing of the smartphone side has no pressure, but the formal software development team generally does not take the legal risk of "listening to users" and bear huge business losses to carry out this "low-level" operation, these illegal speculations are more likely to exist in some uncertified junk software, so it is necessary to remind everyone, Don't be tempted to download unwanted software and third-party malicious programs onto your device.
In addition, there are a lot of magical ways to listen. In 2019, a study published by the University of Cambridge Computer Lab in which researchers designed an AI algorithm to restore the user's input on the phone by listening to the sound made by the user tapping the phone screen. The researchers used this to listen to the sound of people clicking on different parts of the screen, and analyzed the sound wave data generated when they clicked on different areas of the screen, although the model is still in the experimental stage, in the experiment, after entering the 4-digit password 20 times, the algorithm has a correct recognition rate of 61%.
In 2020, researchers from Zhejiang University, McGill University in Canada, and the University of Toronto jointly published a paper in which they mentioned a new "side-channel" smartphone eavesdropping tool based on deep learning accelerometer signals, and they used the sound vibration signal emitted by the speaker to convert the accelerator data into speech signals, and the success rate of the experimental data reached 90%.
Accelerometers and other sensors are generally not restricted by the permission control of the operating system, and any App can apply for use, so the use of sensors to obtain voice data has become a novel way to eavesdrop, but the technical threshold is higher.
And a more powerful way of listening can be scary. Do you still remember the "Prism Gate" incident that was exposed in 2013? Snowden, a former CIA employee, revealed that the National Security Agency (NSA) and the Federal Bureau of Investigation (FBI) have implemented a top-secret electronic surveillance program codenamed "Prism" since 2007, spying on 122 foreign leaders, and using tools that can also hack into the servers of tech giants such as Microsoft, Google, Apple, and Yahoo to monitor the targeted citizens' emails, chat logs, videos, photos and other private information.
This kind of international spy network can be called one of the important tools in the offensive and defensive warfare of modern information networks, and it may be ubiquitous, and with the popularization of artificial intelligence technology, the level of such surveillance networks has become more powerful.
In 2021, a "Pegasus" spyware developed by the Israeli company NSO Group broke through the latest version of the iOS and Android mobile phone system protection at that time, hiding in the phone and turning the phone into a "24-hour monitoring device", hackers can remotely extract user text messages, photos and emails, record calls, and remotely activate the phone's microphone and camera to obtain environmental information without the user's knowledge.
It is understood that the "Pegasus" software is used in at least 50 countries around the world to monitor activities, affecting as many as 50,000 people, although Apple said in a response at the time that the above attack methods are very sophisticated, usually requiring millions of dollars to listen to specific targets, but according to relevant reports, NSO Group has sold software services for hundreds of millions of dollars, and it is still profitable to develop such evil technology.
After the Prism Gate incident, Apple's mobile phone was also banned by many national political departments, no more, that is, many mobile phone manufacturers and communication manufacturers have the ability to remotely spy on mobile phones and obtain information, if they have secret cooperation with intelligence and espionage agencies, the consequences are unimaginable.
I have to say that we don't have permanent security protection, at most, that is, continuous upgrading of security protection technology and continuous improvement of information security regulations and regulatory agencies cooperate with each other to provide a guarantee as much as possible. For the majority of ordinary consumers, the awareness of security risks also needs to be continuously strengthened, minimizing the leakage of key personal information in an unfamiliar environment, strengthening password management, regular software updates, and maintaining a high degree of vigilance against unknown requirements and information requests.
The confrontation with the forces of evil is endless
As far as the industry is concerned, the scale of development of the AI security industry is far from enough.
According to the latest reports from Techopedia and IDC, the global AI market in cybersecurity is expected to reach $133.8 billion by 2030, and the global generative AI industry will exceed $150 billion. But the worrying comparison is that cybercrime is expected to cause a total of $9.22 trillion in global Internet user losses in 2024, and that number could be close to $14 trillion by 2028. If measured by a country's GDP, the global cybercrime volume is enough to become the world's third-largest economy after the United States and China.
A cybersecurity market report by Cybersecurity Ventures predicts that global spending on cybersecurity products and services will exceed $1 trillion between 2017 and 2021. In fact, with the explosive growth and penetration of the AI industry in recent years, many analysts expect such spending to far exceed the $1 trillion annual level by 2025, and a lot of it may be spent in the wrong areas.
Today's cybersecurity threats and privacy intrusions have expanded from targeting and harming computers, networks, and smartphones to people, cars, railways, airplanes, power grids, industrial facilities, and any distributed Internet of Things (IoT) device with signal output or electronic pulses, all of which make the cybersecurity landscape increasingly large and complex.
Big data is the cornerstone of today's AI industry and digital economy, and the opportunities for innovation and malicious behavior around data are immeasurable. As it stands, most cybersecurity budgets or regulations are linear or flat with technological developments, but the number of cyberattacks is growing exponentially. Based on this, security protection should start from the top-level design as soon as possible, predict technical risks in advance from multiple directions and dimensions, and give measures such as security protection, risk avoidance, talent cultivation, and crime crackdown as soon as possible, so as to avoid falling into the dilemma of "bringing a knife into battle during a shootout".
bibliography
[1]https://www.secrss.com/articles/25405
[2]https://zhuanlan.zhihu.com/p/150528851https://www.mittrchina.com/news/detail/4864https://www.jiemian.com/article/3090493.html
[3]http://cjc.ict.ac.cn/online/bfpub/pcl-2023413151919.pdf
[4]https://www.shanghai.gov.cn/nw4411/20240423/42496d66bfe0461e8740f8431906edf8.html
[5]https://www.spp.gov.cn/spp/llyj/202109/t20210908_528821.shtml
[6]https://zh.wikipedia.org/zh-cn/%E9%A3%9B%E9%A6%AC_(%E9%96%93%E8%AB%9C%E8%BB%9F%E4%BB%B6)
[7]https://cybersecurityventures.com/hackerpocalypse-cybercrime-report-2016/?ref=hackernoon.com
[8]https://www.techopedia.com/ai-names-biggest-cybersecurity-threats?ref=hackernoon.com
[9]https://www.idc.com/getdoc.jsp?containerId=prCHC51997124
[10]https://cybersecurityventures.com/cybersecurity-market-report/
This article is reprinted with permission from the WeChat public account "Science China".
Special Reminder
1. Enter the "Boutique Column" at the bottom menu of the "Huipu" WeChat official account to view a series of popular science articles on different themes.
2. "Back to Park" provides the function of searching for articles by month. Follow the official account and reply to the four-digit year + month, such as "1903", to get the article index in March 2019, and so on.