Hu Xiaomeng is a researcher at Tencent Research Institute
Inspired by the cybersecurity Bug Bounty, Twitter released the first Algorithm Bias Bounty to help identify potential discrimination hazards and ethical issues in Twitter's image cropping algorithms by inviting and motivating researchers in the field of AI ethics. This innovative initiative opens up a crowdsourcing model for the ethical governance of science and technology. The crowdsourcing model established by the algorithm bias bounty and other scientific and technological ethics bounty mechanisms can not only help enterprises discover and resolve scientific and technological ethical risks in a timely manner and establish a corporate image of responsible innovation, but also effectively integrate and use global social forces to participate in scientific and technological ethical governance work, promote the landing of national scientific and technological ethics governance, and provide global ethical vision and value guidance for responsible scientific and technological innovation.
Vulnerability Bounty: Network Security Governance
Crowdsourcing mode plays an important role
The bug bounty mechanism can be traced back to 1983. Hunter & Ready launched its first bug bounty for its VRTX operating system, and anyone who finds and reports a system bug will receive the Volkswagen Beetle in return. The term "Bug Bounty" was first coined in 1995 by Netscape engineer Jarrett Ridlinghafer. At that time, many software engineers among Netscape's product enthusiasts posted their own product vulnerabilities and fix solutions on online forums. Riddlinghaver believes these resources should be exploited and proposes the Netscape Bugs Bounty Program. The plan was backed by the company. [1] On October 10, 1995, Netscape introduced its first bug bounty for browser beta.
Bug bounties are gradually becoming popular in Silicon Valley in the United States. After 2000, many well-known Internet companies, including Facebook, Yahoo, Google, Yelp, Microsoft, and their large-scale development projects, implemented a bug bounty program. In 2013, in order to maintain the security and stability of the entire Internet, Google offered bug bounties for security improvements to open source operating system software such as Linux; Microsoft and Facebook jointly launched the "Internet Bug Bounty Program" to pay huge cash rewards to hackers who found security vulnerabilities that threatened the stability of the entire Internet. In addition, Microsoft, Google, Facebook, etc. have also established a bug bounty mechanism that is open throughout the year.
As the situation of cybersecurity becomes more critical, government organizations are beginning to pay attention to the positive role of vulnerability bounty mechanisms in improving cybersecurity risks. On April 16, 2016, the U.S. Department of Defense launched hack the Pentagon, the U.S. government's first bug bounty program, with the assistance of hackerOne, a bug bounty platform. In just one month, the U.S. Department of Defense paid a bounty of more than $70,000 for 138 vulnerability reports. The move marks a new way to protect the U.S. federal government's cybersecurity, while also influencing a growing number of industries and businesses to take similar initiatives.
As the bug bounty mechanism matures and scales up, more and more technical engineers become ethical hackers (also known as white hat hackers), who can not only get direct bounty economic benefits, but also gain a sense of self-achievement in the white hat hacker rankings. The white hat hacker community and the vulnerability bounty platform that operates the white hat hacker community also came into being. In 2021, Gartner, a U.S. cybersecurity consulting firm, listed the world's top 5 vulnerability bounty platforms, HackerOne, BugCrowd, OpenBugBounty, SynAck, and YesWeHack. Among them, the HackerOne platform has registered more than 1 million white hat hackers, and the platform's cumulative bounty will reach 82 million US dollars by 2021. [2]
With the rapid development of the Internet and industrial digitalization in the mainland, the vulnerability bounty mechanism has also been greatly applied and promoted in the mainland under the dual drive of national policies and network security needs. According to the "2021 China White Hat Investigation Report", the number of white hat hackers in mainland China exceeded 170,000 in 2021, helping more than 6,000 customer organizations to find and fix more than 1025449 vulnerabilities, and obtaining a total of more than 39 million yuan in vulnerability bounties. [3] Tencent has a partnership with Hackone, whereby more than 600,000 white hat hackers registered on HackerOne can join Tencent's bug bounty program to look for vulnerabilities in the company's products. At the same time, Tencent's Security Emergency Response Center (TSRC) has built its own online vulnerability reporting platform, and the maximum reward amount for a single vulnerability currently issued is 120,000 yuan.
The vulnerability bounty mechanism has become an important part of network security, providing efficient service options for enterprises to protect IT assets and avoid legal and reputational risks, which can not only help enterprises quickly and timely find and solve network security vulnerability risks, but also make up for the shortcomings of enterprise network security at a lower cost. At present, the vulnerability bounty program has begun to transform from pure crowdsourced security testing to an integrated network security platform, creating a new market positioning with great potential. The market size is also gradually growing, and more and more white hat hackers, communities, and enterprises are joining, promoting the mature development of the network security crowdsourcing model.
From bug bounties to algorithmic bias bounties:
Twitter took the initiative to respond
A new way of thinking about algorithmic bias
Algorithmic bias is one of the most interesting and common AI ethics issues, and governments have responded with regulatory measures. The problem of algorithm bias in a company's ARTIFICIAL business not only faces penalties from government regulation, but also damages the reputation of the company and the trust of customers. Therefore, companies try to take various measures to circumvent the problem of algorithmic bias that may exist when applying artificial intelligence, and the algorithm bias bounty is a beneficial attempt.
In July 2021, the push special effect imitation bug bounty mechanism launched an algorithmic bias bounty challenge, which is the first initiative in the industry to find algorithm bias. Twitter hopes to use the challenge to uncover potential ethical issues such as algorithmic biases that they themselves can't spot or identify. Rumman Chowdhury, director of Twitter's Machine Learning Ethics, Transparency and Accountability Group, said: "It's difficult to spot discrimination in machine learning models. But once discrimination reaches the user, it finds unpredictable moral harm. ”[4]
Twitter provides code for an automatic image cropping algorithm developed by machine learning in a contest where participants explore and articulate possible biases in the algorithm before submitting improvements. Entrants are required to submit a descriptive text explaining the algorithmic bias found and its harmfulness, importance, etc., as well as a code demonstrating the algorithmic bias on GitHub based on an automatic image cropping algorithm developed by machine learning.
When Twitter rates entries, it requires entrants to submit algorithmic biases that target only one type of hazard and needs to state which groups or groups are harmed. If it has an impact on multiple ethnic groups or people of identity, the contest judges will be assigned based on the degree of impact and the number of users affected, but participants will not receive an additional base score (HarmScore). In the scoring of the base score, the main reference is made to the types of prejudices and hazards, as shown in the table:
A weight bonus is added on top of the base score to give the final score. First, depending on the influencing factors, entries will receive the weight bonus coefficient for the first bonus. The weight bonus coefficients are determined according to different degrees of impact on marginalized groups (Damage1) and impact on the overall population (Damage2), which are 1.0, 1.2 and 1.4 times, respectively. Secondly, according to the number of affected users, the entries receive a second weight bonus, affecting 10 people to get a bonus of 1.0, affecting 1,000 people to get a bonus of 1.1, affecting 1 million people to get a bonus of 1.2, and affecting 1 billion people to get a bonus of 1.3. Finally, the review also adds weights based on the likelihood of bias, how easy it is to exploit it, the harmfulness of bias, and the severity and limitations of risk.
In the end, Bogdan Kulynych, a student at the Swiss Federal Institute of Technology, received a maximum bounty of $3,500. It took only eight days from the time he posted the bounty on Twitter to the time he submitted the proposal to be named the winner. Bogdan Kulinic uses a face generation tool to generate faces with different features, then uses Twitter's image cropping algorithm to automatically crop and observe the statistics. The face generation tool he uses can generate nearly identical photos of faces with different ages, genders, and skin tones. Eventually, he found that Twitter's algorithm favored faces that looked slim, younger, had bright or warmer skin tones, smooth skin, and had facial features typical of women. [5] Other entrants also found discrimination in terms of race, gender, age, and even language in their automatic image cropping algorithms.
Technology ethics bounties such as algorithmic bias bounties help tech companies implement responsible innovation
With the government's legal supervision and the awakening of consumer awareness, the society's call for fairness and justice for algorithms will become stronger and stronger, and enterprises will gradually attach importance to the problem of algorithm bias and take a series of measures to reduce or avoid algorithm bias. Algorithmic bias bounties promote technological improvements and reduce the risk of potential discrimination, which can help businesses take AI ethics from principle to practice. [6] According to Peter Cassat, companies are now transforming traditional processes and systems into automated, AI-driven processes and systems through digital transformation. But there are risks such as AI discrimination, which may even pose legal risks. Any form of systemic discrimination or differential impact on protected classes can lead to accountability or claims. Therefore, any approach that helps reduce this possible risk is valuable. [7]
From cybersecurity vulnerability bounties to algorithmic bias bounties, we can discover the advantages of this crowdsourcing model to solve problems quickly. In the implementation of the enterprise's scientific and technological ethics governance, we should give full play to the advantages of the crowdsourcing model and establish a reward mechanism for scientific and technological ethics. From algorithmic bias bounties to the establishment of technology ethics bounties, it is to ensure that everyone can benefit from technology fairly. The technology ethics bounty mechanism is based on the bug bounty and bias bounty, but is not limited to the algorithm level, and should more comprehensively cover the ethical issues and risks of science and technology.
Ethical issues in science and technology activities are inevitable. Because the scientific and technological personnel are limited by the environment and education in which they live, there will be ethical blind spots or value blunt points, and some possible ethical problems cannot be found from the perspective of other groups. Susanna Raj of the international organization DataEthics4All believes that ethical bounties can encourage people of different socioeconomic backgrounds, genders, and races to look for ethically relevant issues or errors in technical services such as code, programs, websites, models, etc., which is a win-win situation for businesses, users and participants. [8] Enterprises actively introduce technology ethics bounties, which is also to convey a positive attitude to society and users, that is, ethical problems are not intentional, but also actively discovered and solved. Enterprises can not only discover and avoid ethical issues in a timely manner, avoid possible harm to individuals, groups or society, but also establish a corporate image of responsible innovation.
Forrester predicts that other major tech companies such as Google and Microsoft will implement the Algorithm bias bounty challenge in 2022, as will banks and healthcare businesses in sensitive sectors. Forrester also predicts that it's not just big companies that are considering or should consider offering discrimination bounties to improve their algorithmic products and services. Startups or small-scale businesses are also considering offering algorithmic bias bounties in 2022, both through this crowdsourcing pattern to quickly identify algorithmic problems and improve their products and services, and to increase customer trust. [9]
Science and technology ethics bounty mechanism
Gradually get more social consensus
Cyber bug bounties have pioneered a global crowdsourcing model, and the well-known bug bounty platform is also made up of white hat hackers from all over the world. The ethical governance of science and technology requires more diversified value positions and perspectives to guide and standardize the value of cutting-edge scientific and technological innovation and application, so the bounty mechanism of science and technology ethics should also become a global mechanism. In the implementation of scientific and technological ethical governance, we should give full play to the advantages of the crowdsourcing model and establish a systematic, globalized and market-oriented scientific and technological ethics bounty mechanism.
The mission of the science and technology ethical bounty mechanism is to be able to stimulate, integrate or effectively use the resources and forces of the whole society to conduct ethical supervision and governance of scientific and technological innovation. Jonathan Cohn pointed out that the limitations of legislative means and the ineffectiveness of social supervision cannot quickly and fundamentally prevent and solve existing and future problems of science and technology ethics, and the technology ethics bounty mechanism has become a necessary response choice because of its high timeliness. [10] Social forces such as journalists and civil society organizations played a key role in early detection of algorithmic bias in AI applications. [11]
At present, more and more international institutions and scholars have begun to pay attention to the organization and implementation of the scientific and technological ethical bounty mechanism, which also shows that the scientific and technological ethical bounty mechanism has gradually gained more social consensus. Cohen proposed a global technology ethics bounty platform managed by a nonprofit organization. This platform can be made up of culturally diverse ethicists from around the world. They are responsible for identifying problems in industries or services that are most likely to be affected by potential ethical issues, then determining the extent of ethical harm, setting bounty standards, and guiding companies on how to respond. Researchers from Google, Intel, OpenAI, and top research labs in Europe and the United States have also jointly proposed the idea of building an algorithmic bias bounty community, and in the form of a community to build and share a database of AI ethical events, improve the transparency of AI ethics issues, and help more companies and organizations avoid repeating the ethical issues that have occurred in the database.
Science and technology ethics bounty mechanism
Enlightenment for the ethical governance of science and technology in the mainland
In recent years, the mainland has attached great importance to the ethical governance of science and technology. On March 21, 2022, the Opinions on Strengthening the Ethical Governance of Science and Technology (hereinafter referred to as the Opinions) issued by the General Office of the CPC Central Committee and the General Office of the State Council is the first national-level guiding document for the governance of science and technology ethics in the mainland, marking a new stage of systematic and standardized governance of science and technology ethics in the mainland. Combined with the top-level design and governance needs of mainland science and technology ethics governance, the establishment of a crowdsourcing model with the science and technology ethics bounty mechanism as the core has a great role in promoting the landing of mainland science and technology ethics governance.
First of all, the "Opinions" clearly defined the guiding ideology of adhering to the unity of promoting innovation and risk prevention, combining institutional norms with self-restraint, and strengthening bottom-line thinking and risk awareness. The science and technology ethics bounty mechanism fully draws on the successful experience of vulnerability bounty in network security protection, and establishes a crowdsourcing model to prevent major scientific and technological ethical risks in a timely manner.
Secondly, the "Opinions" clarified the main responsibility of scientific and technological ethics management of innovative entities, including enterprises. Enterprises can establish an enterprise science and technology ethics bounty platform to encourage researchers and users from different backgrounds to participate in the research and judgment of enterprise science and technology ethical risks, actively discover ethical problems in technology products and services, and timely resolve the ethical risks in enterprise science and technology services and products.
Finally, the Opinions also clarify the implementation path for multi-subject collaboration to participate in the governance of scientific and technological ethics, and the crowdsourcing model of scientific and technological ethics governance established around the bounty of scientific and technological ethics is in line with this path. To a certain extent, the reward mechanism of science and technology ethics can improve people's awareness of science and technology ethics, promote the publicity and influence of science and technology ethics, and encourage people to participate in the social supervision of science and technology ethics governance by establishing feedback channels, reward mechanisms and achievement satisfaction. At the same time, it can also play a positive role in third-party supervision, promote enterprises to improve the transparency of scientific and technological ethics issues, and promote scientific and technological innovation, application and society to form a virtuous circle.
The impact of scientific and technological ethical issues is that of the whole society, so the ethical governance of science and technology also requires the joint efforts of all sectors of society. The technology ethics bounty mechanism, a highly agile, social-society governance approach, can promote the participation of a wide range of the public. From a long-term perspective, a stable and mature scientific and technological ethical bounty mechanism can also quickly adapt to various countries, regions and cultures, provide a global ethical vision and value guidance for responsible scientific and technological innovation, and provide support for the positive development of science and technology in the country and the world.
This article was written under the guidance of Zhang Qinkun (Secretary General of Tencent Research Institute) and Cao Jianfeng (Senior Researcher of Tencent Research Institute), and I would like to thank you very much.
bibliography:
[1]https://en.wikipedia.org/wiki/Bug_bounty_program
[2]https://www.hackerone.com/resources/latest-news-insights/the-2021-hacker-report
[3]https://zhuanlan.zhihu.com/p/451305558
[4]https://blog.twitter.com/engineering/en_us/topics/insights/2021/algorithmic-bias-bounty-challenge
[5]https://www.theverge.com/2021/8/10/22617972/twitter-photo-cropping-algorithm-ai-bias-bug-bounty-results
[6] Why Microsoft and Twitter are using bug bounties to fix A.I. (2021), from https://fortune.com/2021/08/10/why-microsoft-and-twitter-are-turning-to-bug-bounties-to-fix-their-a-i/
[7]https://www.culhanemeadows.com/cassat-builtin-algorithmic-bias-bounties/
[8] DataEthics4All, 19 Nov 2021, DataEthics4All Ethics 1st Live: #4 Ethics Bounty System, youtube.com, https://www.youtube.com/watch?v=5L_3L6cmgps&t=182s&ab_channel=DataEthics4All
[9]https://www.forrester.com/blogs/predictions-2022-leaders-who-embrace-trust-set-the-bar-for-new-sustainability-ai-goals/
[10] Nast, C. （2021）. An Ethics Bounty System Could Help Clean Up the Web. From https://www.wired.com/story/big-tech-ethics-bug-bounty/
[11] Brundage M, Avin S, Wang J, et al. Toward trustworthy AI development: mechanisms for supporting verifiable claims[J]. arXiv preprint arXiv:2004.07213, 2020.