For this year's technology industry, the hottest is definitely ChatGPT, even if you haven't experienced it, at least you have heard of it, it has indeed brought some revolutionary changes to our production and life. But more and more people are also beginning to feel a little worried about the rapid development of ChatGPT, in response to this topic, today we will talk about AI black boxes, for most people, the term "black box" is reminiscent of the recording equipment on airplanes, if something incredible happens, these devices are valuable for accident analysis. But what everyone doesn't know is that black box is also an important term in the world of artificial intelligence.
AI black boxes refer to AI systems that operate internally and are invisible to users. You can provide them with input and get output, but you cannot view the system's code or the logic that produces the output. Machine learning is the main subset of artificial intelligence, which supports generative AI systems such as ChatGPT and DALL-E2. Machine learning has three components: an algorithm or set of algorithms, training data, and a model. An algorithm is a set of programs. In machine learning, an algorithm learns to recognize patterns after training on a large number of examples (training data). Once a machine learning algorithm is trained, the result is a machine learning model. Models are used by people.
For example, a machine learning algorithm can be designed to recognize patterns in images, and the training data could be images of dogs. The resulting machine learning model will be a dog recognizer. You can feed it an image as input and get an output of whether a set of pixels in the image represents a dog and its location.
Any of the three components of a machine learning system can be hidden, or placed in a black box. Typically, algorithms are public, which makes putting them in a black box less effective. Therefore, in order to protect intellectual property, AI developers often put models in black boxes. Another approach taken by software developers is to obfuscate the data used to train the model, in other words, put the training data into a black box.
The opposite of the black box is sometimes referred to as a transparent box. An AI transparent box is a system whose algorithms, training data, and models can be viewed by anyone.
The importance of AI black boxes
In many cases, there are good reasons to be wary of black-box machine learning algorithms and models. Let's say a machine learning model diagnoses your health. Do you want the model to be a black box or a transparent box? What about the doctor who prescribed you? Perhaps he wondered how the model arrived at its decisions.
If a machine learning model that decides whether you qualify for a commercial loan from a bank rejects you, don't you want to know why? If you know why, you can appeal the decision more effectively or change your situation to improve your chances of getting a loan next time.
Black boxes also have an important impact on software system security. For years, many in the computing space believed that putting software in a black box would keep it safe by preventing hackers from checking it. This assumption has largely proven wrong, as hackers can reverse engineer software (i.e., build simulations by closely observing how the software works) and discover vulnerabilities that can be exploited.
If the software is in a transparent box, software testers and well-meaning hackers can check it and inform the creators of weaknesses, thus reducing cyberattacks.