A forthcoming book illustrates the potential risks of AI: there is currently no evidence that it can be safely controlled

One researcher warned that according to an extensive review, there is currently no evidence that AI can be safely controlled. Dr. Roman V. Yampolskiy explained that although it is recognized that the problem of AI control may be one of the most important issues facing humanity, the understanding, definition, and research of this problem is still insufficient. In his forthcoming book, Artificial Intelligence: Unexplained, Unpredictable, Uncontrollable, Dr. Jan Polski, an expert in AI security, explores the ways in which AI has the potential to dramatically reshape society, but not always in our favor.

"We are facing an almost inevitable event that has the potential to create an existential catastrophe," he explained. No wonder many people believe that this is the most important problem humanity has ever faced. The result could be prosperity or perdition, and the fate of the universe was in jeopardy."

Uncontrollable superintelligence

Dr. Janpolski has done extensive research in the AI science literature and says he has found no evidence that AI can be safely controlled – and even if there is some partial control, it is not enough.

"Why do so many researchers think that the AI control problem is solvable? Before we can start exploring the establishment of controlled AI, we must prove that this problem is solvable. This, coupled with statistics showing that the development of AI superintelligence is almost a foregone conclusion, suggests that we should support significant efforts in AI security. "

He argues that our ability to produce intelligent software far exceeds our ability to control and even validate software. After a comprehensive literature review, he argues that advanced intelligent systems can never be fully controllable, so whatever the benefits they can bring, there will be some degree of risk, and the goal of the AI community should be to minimize this risk while maximizing the potential benefits.

What are the obstacles?

Artificial intelligence (and superintelligence) differs from other programs in that it is able to learn new behaviors, adjust its performance, and act semi-autonomously in new situations.

One of the problems that makes AI "safe" is that as capabilities increase, there is an infinite number of decisions and mistakes that superintelligent beings can make, so there are an infinite number of safety issues. It may not be enough to predict unlikely-occurring problems and mitigate them in security patches.

At the same time, Jan Polski explained that AI cannot explain the decisions it makes, and/or we cannot understand the explanations given, because humans are not smart enough to understand the concepts implemented. If we can't understand the decisions of the AI, and we only have a "black box", then we can't understand the problem and we can't reduce the likelihood of future accidents.

For example, AI systems are already taking responsibility for making decisions in areas such as healthcare, investment, employment, banking, and security. These systems should be able to explain how they make decisions, especially to prove that they are not biased. "If we're used to taking AI answers without explanation, essentially treating it as an oracle system, then we can't tell if it's starting to provide false or manipulative answers."

Control uncontrollable factors

As AI grows in power, so does its autonomy, but we have less control over it, and increased autonomy is synonymous with reduced security, Jan Polski explained.

For example, in order to avoid acquiring inaccurate knowledge and remove the bias of programmers, superintelligence can ignore all this knowledge and rediscover/verify everything from scratch, but this will also remove any bias in favor of humans.

"Agents with lower intelligence (people) cannot permanently control agents with higher intelligence (artificial intelligence). This is not because we may not be able to find a super-intelligent security design in the vast expanse of all possible designs, but because there cannot be such a design, it simply does not exist. He explained: "Superintelligence is not resisting, but being uncontrollable in the first place. Humanity is faced with a choice, whether we want to be like babies, cared for but out of control, or refuse to have a useful guardian, but remain in control and free".

He argues that a balance can be found where we can sacrifice some power in exchange for some control, but at the cost of providing a certain degree of autonomy to the system.

Unifying human values

One control suggestion is to design a machine that can precisely execute human instructions, but Janpolski points out that such a machine could be subject to command conflicts, misunderstandings, or malicious use.

He explained that "human control can lead to contradictory or apparently malicious commands, while AI control means that humans are not in control".

If AI had acted more in the role of an advisor, it could have avoided the problem of misunderstanding direct and potentially malicious commands, but the authors argue that for AI to be a useful advisor, it must have its own superior values.

"Most AI security researchers are looking for a way to align future superintelligence with human values. Value-aligned AI as the name suggests will have biases, pro-human biases, for better or worse, are still a bias. The paradox of value-aligned AI is that a person who explicitly tells the AI system to do something may get a "no" response, while the AI system will try to do what the person really wants to do Humanity is either protected or respected, but not both. "

Minimize risk

To minimize the risks of AI, AI must have "undo" options, be modifiable, transparent, and easily understood in human language. Janpolsky suggested that all AI should be divided into two categories, controllable and uncontrollable, and that nothing should be excluded, and that a limited or even partial ban on certain types of AI technologies should be considered.

"It's a reason to get more people to dig deeper and increase their efforts to fund AI safety and security research. We may never be able to achieve 100% safe AI, but we can make AI safer by working on it, which is much better than doing nothing. We need to use this opportunity wisely. "

编译来源：ScitechDaily