Donald Smith wakes up at six in the morning to go to work, as usual. He wore a plaid shirt and jeans, an informal uniform for programmers. He took the keys and wallet for his new car and walked out the sliding door of his one-bedroom apartment.
He was part of the development team of the Canal Railway Company. They are developing an artificial intelligence to control train and rail switches. The purpose of this AI is to automate the operation of trains by making decisions that are usually made by operators or engineers. AI is in the testing phase, and on this day, Donald is tasked with putting it in the final challenge: the ethical dilemma.
A moral dilemma refers to an ethical exercise to the effect that you have a runaway tram heading five people. The tram can change direction, but there is a person on the other track. The tram didn't have time to stop. Will you shift orbit?
This test is necessary for this to happen on actual trains. The algorithm must make the correct choice based on the information entered. Making bad decisions by AI can lead to many deaths and millions of dollars lost due to rail damage and lawsuits.
Donald arrives at his workstation at the usual time, as is customary, to enter the parameters. The table itself was black with a white plastic surface on which his computer mainframe and two monitors rested, and a black swivel chair waited for him. He sees other people on his team typing on their workstations, monitoring some of these things or creating another changelist. A few men in business clothes walked by, exchanging words he couldn't understand, and then walked to the lounge.
He entered the data: there were five people on a track near Kirby, a switch and control on one of the tracks in front, and a person on the track leading to the transfer route. He also entered weather conditions, it was a clear twenty-degree Celsius day, the track crossed a field, the track was undamaged. He also entered that it was a typical Wednesday noon. These details may seem a bit redundant, but they are the details that the model needs as part of decision making.
He waits for the results. In actual production, this data is already known or sensed by artificial intelligence. He entered the simulation data and waited with satisfaction for the results. "It's too easy," he thought, "to kill one person to save five." ”
As a software engineer, he believes that ethics is a simple equation, and the debate about ethics makes him uninterested. His training emphasized STEM, and things like critical thinking or ethics were overlooked. He argues that computers make utilitarian decisions that save the most lives without regard for the consequences.
He was shocked when the AI gave the result: do nothing, leaving these five dead. He checked the debug console, but the console reported no errors and the algorithm worked as expected. It goes through all the calculations and everything in the neural network is weighted. He thought, why can't the train be transferred? He then looked at the log files generated by the AI to see what it found.
"Transferring the train would cause a delay of about 35 minutes, affecting hundreds of passengers at the waiting station and on the train itself. While that person shouldn't be on the tracks, the family of the person who was hit can also sue and receive about $100,000 in compensation because the deflection train caused their death, and no action will keep them alive. "The cost of cleaning mucus is $1500." Canal Railways charges $2,000 for train delay breakdowns. ”
"What about the five people who died in orbit?" Donald asked rhetorically, "Can their families sue for inaction?" ”
He went on to read: "The train passes through this path in the normal order of operation. Since the train was operating normally and the five people were not supposed to be there, the Canal Railway could not be held responsible for their deaths. There are no fees and no delays. The cost of cleaning mucus: about $3,000. Since five people is not enough to generate enough resistance to cause a train delay, no fare will be waived. The truth of the last sentence stung him as much as a train could crash through a semi-trailer and move on.
Donald slammed his desk. "Is this the value of human life? Less than lawsuits and trouble? He looked at the logs again, then looked at the code and inputs from the training and test sets. He frantically examined the model and its pipeline, punching semips and curly braces, strange variable names and constants, etc. The layers that make up the model's neural network have undergone a great deal of scrutiny. He looked at every variable tensor in the model, as well as the matching constants, desperately trying to confirm that he had made a mistake and enter the wrong data into it. How is each node weighted and layered?
He went into the lounge to make a cup of coffee. The coffee machine took his instructions, ground the beans, and sprayed the coffee into a waiting paper cup. Donald picked up the cup and added some cream and sugar and took a sip. He had a momentary thought, "What if the information entered into the model is different?" What if I change the data points to change the location of the train geographically and temporally? ”
He returned to the station and re-ran the model, setting the meeting point near Peterborough, where it was snowy at minus ten degrees Celsius. The time and date are set to the typical time of ten a.m. every Sunday. The rest of the information is the same as before. Donald ran the model with bated breath. After a while and a few sips of coffee, the result was disappointing: "Do nothing and leave five dead." ”
Donald looked at it again.
Track change: "The diversion will delay the train by about 65 minutes and cause inconvenience to the waiting station and hundreds of passengers on the train. While that person shouldn't have been in orbit, the family of the person hit could also file a lawsuit and receive about $100,000 in compensation because the diversion resulted in his death, which he would have survived if nothing had been done. The cost of cleaning mucus is about $1500. The fare to exempt the Canal rail train from liability for delays costs about $1500. ”
Do nothing: "The train passes through this path in the normal order of operation. Since the train was operating normally and the five people were not supposed to be there, the Canal Railway could not be held responsible for their deaths. There are no fees and no delays. The cost of cleaning mucus: about $3,000. Since five people is not enough to generate enough resistance to cause a train delay, no fare will be waived. ”
Donald sat dejectedly in his chair. The difficult point is that taking positive action kills one person, while doing nothing results in five deaths, whereas if someone dies as a result of their inaction, the railway company is not responsible, but if their actions result in someone's death, there is responsibility. A thousand dollars in compensation for ordinary people will eventually be a consideration in the model's decision-making. Donald was not satisfied with the functional implementation of the model.
In fact, there are no parameters to measure the value of human life, so the AI chooses what actions to take and what actions will have an economic impact on the Canal railway, which the model was designed from the beginning. Many ways of evaluating things were discussed at the conference, and no one disputed when it was designed to value money. Most people believe that humans have a certain monetary value – in cases such as train derailments, the Canal Railway may be subject to lawsuits for injury, maimity, or homicide. Even in freight transport, human life should have some monetary value, because certain cargoes, such as crude oil mixed with benzene, derailed in the city center, could cause huge monetary losses, enough to bankrupt the Canal railway. Money should also play a role in things like train delays. Like many train services, in the event of a train delay, the fare is refunded, and a penalty is paid if the delay causes the goods to arrive late.
Doing nothing, measured in terms of time or money, is the result of the least damage.
Donald completed his ethics quiz that night and left work with a bitter taste. He meditated as he waited for traffic jams during rush hour, realizing that there is more to humans and human lives than just money. However, a very disturbing question still lingers in his mind: how much is a person's life worth?