In a recent episode of the NBC comedy The Good Place, a demon (Michael) forces a philosophy professor (Chidi) to literally live out the trolley problem — a thought experiment in which a trolley conductor must decide whether to crash into five people or switch tracks and kill just one person. Unable to overcome his usual “analysis paralysis,” Chidi inadvertently runs over one person after another until Michael finally stops the scenario. Michael says, “There have to be stakes or it’s just another thought experiment.”

As audience members, we can laugh at Chidi’s predicament. But soon, we could find ourselves facing the same predicament in our real lives.

How Will AI Respond In A Crisis?

If advances in artificial intelligence (AI) proceed at the pace promised by its most optimistic proponents, hundreds of thousands of self-driving vehicles may be traveling the highways in a decade or so. But how will that AI be programmed to respond to a crisis, especially the kind of no-win scenario dramatized in the trolley problem?

Let’s say, for example, that your autonomous vehicle is driving at high speed along a busy road when a child darts out in front of you. There’s oncoming traffic in the left lane, and cars are following close behind you.

Will the AI be programmed to prioritize the safety of the car’s passengers and, therefore, run over the child as the “least bad” option? Will it prioritize the lives of pedestrians over passengers? Will it be programmed to sacrifice adults in favor of children? What if it’s somebody’s pet that runs in front of the car? Are animals expendable, even if the alternative is a mere fender bender?

Currently, many people in the automotive and tech industries are downplaying the likelihood of real-life trolley problems — if they even think about the issue at all. Some of those who do think about it believe that AI will prevent such no-win scenarios by ensuring that vehicles’ computer and mechanical systems are hyper-vigilant and properly maintained.

Others (like me) believe that while AI will dramatically reduce accident rates, it will never be able to prevent all accidents.