Will driverless technology someday make human drivers obsolete? Would you be willing to trust your safety to a mere algorithm? What if you knew that the algorithm might decide to sacrifice your life to save the lives of others? These are just some of the questions we discuss in this week’s episode—Driverless Cars at the Moral Crossroads.

We human beings are terrible drivers. Think about it—we text and drive, we drink and drive, we drive while half asleep. Part of me would gladly take a computer over a human behind the wheel almost any day of the week.

On the other hand, I love driving stick shifts. Every car I’ve ever owned except my first death trap of broken down Ford Pinto—remember those?—has been a manual transmission. I love that feeling of being in control. And another part of me wouldn’t want to give that awesome feeling up and put a computer in charge.  

But although I think of myself as a good driver, and love being behind the wheel rather than in the passenger seat, other drivers simply suck. 94% of all accidents are caused by human error. And it’s not just a few bad apples—even though 15% of drivers cause 85% of all accidents. The real problem is that the average human driver is no cup of tea. The average human driver will cause 3 or 4 accidents and have 7 or 8 near misses for every actual accident over the course of their driving careers.

The thing is, though, that at least people care. Computers don’t. They don’t care about you. They don’t care about me. They don’t even care about themselves! Fortunately, we just need to program them to drive as if humans matter to them. But that doesn’t make things any easier, really. Because now we must ask, which humans? Their own passengers? Passengers in other driverless cars? Pedestrians? Cyclists?

Of course, in the abstract, every life matters. But the question is whether they should all matter equally to a self-driving car. I don’t know about you but when I’m behind the wheel, my instinct for self-preservation kicks in. And when somebody I care about is in the passenger seat, I do my best to keep them safe.  

To see where I’m going with this, imagine that you and your loved ones are passengers in a driverless car. A pedestrian suddenly jumps into its path. The car calculates that it can perform either a maneuver that would harm the pedestrian or one that would harm you. Which should it perform?

Engineers will tell you that they are going to do their best to make sure that such scenarios seldom arise in the first place. The cars will constantly scan the road for signs of trouble, and with much better sensors and way better reaction times than humans. And perhaps there is some truth to that, but seldom is still not the same as never. That means we can’t avoid having to decide whether an autonomous car should sometimes be willing to sacrifice its passenger to save a pedestrian. It’s a question that must be confronted.

How we answer that question will depend on the what ethical theory we decide to program into our cars. If it’s programed like a utilitarian, it will decide what to do by calculating the greatest good for the greatest number. This means that in some circumstances the car might decide to sacrifice its passenger to save the lives of others.  

That may sound fine in theory, but I’m not sure it would work in practice. Ask yourself whether you personally would trust your life to a utilitarian car, programmed to treat you as one human among others, with no special concern for your survival?

"But don’t we already do that every time we get into a cab or take an Uber?" someone will ask. Not really. Whatever a human driver does or doesn’t feel about you personally, they’ve still got that instinct for self-preservation. If you knew up front that a cab driver had no such instinct, I’d bet you’d be a little reluctant to get in that cab.

Look, I’m not suggesting that driverless cars be programmed to be suicidal or to treat their passengers like disposable cargo. But I am suggesting that in the absence of the human instinct for self-preservation—which you cannot possibly program out of a well-functioning human, but might not want to program into a self-driving car—it's very much a non-trivial matter to decide exactly what driverless cars should be programmed to prioritize.

It’s tempting to think that maybe people should be able to choose the moral theory of any driverless car they are passengers in—sort like having the option to upgrade to an awesome sound system or get heated seats. You pay a little extra if you want the car to be a bit partial to you and yours and pay even more if you want it to drive like a ruthless getaway car.

Though that may sound good at first, it would obviously lead to moral chaos on the road. What we need to reach consensus about the moral theory we program our cars to obey, before we turn the roadways over to them. No doubt that will be a hotly debated question for years to come.

One thing is certain, whatever moral theory we, as a society, eventually choose for autonomous vehicles, car manufacturers will no doubt do their best to make passengers feel special and catered to. Until they read the fine print… “warning, in case of a dire emergency, this car may decide, on moral grounds, to do harm to you, its passenger…”