The Wiki Man

We let programmers run our lives. So how’s their moral code?

8 October 2015

2:00 PM

8 October 2015

2:00 PM

A few years ago, in the week before Christmas when supermarket sales are at their highest, staff at one branch of a leading British chain regularly did the rounds of local competitors’ shops buying up their entire stock of Brussels sprouts.

It was, in its ethically dubious way, an interesting experiment. You might assume frustrated shoppers would merely buy all the other things on their list and then go somewhere else for their sprouts. They didn’t. As the perpetrators suspected, spending 30 minutes in a shop knowing that you’ll eventually have to make a separate trip to buy sprouts feels like wasted time — so people promptly left to find a shop where they could buy everything in one place. Their branch, with its sprout monopoly, enjoyed record-breaking sales that year.

But here’s the thing. They only did this once. Wiser heads prevailed. Perhaps the management were keen ethicists who realised that sprout-hoarding would violate the categorical imperative or (more likely, I think) they were afraid they’d get rumbled. Either way, they instinctively felt the activity was wrong.

Some people will blur ethical lines occasionally; what’s rarer is when bad behaviour becomes widespread. This is what is so bizarre about the VW emissions affair. For a car company to tweak cars before measurement (‘teaching to the test’ as it’s known) might be expected; when officially testing a car’s mpg, you give it your best shot by turning off the air-conditioning and pre-charging the battery. I understand this. But to extend the ruse to a point where millions of cars carry ‘defeat devices’ seems astonishing.

Why did no one in the car industry cry foul? Contrary to what journalists think, people in large businesses usually span a wide spectrum of political views and opinions; Germans are high-minded about environmental issues to the point of sanctimony (remember the Brent Spar affair). This is a company so Teutonically perfectionist that for many years it lost countless sales in the US by refusing to add cup-holders.

Perhaps everyone believed, rightly or wrongly, they had the tacit approval of regulators for what they were doing? Or was it a case of pathological altruism, where engineers felt they were doing God’s work by reducing CO2 emissions with smaller diesels, and believed any price in other emissions worth paying? Or perhaps very few people knew what was going on?

This last possibility seems implausible — until you remember one thing. The ‘defeat device’ is not a device at all: it is some lines of additional software in the engine control unit. The addition of a physical device to cheat the test would have required approval from many different divisions of the company, some of whom would have spotted the ethical and reputational risks. But if you want to rig up some dodgy ECU software, all that’s theoretically needed is one or two unscrupulous people and a keyboard. The software they produce is largely incomprehensible to anyone else.

We are unwittingly delegating a huge amount of unscrutinised power to programmers. Once, computers largely performed dull repetitive tasks with great speed and precision; today we are delegating complex human judgments to software and to the small clique of atypical people who write code. Most do a fine job, but among them must be a few rogue actors who wouldn’t see anything wrong with buying up everyone else’s sprouts.

The truth is that most companies have little clue what is inside the software they produce. Do any of the editors of the newspapers reporting this affair have the slightest idea about the inner workings of their publications’ mobile applications, or how they handle readers’ data? I suspect not.

Got something to add? Join the discussion and comment below.

Show comments