As much as anything, it’s the relationship between these human operators and their subject that’s most disturbing. Thought experiments like the trolley problem demonstrate something self-evident but extremely significant in moral thinking: how our sense of obligation is modified by distance and immediacy. We feel differently about pushing someone off a bridge with our own hands than we do about pushing a button to achieve the same result. Weapons are real-world examples of this: the arrow differs from the fist, the rifle from the arrow, the bomb from the gun. And by the time you reach autonomous systems, many of the “actions” setting them in motion will have taken place months or even years ago: as part of a process of programming and design whose eventual consequences may be entirely unknowable.
If all this sounds rather removed from real life, it’s worth remembering that the cutting edge of today’s battlefield will represent little more than the state-of-the-art in domestic appliances a decade from now. Such is the speed of technological change, and the steady migration of its moral conundrums towards the mass market.
It’s a field in which the ghostly outlines of future controversy are already clearly visible. If a drone can oversee your child’s journey to school today, will it be driving you both to work tomorrow – and should you insist on this, if the machine is demonstrably a safer driver than you? If robot soldiers can fight for your country without risking your sons’ and daughters’ lives, should you gratefully embrace automated warfare?
These are new questions, but answering them means grappling with ancient ethical issues about responsibility. What does it mean to hold someone responsible not only for their actions, but for the chains of consequence they set snaking through the world? And what does it mean to exercise human ingenuity responsibly, given the ever-expanding scope of our powers?
Ironically, the best contemporary answer may mean giving up some of that power – and asking what it means to create machines that can take responsibility for themselves.