Global Courant
The US Air Force has performed what it now calls a “thought experiment”. where a drone is controlled by artificial intelligence. The drone was supposed to destroy an air defense system. The drone was awarded points for successfully completing its mission. Sometimes the human operator would send instructions to the drone not to destroy the air defense system, meaning the drone would not earn any points. The drone found a solution: it destroyed the communications tower and directed the drone commands to override its mission, “killing” the operator.
The drone had learned not to kill the driver, as it would lose points.
But the drone ignored his instructions and decided that the mission was more important than the points.
The airforce officially denied early reports that musings on such a scenario were tested in a simulation phase where the drone “killed” its operator.
The screenplay was anticipated in the 1987 film Robocop, perhaps the best sci-fi film of the 1980s, perhaps the best ever.
The Robocop story is about a Detroit police officer who is murdered. He is revived by a private company, Omni Consumer Products, owned by Megacorporation. Omni Consumer Products has been tasked with maintaining law and order in Detroit because the police in that city failed to do so. (If failure to enforce law and order sounds familiar, it should.)
The newly minted cyborg, Murphy, is given four orders. Three of the orders fulfill Omni Consumer Products’ Detroit-based mission: to serve the public’s trust, protect the innocent and uphold the law. A fourth injunction is classified: it is not to act against an Omni Consumer Products executive.
In the meantime, the Cyborg begins to regain some of his lost memory and humanity. When he tries to shoot the chief of Omni Consumer Products, who has committed murder, the fourth secret directive comes into effect. The Murphy cyborg hesitates but can’t override the built-in directive. In that tense moment, the CEO is fired by the company’s board of directors so that Cyborg Murphy can attack him, which he does.
Robocop cannot single-handedly overcome the Fourth Secret Directive, which makes the outcome different from the Air Force simulation, in which the drone disobeyed its commands.
If the Air Force wants to move from the imagination phase to simulation, it will have to try to figure out how the drone, which has no human memory because it is not human, could manage to kill its operator. Possibly this would prove to be a challenging task. The short-term solution would be to strengthen the operational rules for the drone. For example, if the drone tries to attack its operator, it will be destroyed. Other quick fixes are also possible.
The main point is that artificial intelligence is in very tricky waters: while it does not strictly encompass human memory, it does involve human thought patterns that are mimicked. Even the developers and programmers may not understand exactly what they have built into these modern cyborgs.
Artificial intelligence is still in its infancy and has many civil and military applications. The more complex these systems become, the more they can become autonomous and even independent of their programming.
While Robocop introduces a human dimension to Murphy as he rediscovers his human past, he still cannot overcome the Fourth Directive.
The satire in the movie, of course, relates to the extreme violence used by Robocop, both as Murphy the cop and Cyborg Murphy.
When the Cyborg is asked his name in the film’s final scene, he replies “Murphy”. But this Murphy is not human after all, as evidenced by his inability to break away from the Fourth Directive.
Real or imaginary, the Air Force’s experimental drone is much worse because it can break rules and make decisions on its own. Has the drone become a kind of human?
The Air Force may want to look into Robocop.
Stephen Bryen is a senior fellow at the Center for Security Policy and the Yorktown Institute. This article was originally published about his substack, weapons and strategy. Global Courant republishes it with permission.
Similar:
Loading…
