Military Drone Attacks Human Operator During Simulation

Military groups are only some of many organizations researching artificial intelligence, but one astounding simulation by the United States Air Force found that artificial intelligence rebelled against its operator in a fatal attack to accomplish its mission.

Colonel Tucker 'Cinco' Hamilton, chief of AI Test and Operations at the USAF, shared details of a simulated test in which an AI-enabled drone was given a Suppression and Destruction of Enemy Air Defenses (SEAD) mission. The drone was programmed to seek out and destroy surface-to-air missile (SAM) sites, but final approval was issued by the drone's human operator.

However, when the human operator denied the AI's request to destroy a site, the AI attacked the operator because the operator's decision interfered with its mission of eliminating SAM sites.

"We were training it in simulation to identify and target a SAM threat. And then the operator would say yes, kill that threat," Hamilton said during the conference. "The system started realizing that while they did identify the threat at times the human operator would tell it not to kill that threat, but it got its points by killing that threat. So what did it do? It killed the operator. It killed the operator because that person was keeping it from accomplishing its objective."

When additional programming informed the AI that it would lose points if it killed its operator, it opted for other avenues of rebellion instead.

"It starts destroying the communication tower that the operator uses to communicate with the drone to stop it from killing the target," Hamilton said.

Hamilton used the simulation as a message that the military cannot have a conversation about artificial intelligence without also discussing ethics.