The heads of the United Nations and the Red Cross called for greater controls on autonomous weapons systems — so-called killer robots — this week, warning that the technology’s advancement is outpacing regulation at a deadly rate.
“We are now dangerously close to crossing a moral red line: the autonomous targeting of humans by machines,” U.N. Secretary-General António Guterres and International Committee of the Red Cross president Mirjana Spoljaric Egger said in a joint statement released Tuesday.
But experts on the use of artificial intelligence in combat say that line already has been crossed. A Russian drone that killed three civilians in southeastern Ukraine in July appeared to have been guided to its final target solely by artificial intelligence, according to news reports.
Though the drone was programmed to travel to a particular gas station by human operators, its final target mostly likely was selected by the weapon’s own internal system.
“The weapons had no antenna for human piloting and had a powerful onboard computer with pretrained target categories,” said Stuart Russell, a UC Berkeley professor of computer sciences and president of the International Assn. for Safe & Ethical AI.
While the three sheltering civilians killed when the drone crashed into a wall appear to be the first non-combatant deaths by autonomous weapons, they may not be the first human casualties.
Ukraine conducted a one-off experimental test of fully autonomous drones two years ago that killed Russian soldiers, according to an official quoted in New Scientist magazine.
“If we need to put a date on it, [that experiment] strikes me as the Rubicon moment,” said Ryan Jenkins, a Cal Poly associate professor of philosophy and senior fellow at the Ethics + Emerging Sciences Group who studies the ethics of automated war. “I believe, unfortunately, that the line has been crossed.”
Military technology has been moving steadily toward greater automation for decades, Jenkins said. The crucial distinction of the most recent class of AI weaponry is that it removes humans from the “kill chain” entirely, instead of just relying on automation for individual functions like detecting or tracking targets.
The U.N. and Red Cross called for a ban on autonomous weapons that target humans, as opposed to incoming missiles or infrastructure targets, or that are “unpredictable,” meaning that a human operator can’t predict the outcome once the weapon is deployed.
It’s the second such appeal the organizations have made in three years. Yet meaningful, comprehensive control of AI both on the battlefield and off has languished, largely because trust between governments and understanding of the technology and its risks is lacking, Russell said.
In the meantime, autonomous weapons have only gotten cheaper and more sophisticated, making them even more attractive to states and harder to keep off the battlefield.
“Once a weapon is used in battle, and once it shows itself to confer an advantage, it becomes much more difficult for countries to give that up,” Jenkins said. “In the last decade or so since I’ve been studying killer robots, it’s often seemed to me that activists were racing to ban these weapons before they were shown to be useful on the battlefield. Now, I think those looking for a ban have basically lost that race.”
International calls for greater restrictions on autonomous weapons systems come as negotiators prepare to meet in Geneva in November to strengthen the existing Convention on Conventional Weapons. The U.N. and Red Cross say the conference “offers the clearest path available” toward binding legal rules on the use of such weaponry.
The years-long effort to rein in the use of autonomous weapons systems has struggled to produce limits. The new push comes against the backdrop of drones and other technologies used in the Russia-Ukraine war and the conflicts in the Middle East, Sudan and other places.
The Associated Press contributed to this report.
The heads of the United Nations and the Red Cross called for greater controls on autonomous weapons systems — so-called killer robots — this week, warning that the technology’s advancement is outpacing regulation at a deadly rate.
“We are now dangerously close to crossing a moral red line: the autonomous targeting of humans by machines,” U.N. Secretary-General António Guterres and International Committee of the Red Cross president Mirjana Spoljaric Egger said in a joint statement released Tuesday.
But experts on the use of artificial intelligence in combat say that line already has been crossed. A Russian drone that killed three civilians in southeastern Ukraine in July appeared to have been guided to its final target solely by artificial intelligence, according to news reports.
Though the drone was programmed to travel to a particular gas station by human operators, its final target mostly likely was selected by the weapon’s own internal system.
“The weapons had no antenna for human piloting and had a powerful onboard computer with pretrained target categories,” said Stuart Russell, a UC Berkeley professor of computer sciences and president of the International Assn. for Safe & Ethical AI.
While the three sheltering civilians killed when the drone crashed into a wall appear to be the first non-combatant deaths by autonomous weapons, they may not be the first human casualties.
Ukraine conducted a one-off experimental test of fully autonomous drones two years ago that killed Russian soldiers, according to an official quoted in New Scientist magazine.
“If we need to put a date on it, [that experiment] strikes me as the Rubicon moment,” said Ryan Jenkins, a Cal Poly associate professor of philosophy and senior fellow at the Ethics + Emerging Sciences Group who studies the ethics of automated war. “I believe, unfortunately, that the line has been crossed.”
Military technology has been moving steadily toward greater automation for decades, Jenkins said. The crucial distinction of the most recent class of AI weaponry is that it removes humans from the “kill chain” entirely, instead of just relying on automation for individual functions like detecting or tracking targets.
The U.N. and Red Cross called for a ban on autonomous weapons that target humans, as opposed to incoming missiles or infrastructure targets, or that are “unpredictable,” meaning that a human operator can’t predict the outcome once the weapon is deployed.
It’s the second such appeal the organizations have made in three years. Yet meaningful, comprehensive control of AI both on the battlefield and off has languished, largely because trust between governments and understanding of the technology and its risks is lacking, Russell said.
In the meantime, autonomous weapons have only gotten cheaper and more sophisticated, making them even more attractive to states and harder to keep off the battlefield.
“Once a weapon is used in battle, and once it shows itself to confer an advantage, it becomes much more difficult for countries to give that up,” Jenkins said. “In the last decade or so since I’ve been studying killer robots, it’s often seemed to me that activists were racing to ban these weapons before they were shown to be useful on the battlefield. Now, I think those looking for a ban have basically lost that race.”
International calls for greater restrictions on autonomous weapons systems come as negotiators prepare to meet in Geneva in November to strengthen the existing Convention on Conventional Weapons. The U.N. and Red Cross say the conference “offers the clearest path available” toward binding legal rules on the use of such weaponry.
The years-long effort to rein in the use of autonomous weapons systems has struggled to produce limits. The new push comes against the backdrop of drones and other technologies used in the Russia-Ukraine war and the conflicts in the Middle East, Sudan and other places.
The Associated Press contributed to this report.
The heads of the United Nations and the Red Cross called for greater controls on autonomous weapons systems — so-called killer robots — this week, warning that the technology’s advancement is outpacing regulation at a deadly rate.
“We are now dangerously close to crossing a moral red line: the autonomous targeting of humans by machines,” U.N. Secretary-General António Guterres and International Committee of the Red Cross president Mirjana Spoljaric Egger said in a joint statement released Tuesday.
But experts on the use of artificial intelligence in combat say that line already has been crossed. A Russian drone that killed three civilians in southeastern Ukraine in July appeared to have been guided to its final target solely by artificial intelligence, according to news reports.
Though the drone was programmed to travel to a particular gas station by human operators, its final target mostly likely was selected by the weapon’s own internal system.
“The weapons had no antenna for human piloting and had a powerful onboard computer with pretrained target categories,” said Stuart Russell, a UC Berkeley professor of computer sciences and president of the International Assn. for Safe & Ethical AI.
While the three sheltering civilians killed when the drone crashed into a wall appear to be the first non-combatant deaths by autonomous weapons, they may not be the first human casualties.
Ukraine conducted a one-off experimental test of fully autonomous drones two years ago that killed Russian soldiers, according to an official quoted in New Scientist magazine.
“If we need to put a date on it, [that experiment] strikes me as the Rubicon moment,” said Ryan Jenkins, a Cal Poly associate professor of philosophy and senior fellow at the Ethics + Emerging Sciences Group who studies the ethics of automated war. “I believe, unfortunately, that the line has been crossed.”
Military technology has been moving steadily toward greater automation for decades, Jenkins said. The crucial distinction of the most recent class of AI weaponry is that it removes humans from the “kill chain” entirely, instead of just relying on automation for individual functions like detecting or tracking targets.
The U.N. and Red Cross called for a ban on autonomous weapons that target humans, as opposed to incoming missiles or infrastructure targets, or that are “unpredictable,” meaning that a human operator can’t predict the outcome once the weapon is deployed.
It’s the second such appeal the organizations have made in three years. Yet meaningful, comprehensive control of AI both on the battlefield and off has languished, largely because trust between governments and understanding of the technology and its risks is lacking, Russell said.
In the meantime, autonomous weapons have only gotten cheaper and more sophisticated, making them even more attractive to states and harder to keep off the battlefield.
“Once a weapon is used in battle, and once it shows itself to confer an advantage, it becomes much more difficult for countries to give that up,” Jenkins said. “In the last decade or so since I’ve been studying killer robots, it’s often seemed to me that activists were racing to ban these weapons before they were shown to be useful on the battlefield. Now, I think those looking for a ban have basically lost that race.”
International calls for greater restrictions on autonomous weapons systems come as negotiators prepare to meet in Geneva in November to strengthen the existing Convention on Conventional Weapons. The U.N. and Red Cross say the conference “offers the clearest path available” toward binding legal rules on the use of such weaponry.
The years-long effort to rein in the use of autonomous weapons systems has struggled to produce limits. The new push comes against the backdrop of drones and other technologies used in the Russia-Ukraine war and the conflicts in the Middle East, Sudan and other places.
The Associated Press contributed to this report.
The heads of the United Nations and the Red Cross called for greater controls on autonomous weapons systems — so-called killer robots — this week, warning that the technology’s advancement is outpacing regulation at a deadly rate.
“We are now dangerously close to crossing a moral red line: the autonomous targeting of humans by machines,” U.N. Secretary-General António Guterres and International Committee of the Red Cross president Mirjana Spoljaric Egger said in a joint statement released Tuesday.
But experts on the use of artificial intelligence in combat say that line already has been crossed. A Russian drone that killed three civilians in southeastern Ukraine in July appeared to have been guided to its final target solely by artificial intelligence, according to news reports.
Though the drone was programmed to travel to a particular gas station by human operators, its final target mostly likely was selected by the weapon’s own internal system.
“The weapons had no antenna for human piloting and had a powerful onboard computer with pretrained target categories,” said Stuart Russell, a UC Berkeley professor of computer sciences and president of the International Assn. for Safe & Ethical AI.
While the three sheltering civilians killed when the drone crashed into a wall appear to be the first non-combatant deaths by autonomous weapons, they may not be the first human casualties.
Ukraine conducted a one-off experimental test of fully autonomous drones two years ago that killed Russian soldiers, according to an official quoted in New Scientist magazine.
“If we need to put a date on it, [that experiment] strikes me as the Rubicon moment,” said Ryan Jenkins, a Cal Poly associate professor of philosophy and senior fellow at the Ethics + Emerging Sciences Group who studies the ethics of automated war. “I believe, unfortunately, that the line has been crossed.”
Military technology has been moving steadily toward greater automation for decades, Jenkins said. The crucial distinction of the most recent class of AI weaponry is that it removes humans from the “kill chain” entirely, instead of just relying on automation for individual functions like detecting or tracking targets.
The U.N. and Red Cross called for a ban on autonomous weapons that target humans, as opposed to incoming missiles or infrastructure targets, or that are “unpredictable,” meaning that a human operator can’t predict the outcome once the weapon is deployed.
It’s the second such appeal the organizations have made in three years. Yet meaningful, comprehensive control of AI both on the battlefield and off has languished, largely because trust between governments and understanding of the technology and its risks is lacking, Russell said.
In the meantime, autonomous weapons have only gotten cheaper and more sophisticated, making them even more attractive to states and harder to keep off the battlefield.
“Once a weapon is used in battle, and once it shows itself to confer an advantage, it becomes much more difficult for countries to give that up,” Jenkins said. “In the last decade or so since I’ve been studying killer robots, it’s often seemed to me that activists were racing to ban these weapons before they were shown to be useful on the battlefield. Now, I think those looking for a ban have basically lost that race.”
International calls for greater restrictions on autonomous weapons systems come as negotiators prepare to meet in Geneva in November to strengthen the existing Convention on Conventional Weapons. The U.N. and Red Cross say the conference “offers the clearest path available” toward binding legal rules on the use of such weaponry.
The years-long effort to rein in the use of autonomous weapons systems has struggled to produce limits. The new push comes against the backdrop of drones and other technologies used in the Russia-Ukraine war and the conflicts in the Middle East, Sudan and other places.
The Associated Press contributed to this report.
The heads of the United Nations and the Red Cross called for greater controls on autonomous weapons systems — so-called killer robots — this week, warning that the technology’s advancement is outpacing regulation at a deadly rate.
“We are now dangerously close to crossing a moral red line: the autonomous targeting of humans by machines,” U.N. Secretary-General António Guterres and International Committee of the Red Cross president Mirjana Spoljaric Egger said in a joint statement released Tuesday.
But experts on the use of artificial intelligence in combat say that line already has been crossed. A Russian drone that killed three civilians in southeastern Ukraine in July appeared to have been guided to its final target solely by artificial intelligence, according to news reports.
Though the drone was programmed to travel to a particular gas station by human operators, its final target mostly likely was selected by the weapon’s own internal system.
“The weapons had no antenna for human piloting and had a powerful onboard computer with pretrained target categories,” said Stuart Russell, a UC Berkeley professor of computer sciences and president of the International Assn. for Safe & Ethical AI.
While the three sheltering civilians killed when the drone crashed into a wall appear to be the first non-combatant deaths by autonomous weapons, they may not be the first human casualties.
Ukraine conducted a one-off experimental test of fully autonomous drones two years ago that killed Russian soldiers, according to an official quoted in New Scientist magazine.
“If we need to put a date on it, [that experiment] strikes me as the Rubicon moment,” said Ryan Jenkins, a Cal Poly associate professor of philosophy and senior fellow at the Ethics + Emerging Sciences Group who studies the ethics of automated war. “I believe, unfortunately, that the line has been crossed.”
Military technology has been moving steadily toward greater automation for decades, Jenkins said. The crucial distinction of the most recent class of AI weaponry is that it removes humans from the “kill chain” entirely, instead of just relying on automation for individual functions like detecting or tracking targets.
The U.N. and Red Cross called for a ban on autonomous weapons that target humans, as opposed to incoming missiles or infrastructure targets, or that are “unpredictable,” meaning that a human operator can’t predict the outcome once the weapon is deployed.
It’s the second such appeal the organizations have made in three years. Yet meaningful, comprehensive control of AI both on the battlefield and off has languished, largely because trust between governments and understanding of the technology and its risks is lacking, Russell said.
In the meantime, autonomous weapons have only gotten cheaper and more sophisticated, making them even more attractive to states and harder to keep off the battlefield.
“Once a weapon is used in battle, and once it shows itself to confer an advantage, it becomes much more difficult for countries to give that up,” Jenkins said. “In the last decade or so since I’ve been studying killer robots, it’s often seemed to me that activists were racing to ban these weapons before they were shown to be useful on the battlefield. Now, I think those looking for a ban have basically lost that race.”
International calls for greater restrictions on autonomous weapons systems come as negotiators prepare to meet in Geneva in November to strengthen the existing Convention on Conventional Weapons. The U.N. and Red Cross say the conference “offers the clearest path available” toward binding legal rules on the use of such weaponry.
The years-long effort to rein in the use of autonomous weapons systems has struggled to produce limits. The new push comes against the backdrop of drones and other technologies used in the Russia-Ukraine war and the conflicts in the Middle East, Sudan and other places.
The Associated Press contributed to this report.
The heads of the United Nations and the Red Cross called for greater controls on autonomous weapons systems — so-called killer robots — this week, warning that the technology’s advancement is outpacing regulation at a deadly rate.
“We are now dangerously close to crossing a moral red line: the autonomous targeting of humans by machines,” U.N. Secretary-General António Guterres and International Committee of the Red Cross president Mirjana Spoljaric Egger said in a joint statement released Tuesday.
But experts on the use of artificial intelligence in combat say that line already has been crossed. A Russian drone that killed three civilians in southeastern Ukraine in July appeared to have been guided to its final target solely by artificial intelligence, according to news reports.
Though the drone was programmed to travel to a particular gas station by human operators, its final target mostly likely was selected by the weapon’s own internal system.
“The weapons had no antenna for human piloting and had a powerful onboard computer with pretrained target categories,” said Stuart Russell, a UC Berkeley professor of computer sciences and president of the International Assn. for Safe & Ethical AI.
While the three sheltering civilians killed when the drone crashed into a wall appear to be the first non-combatant deaths by autonomous weapons, they may not be the first human casualties.
Ukraine conducted a one-off experimental test of fully autonomous drones two years ago that killed Russian soldiers, according to an official quoted in New Scientist magazine.
“If we need to put a date on it, [that experiment] strikes me as the Rubicon moment,” said Ryan Jenkins, a Cal Poly associate professor of philosophy and senior fellow at the Ethics + Emerging Sciences Group who studies the ethics of automated war. “I believe, unfortunately, that the line has been crossed.”
Military technology has been moving steadily toward greater automation for decades, Jenkins said. The crucial distinction of the most recent class of AI weaponry is that it removes humans from the “kill chain” entirely, instead of just relying on automation for individual functions like detecting or tracking targets.
The U.N. and Red Cross called for a ban on autonomous weapons that target humans, as opposed to incoming missiles or infrastructure targets, or that are “unpredictable,” meaning that a human operator can’t predict the outcome once the weapon is deployed.
It’s the second such appeal the organizations have made in three years. Yet meaningful, comprehensive control of AI both on the battlefield and off has languished, largely because trust between governments and understanding of the technology and its risks is lacking, Russell said.
In the meantime, autonomous weapons have only gotten cheaper and more sophisticated, making them even more attractive to states and harder to keep off the battlefield.
“Once a weapon is used in battle, and once it shows itself to confer an advantage, it becomes much more difficult for countries to give that up,” Jenkins said. “In the last decade or so since I’ve been studying killer robots, it’s often seemed to me that activists were racing to ban these weapons before they were shown to be useful on the battlefield. Now, I think those looking for a ban have basically lost that race.”
International calls for greater restrictions on autonomous weapons systems come as negotiators prepare to meet in Geneva in November to strengthen the existing Convention on Conventional Weapons. The U.N. and Red Cross say the conference “offers the clearest path available” toward binding legal rules on the use of such weaponry.
The years-long effort to rein in the use of autonomous weapons systems has struggled to produce limits. The new push comes against the backdrop of drones and other technologies used in the Russia-Ukraine war and the conflicts in the Middle East, Sudan and other places.
The Associated Press contributed to this report.
The heads of the United Nations and the Red Cross called for greater controls on autonomous weapons systems — so-called killer robots — this week, warning that the technology’s advancement is outpacing regulation at a deadly rate.
“We are now dangerously close to crossing a moral red line: the autonomous targeting of humans by machines,” U.N. Secretary-General António Guterres and International Committee of the Red Cross president Mirjana Spoljaric Egger said in a joint statement released Tuesday.
But experts on the use of artificial intelligence in combat say that line already has been crossed. A Russian drone that killed three civilians in southeastern Ukraine in July appeared to have been guided to its final target solely by artificial intelligence, according to news reports.
Though the drone was programmed to travel to a particular gas station by human operators, its final target mostly likely was selected by the weapon’s own internal system.
“The weapons had no antenna for human piloting and had a powerful onboard computer with pretrained target categories,” said Stuart Russell, a UC Berkeley professor of computer sciences and president of the International Assn. for Safe & Ethical AI.
While the three sheltering civilians killed when the drone crashed into a wall appear to be the first non-combatant deaths by autonomous weapons, they may not be the first human casualties.
Ukraine conducted a one-off experimental test of fully autonomous drones two years ago that killed Russian soldiers, according to an official quoted in New Scientist magazine.
“If we need to put a date on it, [that experiment] strikes me as the Rubicon moment,” said Ryan Jenkins, a Cal Poly associate professor of philosophy and senior fellow at the Ethics + Emerging Sciences Group who studies the ethics of automated war. “I believe, unfortunately, that the line has been crossed.”
Military technology has been moving steadily toward greater automation for decades, Jenkins said. The crucial distinction of the most recent class of AI weaponry is that it removes humans from the “kill chain” entirely, instead of just relying on automation for individual functions like detecting or tracking targets.
The U.N. and Red Cross called for a ban on autonomous weapons that target humans, as opposed to incoming missiles or infrastructure targets, or that are “unpredictable,” meaning that a human operator can’t predict the outcome once the weapon is deployed.
It’s the second such appeal the organizations have made in three years. Yet meaningful, comprehensive control of AI both on the battlefield and off has languished, largely because trust between governments and understanding of the technology and its risks is lacking, Russell said.
In the meantime, autonomous weapons have only gotten cheaper and more sophisticated, making them even more attractive to states and harder to keep off the battlefield.
“Once a weapon is used in battle, and once it shows itself to confer an advantage, it becomes much more difficult for countries to give that up,” Jenkins said. “In the last decade or so since I’ve been studying killer robots, it’s often seemed to me that activists were racing to ban these weapons before they were shown to be useful on the battlefield. Now, I think those looking for a ban have basically lost that race.”
International calls for greater restrictions on autonomous weapons systems come as negotiators prepare to meet in Geneva in November to strengthen the existing Convention on Conventional Weapons. The U.N. and Red Cross say the conference “offers the clearest path available” toward binding legal rules on the use of such weaponry.
The years-long effort to rein in the use of autonomous weapons systems has struggled to produce limits. The new push comes against the backdrop of drones and other technologies used in the Russia-Ukraine war and the conflicts in the Middle East, Sudan and other places.
The Associated Press contributed to this report.
The heads of the United Nations and the Red Cross called for greater controls on autonomous weapons systems — so-called killer robots — this week, warning that the technology’s advancement is outpacing regulation at a deadly rate.
“We are now dangerously close to crossing a moral red line: the autonomous targeting of humans by machines,” U.N. Secretary-General António Guterres and International Committee of the Red Cross president Mirjana Spoljaric Egger said in a joint statement released Tuesday.
But experts on the use of artificial intelligence in combat say that line already has been crossed. A Russian drone that killed three civilians in southeastern Ukraine in July appeared to have been guided to its final target solely by artificial intelligence, according to news reports.
Though the drone was programmed to travel to a particular gas station by human operators, its final target mostly likely was selected by the weapon’s own internal system.
“The weapons had no antenna for human piloting and had a powerful onboard computer with pretrained target categories,” said Stuart Russell, a UC Berkeley professor of computer sciences and president of the International Assn. for Safe & Ethical AI.
While the three sheltering civilians killed when the drone crashed into a wall appear to be the first non-combatant deaths by autonomous weapons, they may not be the first human casualties.
Ukraine conducted a one-off experimental test of fully autonomous drones two years ago that killed Russian soldiers, according to an official quoted in New Scientist magazine.
“If we need to put a date on it, [that experiment] strikes me as the Rubicon moment,” said Ryan Jenkins, a Cal Poly associate professor of philosophy and senior fellow at the Ethics + Emerging Sciences Group who studies the ethics of automated war. “I believe, unfortunately, that the line has been crossed.”
Military technology has been moving steadily toward greater automation for decades, Jenkins said. The crucial distinction of the most recent class of AI weaponry is that it removes humans from the “kill chain” entirely, instead of just relying on automation for individual functions like detecting or tracking targets.
The U.N. and Red Cross called for a ban on autonomous weapons that target humans, as opposed to incoming missiles or infrastructure targets, or that are “unpredictable,” meaning that a human operator can’t predict the outcome once the weapon is deployed.
It’s the second such appeal the organizations have made in three years. Yet meaningful, comprehensive control of AI both on the battlefield and off has languished, largely because trust between governments and understanding of the technology and its risks is lacking, Russell said.
In the meantime, autonomous weapons have only gotten cheaper and more sophisticated, making them even more attractive to states and harder to keep off the battlefield.
“Once a weapon is used in battle, and once it shows itself to confer an advantage, it becomes much more difficult for countries to give that up,” Jenkins said. “In the last decade or so since I’ve been studying killer robots, it’s often seemed to me that activists were racing to ban these weapons before they were shown to be useful on the battlefield. Now, I think those looking for a ban have basically lost that race.”
International calls for greater restrictions on autonomous weapons systems come as negotiators prepare to meet in Geneva in November to strengthen the existing Convention on Conventional Weapons. The U.N. and Red Cross say the conference “offers the clearest path available” toward binding legal rules on the use of such weaponry.
The years-long effort to rein in the use of autonomous weapons systems has struggled to produce limits. The new push comes against the backdrop of drones and other technologies used in the Russia-Ukraine war and the conflicts in the Middle East, Sudan and other places.
The Associated Press contributed to this report.
