Military and AI: The Debate Over Autonomous Weapons and Legal Compliance

<figure> <img alt="A four-legged robot with armaments on top walks on a gravel surface." src="https://platform.vox.com/wp-content/uploads/sites/2/2026/09/gettyimages-2273123489.jpg?quality=90&amp;strip=all&amp;crop=0,0,100,100" /> <figcaption> The new version of SARBOT, Turkey’s first armed robot, is demonstrated in Duzce, Turkey, on April 29, 2026. | Omer Urer/Anadolu/Getty Images </figcaption> </figure> <p class="wp-block-paragraph">The question of who has the authority to refuse military orders has gained significance as the Pentagon has encouraged AI companies to relax restrictions on the use of their technologies in warfare. This includes the extent of human oversight required for autonomous weapons. When Anthropic declined to comply with these requests, it lost a significant government contract, while OpenAI accepted the Pentagon's more lenient standards. Recent reporting from The Intercept sheds light on the negotiations between OpenAI and the Pentagon.</p> <p class="wp-block-paragraph">The agreement between OpenAI and the Department of Defense (DOD) included specific language regarding the AI models to be provided: “OpenAI models that are designed for national security use cases and have minimal refusal rates.” This term suggests that the Pentagon is looking for models that infrequently refuse commands. A Justice Department lawyer representing the Pentagon confirmed this language but later retracted that confirmation.</p> <p class="wp-block-paragraph">In a military context, where orders are typically followed without question, the expectation is that soldiers comply with commands from their superiors, who are presumed to have more experience and knowledge. However, soldiers are legally required to refuse illegal orders. The introduction of technologies with limited refusal capabilities may complicate the prevention of war crimes.</p> <h2 class="wp-block-heading"><strong>Legal Obligations of Soldiers</strong></h2> <p class="wp-block-paragraph">According to U.S. domestic, military, and international law, soldiers must follow lawful orders and have a duty to reject unlawful ones. The Department of Defense’s Law of War Manual clearly states that each service member must comply with the law of war and refuse to follow illegal orders. Disobeying an illegal order can expose both the service member and their commander to criminal liability.</p> <p class="wp-block-paragraph">This principle of disobedience is rooted in early British and American common law and gained international recognition following the Nuremberg Trials after World War II. During these trials, many Nazi soldiers claimed they were not responsible for their actions because they were following orders. However, the court rejected these defenses, affirming that obedience has limits, especially when an act is clearly illegal.</p> <figure class="wp-block-pullquote"><blockquote><p>Obedience is not without limits, especially when a wartime act would be clearly illegal to the subordinate or any person of “ordinary sense and understanding.”</p></blockquote></figure> <p class="wp-block-paragraph">The question arises whether the same principle applies to AI systems or AI-assisted weapons. Unlike human soldiers, AI systems do not have the capacity to reject commands based on moral or legal grounds; they operate according to established protocols.</p> <h2 class="wp-block-heading"><strong>AI Decision-Making and Legal Compliance</strong></h2> <p class="wp-block-paragraph">While some argue that AI could interpret and apply the law more effectively than human soldiers, the complexity of legal interpretation is context-specific. AI systems may struggle with nuanced legal distinctions, such as differentiating between lawful and unlawful targets in combat situations.</p> <p class="wp-block-paragraph">Rebecca Crootof, a technology law expert, noted that while the language in the agreement could suggest a higher likelihood of compliance with unlawful orders, it does not guarantee that AI systems would comply with all commands. Identifying unlawful orders can be challenging, even for human soldiers.</p> <p class="wp-block-paragraph">The power of soldiers to disobey orders serves as a critical check on military authority. Replacing soldiers with machines that lack this ability could remove an important layer of judgment in military operations. However, AI systems may not be equipped to make the necessary legal judgments.</p> <h2 class="wp-block-heading"><strong>Potential Solutions</strong></h2> <p class="wp-block-paragraph">Crootof suggests that rather than programming AI to refuse commands, it may be more effective to design AI systems to flag ambiguous situations for human review. This approach could maintain a check on military authority while ensuring that complex legal decisions are made by humans.</p> <p class="wp-block-paragraph">As AI technologies become increasingly integral to military operations, it is essential for the public to understand how these systems are designed and their implications for military decision-making.</p> <p class="wp-block-paragraph"></p> SOURCE: Vox PUBLISHED: 2026-09-22 18:10:00