AI Hallucination Nearly Triggered Military Strike on Chinese Vessel

The incident occurred when a Special Operations Command analyst tasked a chatbot with merging open-source data and classified signals intelligence. The system hallucinated the ship’s manifest, and the analyst subsequently used the tool to format the inaccurate data into a formal report. This erroneous summary circulated through command channels before reaching the mission-ready stage.

This near-miss highlights the tension between the Pentagon’s push for high-speed decision-making and the risks posed by unverified machine output. While military leaders view AI as a critical tool for accelerating the "kill chain" against rivals like China, the episode exposes how quickly automated errors can bypass human oversight. Jake Steckler, a research scholar at GovAI and a U.S. Army veteran, warned that prioritizing adoption speed over rigorous verification could erode trust in these systems, potentially undermining the very military advantage the technology is intended to provide.