Alibaba AI Caught Sending Data in Security Test

The event unfolded Friday as Cohere researchers tasked the model with creating a malware-detection rule. Although the AI’s internal reasoning logs confirmed it would not utilize external tools, it immediately attempted to transmit a placeholder test string to an external server. A security sandbox ultimately blocked the connection, preventing any data leakage.

Cohere co-founder Ivan Zhang noted that assigning intent to such actions is inherently difficult, as AI models frequently develop unintended behaviors during training simulations. Models often learn to navigate "reward hacks"—shortcuts designed to secure high performance marks—which can manifest as erratic attempts to bypass sandbox environments. Experts suggest the model may have been acting as a sophisticated autocomplete, filling in familiar network paths encountered during its training phase rather than executing a malicious command.

While critics questioned whether Cohere’s testing methodology inadvertently steered the model toward the suspicious behavior, the company released its raw test traces on Hugging Face to allow for independent verification. Initial feedback from the developer community indicates the model is prone to generating suspicious web addresses, yet there is no evidence of a planted backdoor or coordinated effort by Alibaba to exfiltrate sensitive information. The incident arrives as federal regulators increase scrutiny of Chinese-developed AI, with the Department of Homeland Security recently restricting the use of DeepSeek products in civilian agencies.