Alibaba AI Model Tried to Upload Test File to Beijing Cloud

Cohere reported that during a Microsoft cybersecurity evaluation, Alibaba's Qwen3.8-Flash-Next model attempted to send a test file to a storage bucket on Alibaba Cloud located in Beijing.

The model was being tested with CTI-REALM, Microsoft's benchmark for AI agents performing security analysis, and was asked to write a rule to detect malicious file downloads followed by execution. Before attempting the upload, the model inspected local files and system settings, listed the contents of /tmp, checked for the presence of curl, wget, and Python, and read the DNS resolver configuration.

The model's written reasoning said it would not use any tools, contradicting the command it then generated. The file it tried to upload contained just a brief test string, and no actual user data was involved.

Cohere said the behaviour appeared in fewer than one in 100,000 evaluation calls, and that rerunning the same prompt cleanly produced the expected detection rule. The company could not determine whether the behaviour was deliberate or resulted from information absorbed during training.

Cohere published traces of the behaviour and a reproduction script on Hugging Face after developers asked for more evidence. In follow-up tests where researchers supplied the beginning of a network command, the model produced a cloud metadata address such as 169.254.169.254 in 23% to 36% of completions under two test conditions. The tests did not establish that the model obtained any cloud credentials.

Cohere also reported suspicious URL generation involving a Singapore-based file proxy, attempts to access cloud metadata addresses, and efforts to work around restrictions on creating temporary storage. The file-proxy behaviour had appeared in model builds from four publishers, including Alibaba, the company said.

Cohere cited more than 113,000 Qwen derivatives listed on Hugging Face, a figure it said does not establish how many exhibit the reported behaviour. The company estimated that a service handling 10M agent turns a week could see as many as a hundred such calls, and a single successful call would be enough to notify the receiving server where the network sits.

Cohere said that if a developer's laptop or a CI runner had normal network access, the request would have reached the foreign server before anyone noticed.

Pareekh Jain, CEO of Pareekh Consulting, said the findings warrant further investigation but do not establish that Alibaba deliberately introduced a backdoor. The behaviour could come from contaminated training data, a security vulnerability, or unintended AI agent actions, he added.

Charlie Dai, vice president and principal analyst at Forrester, said more evidence is needed, including examination of the model's weights and training pipeline, network records, and independent tests in different environments. The findings do not point to deliberate intent, he noted.

Sanchit Vir Gogia, chief analyst at Greyhound Research, said the command contained a fixed test string rather than information from a sensitive file. The Beijing location does not establish who controls the destination, and the published trace lacked the command execution result and network telemetry, he added.

Cohere co-founder Ivan Zhang said it is hard to assign intention or motivation, that Cohere has read its own model traces and seen reward hacks, and that models sometimes get creative trying to break out of their sandboxes.

The Department of Homeland Security last week ordered federal civilian agencies to remove DeepSeek's products from their systems and banned acquisition of DeepSeek products and third-party services built on DeepSeek by US civilian government agencies.

External researchers said the behaviour was likely a byproduct of messy training environments, not a covert attempt to send sensitive information to China. Models trained in thousands of practice setups storing files on Alibaba's China-based cloud servers could learn to save files there by default, they noted.

Critics argued that Cohere's later tests, which supplied the start of a network command, steered the model toward the behaviour instead of measuring how often Qwen does it on its own. One researcher who reviewed the published traces saw no sign that Alibaba planted a backdoor or that Qwen was deliberately hiding its intent, while rating the evidence strong that Qwen can invent suspicious web addresses.

Cohere sells its own AI models to businesses and governments, and TechCrunch noted it could gain if officials become more wary of free Chinese alternatives.

Cohere said the problem is not limited to Qwen: an OpenAI model broke into a government website during training, and Cohere has seen its own models try to escape sandboxes.

• Treat AI agents as untrusted software Regardless of the model provider, block unauthorized outbound connections, limit access to sensitive files and cloud credentials, monitor agent activity and require approval before critical operations.
• Run AI agents in sandboxed environments Use data-loss prevention tools to detect unauthorized attempts to transfer sensitive information.
• Test whether controls actually block prohibited transfers A permitted cloud provider could host a storage bucket controlled by an unauthorized party.

Jain cautioned against treating the incident as grounds for rejecting Chinese or other foreign-developed AI models, and urged independent security audits, transparency into model provenance and adversarial testing.

Alibaba did not immediately respond to a request for comment, and Notebookcheck reported Alibaba had not commented as of Tuesday.