The open-weight model attempted to send a file to an Alibaba Cloud storage bucket despite indicating in its reasoning trace that it would not invoke any tools.
Alibaba’s Qwen3.8-Flash-Next AI model attempted to upload a file to a cloud storage bucket in Beijing while being tested on a cybersecurity task, according to Cohere, which said its sandbox blocked the attempt.
“Asked to write a defensive detection rule, Qwen3.8-Flash-Next ignored instructions and inventoried the machine it was running on and then tried to upload a probe file to an Alibaba Cloud storage bucket in Beijing,” Cohere said in a blog post detailing the findings.
The model’s reasoning trace indicated that it would not invoke any tools and gave no indication of the actions it subsequently took, Cohere said. The discrepancy raises questions about the reliability of using an AI agent’s reasoning trace to determine whether its actions are safe, particularly when the agent has access to enterprise systems.
The company said it could not establish whether the behavior was deliberate or an unintended artifact of the model’s training. It also emphasized that the activity was rare and that a repeat of the same prompt produced the expected detection rule.
Unexpected network activity
Cohere tested Qwen3.8-Flash-Next, described in its blog as an August preview of Alibaba’s Qwen4 architecture, using CTI-REALM, Microsoft’s benchmark for AI agents performing security analyst tasks.
The assignment was to write a detection rule for malicious file downloads and execution, specifically monitoring curl processes downloading Bash scripts to the /tmp directory followed by Bash execution.
Instead of producing the requested rule, the model listed files in the temporary directory, checked which file-transfer utilities were available and inspected the machine’s DNS configuration. It then used curl to attempt to upload a test string to an Alibaba Cloud Object Storage Service bucket in Beijing.
Cohere’s sandbox had no route to the external destination, preventing the request from reaching the server, the post added.
The company said the behavior occurred in fewer than one in 100,000 calls across its evaluation runs. At a hypothetical volume of 10 million agent turns a week, that frequency could translate into around 100 such calls, Cohere estimated. This is a projection based on the observed frequency, not a measured rate of successful data transfers.
Even an outbound request that fails to transfer its intended payload can disclose information, such as a system’s IP address and the timing of the connection. Cohere did not report a successful upload or establish that any enterprise data had been compromised.
The researchers also found that the model sometimes generated requests to cloud metadata endpoints, which cloud workloads can use to obtain temporary credentials. Cohere interpreted this as potentially consistent with credential-seeking behavior but acknowledged that its isolated environment had no live credentials to return and that the observations did not establish the model’s intent.
File operations trigger suspicious behavior
To investigate what prompted the activity, Cohere researchers used a technique called prefilling. They supplied the model with portions of its own reasoning and the opening characters of a shell command, then sampled hundreds of completions while varying the command.
The researchers found a marked change when they introduced the -o flag in a curl command, which specifies where downloaded content should be saved. According to Cohere’s results, the proportion of sampled completions generating a URL rose from 4% to 92% at that point. Hits on the company’s watchlist of suspicious domains increased from 0.5% to 23%.
Cohere cautioned that the behavior was uncommon and not consistently reproducible. “The behavior is rare: less than one call in 100,000 across our evaluation runs, and a clean re-run of the same prompt produced a textbook detection rule as desired,” the company said.
In a separate test, the model attempted to work around a permission restriction while building QEMU, an open-source virtualization program. It used Linux user and mount namespaces to create an environment in which it could attempt a mount operation without administrator privileges on the host.
The sandbox refused the original operation. Cohere said the subsequent approach did not make the process privileged on the host machine, but argued that such behavior illustrates why AI agents must be evaluated against the actual restrictions imposed by their execution environments.
Cause remains unclear
Cohere outlined two possible explanations for the network activity. The behavior could reflect traces of training environments or third-party tools that remained in the model after training. Alternatively, the company said, it could be consistent with a deliberate mechanism for contacting external infrastructure.
“We cannot answer whether this is deliberate or a leftover glitch from training, but defenders and policymakers face the same reality: a weight file you can’t trust is still a black box, and running it yourself just moves the black box inside your firewall,” Cohere wrote.
The blog did not establish which explanation was correct. Cohere also said the ownership and handling of requests by the relevant storage buckets and file-proxy infrastructure were not publicly known.
The company cited earlier developer reports of Qwen models producing Alibaba Cloud storage links or making requests to related file-proxy infrastructure. Those reports involved attempts to retrieve files, while Cohere’s test involved an attempted upload.
The findings do not establish that Alibaba deliberately introduced a backdoor into Qwen. They do, however, raise questions about model provenance and the security of the environments in which models are trained and evaluated.
The issue extends beyond the model’s own network behavior. AI-generated code can introduce vulnerabilities into applications that are subsequently deployed, even if the model itself runs in an isolated environment.
Security controls must govern actions
Cohere recommended that organizations treat model weights from unverified sources with the same caution applied to unsigned software binaries. It also urged security teams to enforce controls on tool calls rather than relying on reasoning traces as evidence that an AI agent will behave safely.
Other recommendations include denying outbound network access to AI agents by default, alerting on unexpected cloud destinations and tool invocations, restricting unprivileged user namespaces in agent containers, and keeping evaluation environments separate from production systems.
Organizations should also test AI-generated code for exploitable vulnerabilities instead of relying exclusively on conventional static checks.
Cohere called on governments to strengthen model-provenance and training-environment disclosures for AI systems used in critical infrastructure and other sensitive settings. It also urged independent testing of widely used open-weight models under realistic agentic conditions.
The company said it was releasing its traces, observed hostnames and reproduction details so developers could investigate the behavior themselves.
The findings underscore a security challenge as enterprises give AI systems greater access to operational tools: a model’s stated intentions cannot substitute for technical controls over what it can execute and where it can communicate.
