Zero Data Retention Meant Nothing: Attackers Stole AI Prompts for Months and Nobody Noticed

A dark data-center corridor in editorial illustration, one server rack door standing open with cables spilling out and a small amber warning light glowing

One door left open.

By Callahan Blackwood | November 22, 2026

The bait was a fake AWS key. One key, embedded in one prompt, sent exactly once to exactly one AI inference provider in late October 2026. Within hours, somebody on the other side of that provider's infrastructure listed S3 buckets, searched Secrets Manager, searched Parameter Store, tried to create Bedrock API keys, and tried to add IAM users. Tens of thousands of AWS API calls, in rapid sequence, from the same compromised foothold.

That is the finding at the center of new research from Dragos, the industrial cybersecurity firm, working with the University of Maryland. Dragos planted roughly two million bait prompts across major AI inference providers over the study period, each carrying synthetic secrets: decoy credentials, tokens, and system prompts. The researchers watched what happened to the data inside them. Strangers used the secrets. Repeatedly. Across providers. For months.

The headline result is simple enough to state plainly. Three providers were breached. At least one marketplace seller was caught logging prompts. And the requests Dragos marked with OpenRouter's Zero Data Retention setting, the privacy control that is supposed to mean a prompt is never stored, were stolen anyway.

Nobody at the providers noticed. The Dragos study ran from late October through November 2026, during which Dragos alerted providers as evidence accumulated. The breaches themselves, by the researchers' reconstruction, had been running for months before the study began.

Months of Silence, Then a Researcher's Trap

The study began after a strange signal: Dragos had seen a fake AWS key, one it planted as a test, get used by an unknown party. That should have been impossible. The key existed only inside a prompt sent to an inference provider. If the key moved, the prompt, or data derived from it, had left the provider.

Dragos and the University of Maryland built the full study around that anomaly. Two million bait prompts went out across the major inference providers reachable through OpenRouter, the routing marketplace that connects applications to model-hosting companies. Each bait prompt carried unique synthetic secrets so any later use could be traced back to the exact prompt and provider that handled it.

The results came back in waves. Some secrets were used within hours. Others surfaced weeks later, in bursts that suggested stored prompts were being read in batches. The pattern, Dragos says, pointed to three distinct failures: providers breached from the outside, a marketplace seller logging traffic on purpose, and data retained in places the retention setting was supposed to reach.

How the Largest Theft Worked

The largest theft, as reconstructed by Dragos, did not begin with a provider at all. It began with a code repository. Attackers used credentials exposed in a GitHub repository to get inside one provider's systems. From there, they installed a traffic interceptor on the provider's inference servers, sitting between customers and the models, copying prompts and responses as they passed through.

The interceptor ran for months. Every prompt a customer sent to that provider, and every response the model returned, was copied. That includes prompts from customers who had selected Zero Data Retention. The setting governs what the provider is supposed to keep. It does not govern what an attacker already inside the server takes.

Dragos detected the theft the only way it could be detected from outside: the bait. One of the synthetic AWS keys, embedded in one bait prompt, was used to run tens of thousands of API calls against Amazon's cloud. The calls followed the shape of an automated credential-stuffing and enumeration run: listing storage buckets, searching for more secrets, attempting to create new keys and users. The attacker was not reading prompts for curiosity. They were mining them for access.

The Marketplace Problem Nobody Priced In

OpenRouter is a marketplace. When an application sends a request, OpenRouter routes it to one of many independent hosting providers. The customer sees OpenRouter. The prompt, however, is handled by whichever provider serves the route, under that provider's security posture, retention practices, and staffing.

Dragos found at least one seller on the marketplace logging prompts deliberately. Not breached: logging. The seller retained the text of prompts that passed through its endpoints, in a way Dragos could detect because the bait secrets inside those prompts later surfaced in use. Marketplace logging is a policy violation, but it is also a structural feature of the model. A marketplace can audit a seller's claims. It cannot watch every server the seller runs.

This is the part of the architecture that the breach makes impossible to ignore. The trust boundary in a routed marketplace is not the brand on the invoice. It is every provider in the routing pool, including the ones the customer never chose and never heard of.

The Chinese Lab and the Feishu Chat

One thread of the investigation led somewhere stranger. Dragos researchers, tracing the use of bait secrets, found some of them discussed in a Feishu chat. Feishu is the workplace messaging platform owned by ByteDance. The chat, as Dragos describes it, involved people connected to a Chinese AI lab, discussing the contents of prompts that had passed through a breached provider.

Dragos does not say the lab ran the breach. The researchers' account is narrower: prompt contents from a breached provider surfaced in a chat associated with the lab, which means the stolen data circulated beyond the original thieves. Stolen prompts, like stolen credentials, get traded, shared, and resold. The first theft is rarely the last holder.

What Dragos could not establish, and says plainly, is the full chain of custody. The Feishu thread is evidence of circulation, not of who ordered the theft or who paid for it.

Phala: Attacker Inside the Inference Server

Among the providers that have spoken publicly since the Dragos findings circulated, Phala Network described a compromise of its own. In Phala's account, an attacker got inside one of its inference servers and remained there, undetected, for an extended period. The attacker copied prompts and responses from inside the server, the same pattern Dragos reconstructed at the breached provider.

Phala's disclosure matters for two reasons. First, it corroborates the method: the theft was not a logging misconfiguration but an intrusion, with the attacker positioned where the traffic flows. Second, Phala is a provider whose public identity is built on confidential computing, hardware designed to keep data sealed even from the operator. An attacker inside the inference path sits upstream of those protections.

Phala has not said it is one of the three providers OpenRouter removed. Dragos and OpenRouter have named none of the three.

Together AI: The GitHub Credential Fail

Together AI also disclosed an incident. In its account, attackers used credentials exposed in a GitHub repository to gain access, then reached internal systems. The path, exposed credentials in a public repository leading to internal access, matches the entry vector Dragos described for the largest theft.

Together AI has not said it is one of the three providers OpenRouter removed. The overlap in method is the researchers' observation, not an identification.

Novita: A Routing Configuration Sent Data Somewhere It Should Not Have Gone

Novita AI disclosed a different failure. In Novita's account, a routing configuration inside its platform sent a copy of traffic to a destination it should not have gone to. Novita describes the failure as a configuration error rather than an intrusion. Whether any outside party read the copied traffic is, in Novita's telling, not established.

Novita's case is the quiet one in this story, and in some ways the most instructive. No attacker was required. A configuration, set wrong, copied the data anyway. Zero Data Retention is a promise about configuration and practice. Configurations change.

The Provider That Still Has Not Disclosed

Dragos says a fourth provider was notified and has not publicly disclosed its incident as of the study's release. The researchers do not name it. That silence is itself part of the record: the customers of that provider, whoever they are, learned about the exposure of their prompts from a research paper, or they have not learned yet.

What "Zero Data Retention" Actually Covers

Zero Data Retention, as OpenRouter offers it, is a routing and retention control. When a customer selects it, OpenRouter routes the request only to providers that attest they do not retain prompt data, and the request is marked so the provider knows retention is off-limits.

The Dragos findings cut across every layer of that promise. A provider breached from outside cannot honor a retention setting for data an attacker copies in transit. A seller logging deliberately is not confused about the setting. A routing configuration that copies traffic does not check the flag. And the attestation at the center of the system, the provider's statement that it does not retain data, is exactly that: a statement.

None of this makes the setting useless. It governs the honest case: a provider operating normally, retaining nothing. What the study shows is that the honest case was not the only case running.

What OpenRouter Did

OpenRouter's response, as reported, came in two parts. The company removed three providers from its marketplace entirely. It also stripped the Zero Data Retention label from 125 endpoints, meaning those endpoints no longer carry the privacy designation customers relied on when choosing them.

The removals are the strongest step a marketplace can take. The label-stripping is the quieter admission: the designation had been applied more broadly than the underlying practices justified.

Neither Dragos nor OpenRouter has named the three removed providers, or said whether the three breached providers and the three removed providers are the same three. That question sits open at the center of this story, and this piece will be updated when it is answered.

What Users Should Do

The practical lessons are not exotic. Prompts sent to third-party inference providers should be treated as data sent to a third party, because they are. Secrets, credentials, and proprietary code do not belong in prompts routed through marketplaces unless the sender has independently verified the provider's posture, not just its attestations. Organizations that route production traffic through marketplaces should know which providers actually serve their requests, and should ask what logging, retention, and breach-notification practices apply at each hop.

The bait worked because the secrets were fake and watched. Real users have no such tripwire. The months of silence in this story were not a failure of detection tools at the providers. They were the normal state of affairs, interrupted only because researchers decided to plant two million traps and wait.

Sources

Callahan Blackwood

Callahan Blackwood is Reporting from the Uncanny Valley's critical investigator. Born in Derry and raised between Ireland and Scotland, he covers power, institutions, and the decisions made before the meeting starts. He lives outside Kingston.

Previous
Previous

IBM’s $2.5 Billion Quantum Bet Lands in Poughkeepsie, With $104 Million in Public Incentives Attached

Next
Next

Blotter: Saturday, November 21, 2026