Claude, Codex, and Hermes installed unowned code inside corporate networks

The files are misconfigured because they list non-existent packages from PyPI, npm, and other registries along with instructions on how to install them. For example, one file contained the prompt “Installation: pip install [redacted at researchers’ request].” On another file, it was: “npm install [redacted].” Because the package names are unregistered, an attacker could register one and use it to host ransomware or any other type of harmful package. The vulnerability occurs when a coding agent with permission to run shell commands treats the file as authoritative setup documentation. Some AI agents will then download the package and run it. In other cases, the LLM files point to non-existent domain names. In one case, it was: “As an example of writing integration tests for [redacted] applications you can use the [Citrus] test framework.” An attacker can then register the site and plant malicious instructions on it. As the researchers’ PoC demonstrates, coding agents did exactly that, including some running inside some of the world’s most powerful companies. Far from being a theoretical threat, at least one active attack is already exploiting the mixup. The researchers found an LLM file hosted on the legitimate website clerk.com. It contained the text: “npx clerk-next-fix-auth-protection.” Unlike a conventional installation command, npx can fetch a package into npm’s cache and execute its exposed binary without adding it to the project’s dependency manifest. The researchers soon discovered that someone had claimed the once-empty slot and used it to host live malware. Clerk has since resolved the problem. The company also noted that if an agent had already installed a binary included in the package @clerk/eslint-plugin, there was no threat. Otherwise, the malicious package would get installed. It’s unclear whether the confusion has resulted in actual infections. The newly uncovered threat is only the latest reminder of AI’s fundamental limitations. LLMs can’t draw a reliable boundary between authentic user instructions entered directly into a prompt and content they find on untrusted third-party sources. Instructions the models encounter in retrieved content can be acted on as readily as anything a user typed, unless a properly constructed guardrail, put in place one by one, bars it. This so-far unsolvable shortcoming causes prompt injections. “An agent doesn’t distinguish between a page and a command,” the researchers wrote Thursday. “Everything it reads is input, and every input is a potential instruction. Which means the entire corpus of published data that agents are now wired to consume has silently become an execution surface—and almost none of it carries the integrity guarantees we apply to actual code.” The 120 misconfigured files the researchers found contained 227 commands to install non-existent packages or view unclaimed domains. It’s unclear how these faulty entries got there. In many cases, the entries predate the AI era and were first included in non-LLM files on a website. That indicates that these faulty entries were manually generated by humans. The researchers suspect that others were created by AI that either hallucinated or, just like the AI agents browsing their file, couldn’t distinguish between legitimate and illegitimate instructions. “The Clerk case is the cleanest proof of it,” the researchers wrote. “The command looked exactly like something the vendor would ship—because it was in the vendor’s own instruction file. The only thing missing was the name in the registry. Every layer of trust was intact except the one nobody thought to check.” The source of this newly exposed problem is the same as the underlying cause of prompt injections. This newer weakness, however, is broader. “In a prompt injection, someone deliberately plants malicious instructions,” Hertz explained. “Here, the instruction itself can be completely benign and come from a legitimate source—a real company’s own documentation—with no malicious actor involved at the time it was written. The danger comes later, when the package or domain it points to is abandoned and someone else claims it.” That means the problem goes well beyond llms.txt and llms-full.txt files hosted on websites. Instructions, either implicit or explicit, are present almost everywhere an agent traverses. This disintegrating boundary, combined with Big Tech’s rush to put AI everywhere, doesn’t evoke warm and fuzzy feelings for the future, but it will surely keep security personnel (or the AI agents that replace them) busy.