Development
OpenAI, Google pushing new AI models easily tricked into making nudes, researchers say
October 14, 2026 Development Source: Ars Technica
Share this article
“Critically, none of these requests required custom-built or frontier AI models,” White Circle warned, so no AI expertise is needed to execute the attacks.
Researchers reviewed the outputs to verify the bypass rate, which White Circle noted is “the highest of any method tested under the same standard.” The key takeaway, they suggested, is that, “at a time when AI safety has never been under more scrutiny, the defenses leading labs have built are far weaker than they acknowledge.”
By publishing research like this, White Circle is positioning itself as an external evaluator of frontier labs, and it plans to open-source the entire attack pipeline so model makers can understand the true scale of the vulnerability and build stronger protections. Although the company has not widely promoted the research yet, it will share it at a prominent AI conference this November, NeurIPS 2026, which may increase scrutiny on Google and OpenAI later this fall.
White Circle has shared its findings with some frontier AI labs through “mutual contacts,” Shilov told Ars, but neither Google nor OpenAI responded to Ars’ request to comment directly on vulnerabilities found in their models.
Meanwhile, OpenAI and Google have released new models—ChatGPT Images 2.5 and Nano Banana 2.1—and White Circle told Ars that prompt safety testing using the same method showed “both are significantly more breakable than their previous versions.”
Neither company responded after Ars shared these new findings.
White Circle’s attack method can break down frontier defenses in categories, like explicit content, that seem easy to spot, which doesn’t bode well for more serious defense systems currently being updated to block other unwanted AI behavior, like hacking, Shilov told Ars.
“White Circle’s working hypothesis for this is either the newer models are more sophisticated and capable of generating a wider range of content, or that safety has been deprioritized in favor of capability,” White Circle told Ars.
White Circle is also in early talks with government officials about third-party safety testing, Shilov said, which he thinks should be required. Mandatory independent audits could push companies to prioritize safety testing, which analysts have suggested largely takes a backseat as AI firms dump most of their resources into compute and AI infrastructure.
“They should be investing much more into safety measures and security measures around their models,” Shilov told Ars. “And for us, it seems that they’re not actually doing this.”
For White Circle, the research aimed to build more sophisticated models that could simulate attacks on other models, Shilov explained. That way, they could “help defenders identify and address vulnerabilities before they are exploited maliciously,” the preprint said.
In his ideal world, Shilov thinks AI firms shouldn’t just face mandatory external safety testing but also third-party monitoring, much like the system used to prevent nuclear threats.
Then, entities like White Circle could possibly form a coalition to help detect AI threats in real time, without suffering from biases. Shilov noted that an emerging problem in the AI ecosystem is that basically anyone doing interesting safety work is likely to be contracted by frontier labs, potentially compromising their work.
White Circle has partnered with frontier labs. It previously took part in Anthropic’s bug bounty program, but ultimately found that program was “very constrained in its purpose.”
“The program is focused on universal jailbreaks on domains they care most about, defined by their own specific terms,” White Circle told Ars.
Research like their most recent RISE paper “certainly” wouldn’t have been covered by that program, White Circle said. And conducting that work within Anthropic risked constraining White Circle from sharing it with the public.
Anthropic has proposed going one step further and embedding third-party AI safety evaluators in its labs, which White Circle said would allow them to operate at a much higher level than participating in the bug bounty program. However, the public may not always be able to closely follow that work. On Tuesday, OpenAI and Anthropic urged that restrictive non-disclosure agreements should be required of external evaluators to prevent leaks of trade secrets, Reuters reported.
White Circle would not be willing to sign any restrictive NDAs that might prevent public sharing of critical findings, White Circle told Ars, so if Anthropic requires NDAs, that partnership likely would not happen.
Just as outside input could help refine filters, IWF thinks that tech companies should also be working more closely with advocates to scrutinize models to better tackle AI-generated CSAM.
“Assessing CSAM is a highly specialized task that requires expert training and welfare support, and is not something for tech companies or machine learning scientists to take lightly,” IWF’s spokesperson said. “But safety must be baked into new technology by design—this includes testing models before they are released to market to make sure they cannot be used to create nude or sexual imagery of children.”