Google's SynthID watermark is hard to break, but it doesn't solve AI misinformation

“Through the whole development process, we sort of assumed that a technology like this will be attacked,” said Kohli. “So we did a lot of research in making SynthID robust to different kinds of transformations. Whether people are adding some sort of filter or cropping the image, we used these transformations and made sure that the detector was robust against them.” Google has been reluctant to provide much technical detail on SynthID’s functionality beyond the original paper, so it’s worth testing these claims as the technology expands across the AI landscape. But who has the time to wait for AI images to degrade naturally as they get passed around the Internet? I used the Python Pillow library to simulate data loss from repeated sharing and downloading at a vastly accelerated rate. The Python script chooses random compression and resizing values within set ranges to apply to the test image, then uses the output image as the basis for the next iteration. This test focused on two types of AI images: one created entirely from scratch by the model and an original photo edited by AI. Both images generated by Nano Banana Pro include the SynthID watermark, and no amount of metadata editing can hide it. Still, security is never perfect. If SynthID is successfully bypassed, which seems plausible if it continues to grow in popularity, all those billions of AI images and videos could be unlabeled in a snap. Google may find itself in a cat-and-mouse game as it updates the standard to block those attacks. Google is quick to point out that SynthID is not a silver bullet for digital providence. But SynthID is becoming a much bigger target. Between Google and OpenAI, most of the consumer AI space is now using this technology. The team behind SynthID has plans in place to combat hacks, but Kohli says Google needs to keep much of that to itself. “We can, of course, change the watermark,” said Kohli. “There are a lot of things at our disposal. I cannot publicly disclose many of these mitigations because they help us make sure that the whole system is protected.” At least some of the security model for SynthID becomes apparent when you try to use it. Google doesn’t want to give people an easy path to attack SynthID, so access is intentionally limited. If you see a suspicious AI image, you have to ask Gemini to call the verifier—there’s no API or publicly accessible SynthID detector webpage. That’s not the only limit, either. To prevent people from using successive AI checks to tune a bypass workflow, Google limits SynthID to “approximately 10 image checks” per day. In my testing, the verifier will lock you out even faster if you upload too many similar-looking images. If you’re suspicious that an image may be generated by AI more than a few times on a given day, you may be out of luck. At a time when politicians are gleefully using AI to defame and attack opponents, verifying the truth shouldn’t come with a cooldown period. Even today, there are plenty of open models that generate images with no labeling whatsoever. These AI models can be shared and improved upon indefinitely, even if most of the major AI players adopt a multifaceted approach to labeling AI content. The generative genie is out of the bottle, and there’s no stuffing it back in. Since there will always be unlabeled AI content, you can’t count on watermarks to tell you what’s true. As AI labeling becomes more common, people may think an image or video that lacks one is automatically legit. That’s a dangerous but understandable mistake for someone to make. While the existence of invisible watermarks might give people some sense of security as they explore an increasingly distorted Internet, they aren’t a reliable way to know what’s true. “The future that we live in will increasingly be a world of unlimited content,” said Rose. “It’s economics 101, supply and demand. Supply goes through the roof, and the value of content is going to continue to crash. But increasingly, there is going to be a demand for authenticated content, a way to prove that something is what it purports to be.” The scale of AI content alone will make it impossible to label all of it. Google’s 100 billion SynthID images and videos are just the tip of an iceberg that has already eclipsed authentic content on the Internet. It may be smarter to focus on verifying truth rather than outing falsehoods. “There’s a limit to how much content can be created with actual physical cameras in the world,” explained veteran photojournalist and Starling fellow Mike Caronna. “There’s no limit to how much synthetic content can be made. That’s why we want to pivot to what’s scarce, and what’s scarce is high-value, truthful information. If we can secure that and prove that it’s authentic, we can defend against the liar’s dividend.” Maybe you haven’t heard that term, but you’ve experienced it. The liar’s dividend is a rhetorical strategy wherein someone is confronted with evidence of wrongdoing, and they claim the facts are manipulated. Or they simply shout “fake news.” It’s been a depressingly effective tactic for public figures to escape accountability, and it may become even more powerful as AI content proliferates: “That’s not me—it’s AI!” That brings us back to C2PA. While it’s easy to scrub digital files, C2PA may actually offer a better way to determine what’s real. This and similar technologies are essentially tamper-evident seals for digital files. Cryptographically verifiable metadata can prove all that comparatively scarce real content is, well, real. That may be far more useful than looking for proof that something is AI-generated in a sea of slop with unknown origins. The problem is that C2PA is still rare. Few phones and cameras create these digital fingerprints, and most websites and apps don’t expose C2PA data in images (you can look at C2PA data with various online tools, though). Google is actually ahead of the curve here. The company’s Pixel phones are the only mainstream cameras with deep C2PA integration. Both images and videos captured by devices like the Pixel 10 include cryptographically protected C2PA metadata that reveals when and how an image was captured. It also lists whether AI was used to alter the image. In the future, we may have no choice but to treat content that lacks verifiable providence data as suspect. That’s just the consequence of living in a world in which typing a few words can deliver an endless stream of misinformation. Labeling AI content is a good policy for tech firms, but it won’t save us.