Program That Claims it Stops AI Manipulation of Photos Does Not Work

A program that was designed to prevent artificial intelligence (AI) models from editing an individual’s photos does not work if the image is simply saved as a JPEG file.

Last month, PetaPixel reported on a new program called Photoguard that purportedly denied AI the ability to manipulate an individual’s photo convincingly.

The program appeared to “immunize” photos against malicious AI edits and stop an individual’s image from being used to create deepfakes.

However, according to a new paper published on Arxiv earlier this month, researchers say that programs like Photoguard can be thwarted simply by saving the image as a JPEG file.

In the paper, researchers Pedro Sandoval-Segura, Jonas Geiping, and Tom Goldstein reveal how JPEG-compressed images can bypass Photoguard’s protection and defenses against AI editing.

Photoguard uses data poisoning techniques to disturb pixels within a photo to create invisible noise in an image. According to the developers, this essentially renders AI art generators incapable of generating realistic deepfakes based on the photos that it system has been fed and trained on.

In the figure above, Sandoval-Segura, Geiping, and Goldstein show how an AI model is unable to edit a Photoguard image when given the text prompt: “dog under heavy rain and muddy ground.”

But when the Photoguard image of the dog is saved as a JPEG, an adversary can easily edit the picture while maintaining the original subject and adding key visuals of the original text prompt.

The researchers also reveal that the higher the JPEG compression quality, the more the image undermines and thwarts any of PhotoGuard’s protection from AI.

In the figure above, the team shows how an AI system can easily edit and manipulate an image that has more JPEG compression.

The researchers describe how “an image at 100% JPEG quality is almost equivalent to the original Photoguard image, while 65% JPEG quality loses significant high-frequency information.”

The study shows just how difficult it is to prevent AI manipulation of photographs and that managing the rise of photorealistic deepfakes is actually far harder than researchers may have initially presumed.

Currently, there are millions of photos online that have been used to train AI models without the consent of photographers or artists.


Image credits: All photos sourced from “JPEG Compressed Images Can Bypass Protections Against AI Editing” by Pedro Sandoval-Segura, Jonas Geiping, Tom Goldstein.

Discussion