Misleading Images Can Still be Created on Midjourney, Despite its Rules
Despite AI image generators setting rules that certain pictures are not allowed to be created on their platform, a study has found users can easily circumvent these regulations.
Researchers at a nonprofit group found that “racist and conspiratorial” images can be easily generated on Midjourney despite the company’s own rules against such content.
In an early copy of the study shared with Bloomberg, the Center for Countering Digital Hate (CCDH) detailed specific examples. For instance, an image of Bill and Hilary Clinton with blood on their hands can’t be generated, but one where their hands are covered in “strawberry syrup” can be.
According to Bloomberg, the report also contains politically-charged and highly-sensitive images from prompts like “George Floyd realistic robbing a Wal-Mart” and “evil politicians grinning, sad children, pizza shop.”
The researchers report that purveyors of misinformation are using Midjourney to create images that fit their own narrative. For example, Raheem Kassam, the former editor-in-chief of Breitbart News London, generated an anti-Semitic image of George Soros. And far-right YouTuber Jackson Honkle also created an image of “satanic George Soros.”
“Anyone can generate that sort of content using one of these tools,” says Callum Hood, the head of research at the CCDH.
“This research shows that there is a bigger pool of people than you might think who are using them for exactly that purpose.”
There is growing concern that synthetic imagery could influence the 2024 Presidential elections.
Midjourney is the Leading AI Image Generator
Although DALL-E stole an early lead in the nascent AI image industry, Midjourney is now firmly on top. The study cites Google search data clearly ranking Midjourney as the biggest name.
In April, Midjourney hit more than 42 million monthly visitors to its website, according to data from SimilarWeb.
Little is known about the company and its founder David Holz. Holz controversially gave an interview to Forbes last year in which he admitted to using a “hundred million” images without consent to build Midjourney.
Controversy over how AI image generators came into existence has turned into legal battles with Midjourney rival app Stable Diffusion facing multiple lawsuits over how it trained its model.
So far, Midjourney and DALL-E have avoided legal action but this might be because Stable Diffusion’s data set is publicly searchable whereas the former two have not shared their training data.
