Twitter Axing AI Photo Cropping After Tests Reveal Race/Gender Bias
Twitter recently announced full-size images in mobile feeds, much to the delight of photographers. The company has now revealed that one of the reasons for the change is that internal tests revealed race and gender biases in its AI photo-cropping system.
The feature made headlines, however, in late 2020 after users found that the algorithm seemed to exhibit a racial bias when determining how to crop photos. One prominent example was a test featuring former President Barack Obama and then-Senate Majority Leader Mitch McConnell. When the two photos were uploaded with a significant amount of white space between them, the algorithm would always center the crop on McConnell’s face.
When this controversy reached Twitter’s ears, the social media company promised to analyze its model again for possible biases. This week, Twitter shared the results of its findings.
“The saliency algorithm works by estimating what a person might want to see first within a picture so that our system could determine how to crop an image to an easily-viewable size,” writes Twitter software engineering director Rumman Chowdhury. “Saliency models are trained on how the human eye looks at a picture as a method of prioritizing what’s likely to be most important to the most people.
“The algorithm, trained on human eye-tracking data, predicts a saliency score on all regions in the image and chooses the point with the highest score as the center of the crop.”
To quantitatively test its algorithm for bias, Twitter created an experiment consisting of randomly linked photos of people of different races and genders.
Here’s what the study revealed:
1. “In comparisons of men and women, there was an 8% difference from demographic parity in favor of women.”
2. “In comparisons of black and white individuals, there was a 4% difference from demographic parity in favor of white individuals.”
3. “In comparisons of black and white women, there was a 7% difference from demographic parity in favor of white women.”
4. “In comparisons of black and white men, there was a 2% difference from demographic parity in favor of white men.”
Twitter also aimed to determine whether its algorithm had a “male gaze” objectification bias, as some Twitter users reported that photos would be cropped on a woman’s chest or legs rather than her face. The study found that:
1. “For every 100 images per group, about three cropped at a location other than the head.”
2. “When images weren’t cropped at the head, they were cropped to non-physical aspects of the image, such as a number on a sports jersey.”
While there wasn’t any evidence of a “male gaze” bias, Twitter did find that there seems to be some degree of race and gender bias in its AI cropping. As a result of these findings, Twitter is now moving away from AI photo cropping and putting the decision-making back in the hands of users.
“We considered the tradeoffs between the speed and consistency of automated cropping with the potential risks we saw in this research,” Chowdhury says. “One of our conclusions is that not everything on Twitter is a good candidate for an algorithm, and in this case, how to crop an image is a decision best made by people.”
Twitter’s latest update included a true preview of photos during Tweet creation so users will know exactly how the images will look prior to publishing.
“This release reduces our dependency on ML for a function that we agree is best performed by people using our products,” Chowdhury states. “We’re working on further improvements to media on Twitter that builds on this initial effort, and we hope to roll it out to everyone soon.”
If you’re interested in reading the full academic paper that was produced by this study, you can find it on arXiv.org.
Image credits: Stock AI illustration licensed from Depositphotos
