Google Uses AI to Create Professional Photos from Street View Shots

Can machines be trained to create photos that most people would think were shot by a professional human photographer? That’s what Google is trying to figure out with a machine learning research project.

Whether or not a photo is “beautiful” is a highly subjection question, and Google wanted to see how machines can be taught subjective concepts, so it created “an experimental deep-learning system for artistic content creation.”

The artificial intelligence Google built aims to mimic the workflow of a real professional photographer. It roams landscape panoramas found in Google Street View and hunts for the best compositions. Once it finds a nice-looking photo, it uses post-processing techniques to improve the look of the shot like photographers do in Photoshop or Lightroom.

Edits include cropping, tweaking saturation, applying HDR effects, adding dramatic lighting with “content-aware brightness adjustments.”

“A panorama (a) is cropped into (b), with saturation and HDR strength enhanced in (c), and with dramatic mask applied in (d). Each step is guided by one learned aspect of aesthetics.”

“Our virtual photographer ‘travelled’ ~40,000 panoramas in areas like the Alps, Banff and Jasper National Parks in Canada, Big Sur in California and Yellowstone National Park,” Google writes. It “returned with creations that are quite impressive, some even approaching professional quality — as judged by professional photographers.”

Here are some examples of “professional photos” the system created by finding, cropping, and editing Street View photos:

Interlaken, Switzerland.
Park Parco delle Orobie Bergamasche, Italy.
Jasper National Park, Canada.
Jasper National Park, Canada.

What’s impressive is that Google’s machine was trained only on a collection of “professional quality” photos rather than before-and-after pairs, so it had to learn the features that make photos beautiful based purely on finished works. To teach the machine what looks “bad,” researchers applied filters to the pro photos that modified the brightness randomly to degrade their appearance.

To see how good the resulting AI-generated photos were, Google asked professional photographers to rate a collection of images that included both the machine-generated ones and others with different qualities. Researchers found that photos that were rated highly by their machine were also judged to be “semi-pro” or “pro” at about a 40% rate by real photographers.

Google says one day this technology could be used to help people shoot better photos in the real world.

Update on 7/17/17: You can find the complete collection of Google’s “Creatism” landscape photos here.