‘Quantum Mirror’ Sony Camera App Uses DALL-E 2 to Reimagine Photos
Photographer Nicholas Sherlock has created an app for Sony cameras that submits any photos taken directly to OpenAI’s DALL-E 2, which he says allows users to “peer into parallel universes” and see reimagined versions of those scenes.
Called Quantum Mirror, and published on Github, the app is designed for Android-based Sony cameras, which includes older models like the Alpha 7 and Alpha 7R II but not the Alpha 7 III cameras or newer, and submits photos captured to the DALL-E 2 artificial intelligence (AI) image generation system that then uses the photos to generate variations of them.
“Because the app allows you to see the results from the AI immediately in the field, it drives a fun cycle of discovery and experimentation, and also makes for a good shot-and-reaction format for a video presentation,” Sherlock tells PetaPixel.
“I tether my camera to my phone’s hotspot via WiFi so it can submit my images in the field and show the results for playback after about 20 seconds of processing by the artificial intelligence.”
DALL-E 2 is an artificial intelligence (AI) system developed by OpenAI that is capable of creating photo-realistic images based only a brief description. Originally showcased in April, the app entered beta on July 20.
“My photos are sent to the DALL-E 2 AI as I take them, which dreams up new versions of them, revealing a world-that-might-have-been,” he explains. “Join me on a photowalk through an alternate-reality version of Dunedin, New Zealand.”
The original photos below were captured on Sherlock’s Sony Alpha 7R combined with either a Canon FD 35mm TS SSC lens or Laowa 85mm Macro lens.
DALL-E 2 seems to have the easiest time recreating simple landscapes, as the images that include text or logos become strange, twisted, nightmare versions of themselves when the AI attempts to recreate them.
As Sherlock says, “spelling is not the AI’s strong point.”
While the results are interesting and the process of creating them unique, Sherlock says that OpenAI considers his app a violation of its Beta program terms of service.
“Currently, using this app will get your DALL-E 2 account automatically banned, because OpenAI considers it to be a ‘web scraping tool’ in violation of their Terms and Conditions,” he says.
Sherlock intends to adjust the app after OpenAI has released its public API and reformat the version to comply with those new terms.
The Quantum Mirror Github page is the best place to read more about the app for those interested in seeing how it works. Additionally, Sherlock’s Quantum Mirror site hosts many more photo comparisons between what he captured with his Sony camera and several interactions of what DALL-E 2 reimagined from them.
Image credits: Photos by Nicholas Sherlock, AI images generated with DALL-E 2
