This Guy Figured Out How to ‘Cut and Paste’ the Real World Into Photoshop

Artist, designer and programmer Cyril Diagne recently created a bit of tech that looks more like science fiction that science fact. Using a combination of augmented reality and machine learning tech, he’s figured out a way to “copy and paste” objects from the real world into Photoshop, using just a smartphone.

Diagne unveiled his creation over on Twitter earlier this week, where even the Adobe Photoshop account was blown away by the results:

What might, at first blush, seem like special effects trickery or an April Fools prank is actually a piece of bona-fide tech that he’s actually published to GitHub. As the description on GitHub explains, this is “an AR+ML prototype that allows cutting elements from your surroundings and pasting them in an image editing software.” He’s using Photoshop, but it could, in theory, work with anything.

According to Diagne, the “secret sauce” is a piece of machine learning (AKA “Artificial Intelligence”) tech called BASNet, which automatically recognizes and cuts out real-world objects when taking a photograph. That takes care of the “copy” in “copy and paste.”

BASNet is then combined with another piece of tech called OpenCV SIFT, which allows Diagne’s computer to sync up with his phone and detect where exactly he’s pointing the camera for the “paste” part of this trick.

All of this comes together into an app that Diagne designed for his smartphone, and the results are nothing short of astounding. It takes about 2.5 seconds for the phone to cut an object out of the real world, and another 4 seconds to paste that object into a Photoshop document.

As we mentioned above, the code behind AR Cut and Paste is available on GitHub, but keep in mind that this is a “research prototype” and not a consumer-ready feature. That said, now that we know Adobe is watching, we sincerely hope they’re planning to integrate something like this into Photoshop ASAP.