This App Uses AI to ‘Paint’ New Things Into Your Photos

Jan 28, 2020

There’s amazing work being done in the area of using neural networks to edit or create photos. If you’d like to experience some cutting-edge technology for yourself, check out GANPaint Studio. It’s a free online demo that shows how photo editing tools of the future could work.

The app, born from research being done at MIT and IBM, does “semantic photo manipulation with a generative image prior.” Basically, it takes a photo and allows you to “paint” onto it, but instead of painting colors, you can realistically add objects — things like trees, brick textures, windows, etc.

The demo has a number of example photos you can draw on, but you can also upload your own photo (limited to images of churches for now).

Here’s a 2-minute video showing the tool (released with the research back in mid-2019) in action:

“The core part of GANPaint Studio is a neural network (GAN) that can produce its own images of a certain category, e.g. kitchen images,” the researchers write. “In previous work, we analyzed which internal parts of the network are responsible for producing which feature (project GANDissect). This allowed us to modify images that the network produced by ‘drawing’ neurons.

“The novelty we added for GANPaint Studio is that a natural image (of this category) can now be ingested and modified with semantic brushes that produce or remove units such as trees, brick-texture, or domes. The demo is currently in low resolution and not perfect, but it shows that something like this is possible.”

If you’d like to give it a shot yourself, head on over to the GANPaint Studio website. If you’d like to peek at the research being done, you can download the paper here.

(via DPReview)

P.S. Last year, NVIDIA shared a AI GAN app of its own called GauGAN that impressively turns your doodles into landscape “photos.”