This Neural Network Can Turn a Hand-Drawn Sketch Into a Realistic Portrait
Researchers at Radboud University in Holland have managed to do something pretty crazy. Using hundreds of thousands of photographs, they’ve trained a neural network to take a hand-drawn sketch and create a photorealistic portrait from it.
Computers have been able to do the reverse task for years: take a portrait and make it look like a sketch. For that matter, so have humans. But to take a hand-drawn sketch and turn it back into a photograph, that’s a different challenge entirely, and one that took 200,000 images to crack.
Yagmur Gucluturk, Umut Guclu, and their team at Radboud began by taking 200K images of faces from the Internet and converting them to line drawings, grayscale sketches, and color sketches. Then they fed all of that data into a neural network, essentially teaching it how to turn a sketch into a photorealistic portrait.
Once trained, the team put the computer to the test. Here’s an example they provided in their paper. On the far left is the real portrait, followed by the kind of drawing that was given to the network. On the far right is the final image the network created from that drawing:
But this was just round one. Round two used hand-drawn sketches rather than computer-generated ones. Yet again, the computer performed admirably. It was even able to guess hair color and ethnicity based on the black-and-white sketches, producing accurate color portraits without any actual color information.
And finally, during the last round, the computer was given self-portrait sketches by the likes of Van Gogh and Rembrandt. You can see those results below:
The immediate application of this technology is obvious: turning police sketches into photorealistic images of the suspect. But there are serious implications for photo editing too.
The MIT Technology review points out that these kinds of neural networks can accurately colorize black-and-white images and even increase the resolution of a low-res image (ENHANCE!). The next crazy photo editing tool may just rely on a neural network like this one to do its magic.
(via Engadget)
