AI Photography Restores Missing Parts of Rembrandt’s ‘The Night Watch’

For the first time in 300 years, viewers can enjoy “The Night Watch” — the iconic painting by Rembrandt — in its entirety thanks to high-resolution photography and computer learning efforts.

The famous masterpiece is a 1642 painting by the Dutch painter Rembrandt van Rijn and is currently displayed in Amsterdam’s Rijksmuseum. The painting has been on display in numerous homes throughout the centuries prior.

One of the features the painting is celebrated for is its size, which measures 142.9 inches by 172 inches. However, in 1715 the canvas was trimmed by 23.6 inches from the left side, 9 inches from the top, 4.7 inches from the bottom, and 2.8 inches from the right. Not as an act of vandalism, the trim was performed so that the painting could fit between two doors at Amsterdam’s city hall, The Guardian reports.

This left parts of the painting missing, but with the combination of high-resolution photography of what remains as the original, computer models of the painter’s techniques that leverage artificial intelligence, and a current copy of the full painting by Gerrit Lunden — which is on display in London’s National Gallery — Rijksmuseum in Amsterdam was able to temporarily restore it.

This is not the first time the painting has undergone restoration. “The Night Watch” has suffered numerous attacks in the past: it was slashed first by a jobless shoemaker and former Navy chef and later by an unemployed school teacher. It was also sprayed with acid by an escaped psychiatric patient. As such, it has been the subject of numerous complex restoration measures.

As part of the latest restoration, the team used convolutional neural networks — a type of AI algorithm that helps computers understand images — which helped recreate what was lost pixel by pixel.

“As part of our renovation of The Night Watch, last year we took the most detailed photograph of the painting ever and we had a lot of information,” Taco Dibbits, director of the Rijksmuseum, says.

“We did three types of algorithms: one to single out the perspective distortions and to correct those; the second to recognize the color scheme of the original and to project that on the pieces that are missing; and the third one is the brushstrokes — the technique that Rembrandt used. Then you get a computer reproduction that looks as much as possible as The Night Watch.”

After that, a canvas was printed in scale and attached to the original painting, where it will remain for three months so that the museum visitors can enjoy the original composition as the painter intended it.

Dibbits explains that the “newly augmented original” will be removed after three months because he doesn’t want to deceive the public and trick people into thinking that they are viewing the original in its entirety.