This AI Can ‘Bring a Person to Life’ from a Single Still Photo
Researchers have created an AI that can animate a human subject from a single still photograph, “bringing them to life” by making them walk, run, sit, or jump out of a photo in 3D.
After analyzing a photo to detect the human(s) within, the AI turns the person’s silhouette into an animatable mesh after identifying and labeling the various body parts. The 2D-to-3D person has details extracted from the original photo as well as a guessed “weight” that informs how it moves around in 3D space.
The AI also fills in the hole left behind from extracting the person in the original photo (like what Photoshop does with Content-Aware Fill), allowing the person to break out from the photo without strange missing areas showing up behind them.
“The big challenge here is that the input is only from a single camera position, so part of the person is invisible,” says UW professor Ira Kemelmacher-Shlizerman. “Our work combines technical advancement on an open problem in the field with artistic creative visualization.”
Here are some examples of what the AI can do with still photos:
You may think that the results look like a poor-quality character model from an old school video game, but the researchers note that this is just a big first step — the technology is only going to get better over time.
“Photo Wake-Up is a new way to interact with photos,” says lead author and Ph.D. student Chung-Yi Weng. “It can’t do everything yet, but this is just the beginning.”
