Applying Eye-Contact AI to Movies Makes Scenes Very, Very Weird

Jaron Schneider

Eye Contact AI

Earlier this month, NVIDIA released a new beta software that uses artificial intelligence to fake eye contact. While designed for video calls and live streaming, a VFX specialist applied the effect to movie scenes and the result is… unsettling.

Daniel Hashimoto, a VFX artist that operates the Action Movie Kids channel on YouTube, used the artificial intelligence (AI) technology to alter scenes from movies, The Verge reports.

Outside the rare case where an actor will speak to the audience directly, which is known as “breaking the fourth wall” and is a principle aspect of Marvel’s Deadpool, actors on screen play out their roles as if the audience doesn’t exist. Changing that by altering eye contact dramatically affects how a viewer feels when watching certain scenes.

Hashimoto tells The Verge that the underlying technology that NVIDIA is using is extremely impressive.

“From a technical perspective, the technology is incredible. The tracking is real-time, and the lighting and color matching is very impressive,” Hashimoto says.

He even found that the effect can animate in and out and will even respect a target person’s head direction.

“It will correct the eyes, when they are looking just off screen, but respects if a person is clearly turning their head and looking at something else,” he adds.

Hashimoto’s changes to scenes were not done to improve the original films, but rather to just see what happened. He even goes so far as to agree that altering some of these extremely famous scenes is “pretty sacrilegious.”

If nothing else, altering the eye contact of these actors can give everyone a sense of what it would be like to actually take part in these scenes. The emotions that Robert De Niro or Javier Bardem elicit when speaking to a person directly are at the very least amplified significantly when compared to how someone would feel watching them speak to someone else. The exercise may allow viewers to get a better sense of how characters might actually feel if they were put in these scenes personally. Bardem’s scene from No Country for Old Men is particularly chilling when his character is speaking directly to a viewer, for example.

Image credits: Daniel Hashimoto

,
, , , , , , ,
Related Articles
Before and After Comparisons of the Visual Effects in Mad Max: Fury Road
'The Creator' by 20th Century Studios How Visual Artists Used Photo Skills to Make ‘The Creator’ Feature Film
Singapore Amazing Timelapse Made From 3.5M Photos Shot Over the Last Decade
Peris Digital Rig Massive Array of 144 Sony Mirrorless Cameras Generates Detailed 3D Assets
Discussion