Netflix’s New ‘Magenta Method’ Makes Removing Green Screens Much Easier
Researchers at the streaming powerhouse Netflix have discovered an innovative way to quickly, easily, and potentially more accurately remove green screens from a scene by turning their actors a bright magenta using red and blue foreground lights.
Originally reported by Gizmodo, this new method of green screening leverages artificial intelligence (AI) to take advantage of a digital camera’s “blind spot” with magenta lighting which creates a near perfect silhouetted green channel, therefore making it much easier to eliminate in post production.
According to the abstract, “in this configuration, the green channel shows the actor(s) silhouetted against a bright, even background, which can be used directly as a holdout matte, the inverse of the actor’s Alpha Channel.”
The AI colorization model is trained with “an example sequence lit by standard “white” lighting and then further shown that time-multiplexing the lighting between magenta green screen and green magenta screen allows the technique to be practiced under what appears to be mostly normal lighting.”
Capturing video using the standard green screen techniques has several challenges for production teams including ensuring the clothing and items in the scene don’t match the color of the backdrop, additional motion blur clean up in post, as well as much of the time, the talent can’t wear make up during these scenes to ensure additional glare and glows are not present. Since these can be quite costly and time consuming, it makes sense that a company the size of Netflix would seek a faster and easier method to create and capture this composited content.
Using the red and blue lights to completely wrap the talent and sets in magenta will definitely look strange and off-putting to the human eye, but to a digital camera, it makes perfect sense, as the sensors that record in reg, green, and blue channels can take this information and produce a green channel where the scene is practically perfectly silhouetted against the bright and evenly lit background. This, effectively, creates a perfect matte mask.
The green “alpha” channel shows the AI what parts of the fame that should stay, and what needs to be removed and replaced digitally, even including transparent objects like bottles and windows. Additionally, this new format of green-screening allows the talent to be able to wear anything (including green) and it won’t “disappear” like it would in standard green-screen video.
According to the team, the challenges this method of green-screening presents is it can make checking the footage captured on set to ensure quality pretty difficult since they have to render the video using AI after the fact. Meaning, it can’t be presented and viewed in real time like standard green-screens so visualizing the shots for on-set previews can’t really happen in its current form. At least currently. The researchers are still working on a way to preset real-time previews of the color corrected footage so the production teams can be certain they’ve gotten the shot before moving on.
Image credits: Magenta Green Screen: Spectrally Multiplexed Alpha Matting with Deep Colorization
