New Sensor Allows Smartphones to Reliably Freeze Action in Photos
French startup Prophesee is teaming up with Qualcomm to deliver a new type of image sensor that they say would enable smartphones to capture and freeze fast action better than anything currently available.
The multi-year agreement would allow Qualcomm to embed Prophesee’s technology into its chips. Qualcomm doesn’t make image sensors, so rather than replace existing ones that already come from other brands, the technology would step in when a photo is taken to deal with movement and light changes.
Sports and fast action scenes are difficult to freeze in motion when shooting with phones without some blur setting in during movement. Shooting in low light also often leads to blurry results. The Paris-based startup says it wants to remedy that by breaking changes in a scene down to each pixel.
The idea is to reduce the power, latency, and data processing necessary to help traditional frame-based sensors process otherwise invisible pixels. Prophesee’s Metavision sensors and algorithms are supposed to mimic how the human eye and brain work to remove motion blur and image artifacts.
The Metavision sensor embeds a logic core that enables it to act as a neuron. Prophesee calls each pixel that activates under this an “event.” When the pixels activate themselves, they sense the number of photons intelligently and asynchronously, driven by the scene’s dynamics. To ward off blurring, the event-based sensor syncs up with an image sensor’s frame-based image capture and “fills the gaps between and inside the frames with microsecond events to algorithmically extract pure motion information and repair motion blur.”
This process only focuses on changes in a scene at an extremely high rate of speed. It doesn’t do anything as far as processing or composition in that it ignores pixels that don’t show any movement so as not to interfere with how the phone’s image sensor captures an image. The result should be a high-quality image that freezes motion, yet loses none of the phone brand’s software computation.
It’s not clear what Qualcomm plans to do or when it will implement Prophesee’s event-based Metavision technology, though it would likely be in the former’s Snapdragon chipsets commonly found in many Android phones.
It’s also not clear if this would work with all photos taken in a camera app, including manual, high-resolution, and RAW images, or if it would require phones to activate it through a specific mode. Prophesee says it will release a development kit compatible with its sensor technologies sometime in 2023.
Image credits: Prophesee