Apple Patents Image Sensor with 20 Stops of Dynamic Range

Apple filed a fascinating image sensor technology patent last month, which describes a stacked image sensor with vast dynamic range and very low noise.

The patent, first reported by Y.M. Cinema, describes an image sensor with a dynamic range of 120 dB, which is equivalent to around 19.9 stops, on the lower end of a typical human eye’s dynamic range. If Apple were to develop this sensor, it would be a class leader and outpace even the best large-format cinema cameras on the market today.

However, it is worth noting that just because Apple has filed a patent, it does not mean the described image sensor technology will ever see the light of day. Apple, like every other technology company, regularly files patents, many of which never result in a commercially viable product. Apple’s dreams of an entirely in-house image sensor design may remain just that, a dream.

With that massive caveat out of the way, what does the patent actually describe? The patent, “Image sensor with stacked pixels having high dynamic range and low noise,” outlines an image sensor comprising two components: a sensor die and a logic die. The image sensor itself features, as expected, an array of detector elements (photodiodes). It also includes a lateral overflow integration capacitor (LOFIC) circuit to improve dynamic range in challenging situations. Each photodiode is paired with a pixel circuit on the logic die, which has a memory circuit that senses the amount of noise at the image detection level.

At a basic level, Apple’s patent is for a stacked image sensor that can store different levels of light across each pixel, allowing different photodiodes to be more or less charged based on the scene’s brightness. So, if a pixel needs to be more sensitive to the light level striking it, it could be, while another can remain less sensitive to avoid blowing out if it is struck with more light.

There are two particularly exciting aspects of this patent. The first is that each pixel appears able to be controlled independently based on the level of light hitting it, including bespoke noise reduction and sensitivity levels. Another interesting element is that Apple introduces image processing at the sensor level, which is not only required for the pixel-level performance described above but also enables the sensor to achieve an extreme dynamic range and image quality performance that is otherwise impossible with modern image sensor technology.

The computational photography that currently helps overcome the limitations of small image sensors in smartphones requires a separate image processing pipeline, which can combine multiple images of the same scene to deliver an expanded dynamic range. However, Apple’s patent not only moves that processing pipeline to the image sensor itself, but it opens up the door for achieving similar, or better, dynamic range results in a single image capture, which would make it much more helpful for burst photography and video applications.

Of course, Apple could theoretically employ a sensor like this on a much larger device than a smartphone, which could be pretty spectacular. While this specific sensor technology may never see the light of day, it is entirely plausible that Apple would develop its own image sensors, as PetaPixel reported in 2023.

Image credits: Apple