The Science Behind Lytro’s Light Field Technology and Megaray Sensors

The shutter fires and your camera’s digital image sensor is hit by photon particles, creating a two-dimensional photograph; this process is one that photographers are familiar with in their day to day work. However, when Lytro introduced the first commercially available light field camera, the game was changed with a sensor that could capture more than before – aperture and focus became adjustable in post-production, and an interactive perspective became possible.

If you venture out to purchase a $1,299 Lytro ILLUM, asking the sales representative how the camera works will most likely result in a shrug of the shoulders. If you decide to visit Lytro’s website to read up on the technology, you will be presented with the a 187 page Stanford Ph.D. thesis written by the company’s founder, Ren Ng. Needless to say, the technology behind Light Field Technology doesn’t seem like it will be easy to explain.

We decided to sit down with the Lytro team and ask them to explain to us, in layman terms, how their seemingly futuristic technology manages to accomplish what it does. Here’s what we learned.

The secrets of Lytro live within two main packages: a microlens array and advanced calibration software. The Lytro ILLUM contains a sensor that is ‘directional sensitive’. A standard image sensor reads the amount of light that hits each pixel and determines color information, but it doesn’t know where the light came from before it entered the camera. The key behind Lytro is that their hardware and software know the origin of the light entering the camera.

It may fascinate you to learn that the sensors used by Lytro are simply ‘off the shelf’ pre-made pieces of silicon – just as you may have in your current camera. But the Lytro is unique due to what is placed on the sensor to create the ‘custom package’. The ILLUM and its predecessor contain a sophisticated and precision designed layer that sits atop the sensor called the microlens array.

This microlens array is the real secret behind Lytro. Instead of allowing light to directly hit the sensor, a microlens array consisting of 200,000 hexagonal lenses is placed in front of the silicon to obtain slightly different perspectives of the image entering the camera’s lens. Each micro hexagonal lens is also mapped to a number of pixels on the image sensor. What this means is that every photon that enters the camera hits the array and lands on a particular sensor pixel that is determined by where the light originated.

Let’s say you are in a dark room and a single small light shines off to your right-hand side. With the Lytro ILLUM, that bit of light will always hit the same exact pixels after being filtered through the microarray. The Lytro sensor package has moved beyond the information captured by a standard sensor to become a sensor with ‘directional sensitivity’.

Lytro measures the resolution of their sensors in ‘Megarays’, which is just a simple term describing how many megapixels the underlying sensor has and denoting that a microlens array has been placed on top. For example, the 40 Megaray Lytro ILLUM is in fact a 40 MP sensor that has a 200,000 microlens array placed on top of it. However, the end resolution won’t be a 40 MP image due to the way data is broken down and analyzed. In the ILLUM, the resulting image has a ‘peak’ of 4 MP, which may sound small, but is still beyond 1080p (2 MP) by a decent margin.

For those of you who decided to play with Lytro’s first generation camera, it has a stamp of 10 Megarays. The unit can produce images with a resolution of 1.2 MP, which is slightly above the 720p HD cutoff.

While the hardware is not enough to enable the technology to work, extremely sophisticated software that can process the data has been in development since 2008. Every piece of glass within the camera body has the possibility of being slightly different at a microscopic level. Before a camera is shipped out to the end user, each unit that comes off the manufacturing line is calibrated to know exactly how light travels within the body before depositing onto the sensor.

While the Lytro team didn’t want to give away all of their secrets, they mentioned that the calibration process could be imagined as a camera sitting on a platform and receiving beams of light from different targets. Because the Lytro’s hardware is completely electronic, systems can automatically zoom the lens during testing to see how light is affected.

To summarize in layman terms, each of Lytro’s cameras use a special microarray consisting of 200,000 hexagonal lenses. The lenses can capture light and deposit it onto specific pixels depending on where the rays originate. Advanced software and calibration data for each camera then produce an image in which we can manipulate aperture and perspective.