Interview: Lytro CEO Jason Rosenthal on the Future of Light Field Technology
Lytro’s research into the world of light field technology has produced two consumer devices. Their first camera was released in 2012 and introduced photographers to the concept of being able to refocus images after they had been taken. Then in 2014, Lytro released their flagship: the ILLUM. Armed with an integrated 30-250mm f/2.0 lens, a 40 megaray sensor, and upgraded software, Lytro was ready to show the world that their technology wasn’t just a gimmick.
Thanks for speaking to us and our audience today. There is a lot of interest around Lytro and the work that your team is doing. We were very happy to recently post an article explaining how light field technology works.
Jason Rosenthal: Yes, thank you for that article. From our perspective, we feel that the PetaPixel audience is the core group that we are focused on and we think are perspective Lytro customers. we enjoy talking to you guys.
One of the things we wanted to focus on is the future of Lytro, where the company is now, and where your team has come from. Why don’t we start there?
Yeah, absolutely. We shipped Lytro Illum 11 months ago and it was a big step forward for us as a company in terms of demonstrating the potential of light field technology and it helped us zero in on what we think is the core audience – everyone ranging from consumers to professionals. And with the new updates last week, Illum 2.0 and Lytro Desktop 4.3, what we have done is deliver a number of new capabilities through software upgrades.
One is that we’ve brought the entire experience of living pictures to the camera itself. Now, through new software development, we have been able to bring the desktop editing experience to the camera.
The second big thing that we have done is refresh the UI itself – a total redo of the capture and review UI on the camera. We devoted a lot more space to the composition of the picture. We have made all of the data and information semi-transparent so that you can see all the information you need, but you can use the entire LCD to frame your shot. We have also really enhanced the Lytro depth assist to help determine what objects are at what points of depth within your composition – it is much more powerful.
Third thing we have done on the camera is that we have revamped our autofocus capability. There is a specific aspect of light field that allows us to turn the entire sensor into an autofocus sensor. In a typical DSLR, you’ll have the primary imaging sensor, and then you’ll have a few pixels either spread out amongst the sensor or in the corners for autofocus calculations. We are able to make the entire sensor an autofocus sensor, so the advantage is an incredible autofocus speed; about three times faster than where we were before. Any part of the picture that you see on the LCD, you can focus on that exact point down to the pixel level.
We are excited about that, and we think that it is a big upgrade for our customers. It is available now for free to any Lytro Illum customer.
On the desktop, we have done three notable things. One is that we have improved and sped up the processing time by about a factor of three, so your pics process much faster. We have made it so that you can prioritize your best shots and have them processed first; it works really nicely.
Second thing is that we have done a big integration with Adobe Photoshop. What we found is that our customers use Lytro desktop to do light field specific things, but almost all of them are current Photoshop customers, and they are very familiar with Photoshop adjustments and the ability to use layers. So, what we have done is that by right clicking on a photo in Lytro desktop you can now open that picture in Photoshop and have it work with all Photoshop adjustments, then save it, and have them all put back into the Lytro desktop. So really, we have a nice closed loop and seamless integration.
The third thing, which is a big deal for Lytro, is that since all of the content you shoot in Illum is natively 3D, it turns out to be the perfect consumer device for creating content for a lot of the new virtual reality headsets like Gear VR, Google Cardboard, and hopefully, Oculus Rift when it ships.
You can take photos on your Illum and create a native 3D slideshow that works perfectly with the existing VR headsets. It is a nice way for a consumer or enthusiasts to start experimenting with 3D content. Our belief from the very beginning is that light field will really enable a whole new class of imaging applications.
One of the big questions that we get is what Lytro’s practical application could be for existing still photography because not everyone is completely sold on living pictures. How can you incorporate Lytro into a still picture workflow? What does Lytro see as the number one ‘thing’ the company will be mastering?
The number one thing that we see is that there is a whole set of emerging kinds of imaging applications for which light field is essential. The number one thing that won’t work without light field is virtual reality capture; that is the top of the list for us and where we think light field is just essential to give the experience of presence. It is something you simply can not do in a compelling way with existing 2D gear.
A second big application that we are focused on is within using light field technology for still photography. It is interesting in that it solves issues like focus and controlling depth of field after the fact; that is something that we have done in the Illum and will continue to focus on.
But, if you extend the technology to video and of things at video rates, that is where we think there are a whole bunch of things that are truly unsolved problems today in the current set of gear that will be revolutionized by light field, so we will heavily be focused on that as well.
Do you know whether most of your users are displaying their content via video, a still output, or a ‘living photograph’?
By far, the number one mechanism is through interactive living pictures. We see it in our analytics because everything is hosted on our servers. The biggest use of living pictures is people embedding them in their stories and their articles; that is where we get the most engagement and the most page views. Second, I would say embedding living pictures either within Facebook or Twitter.
I wanted to go back to utilizing fast focus for sports photography. One of the things that we regularly hear is that changing focus after the fact is more of a gimmick than useful practicality. What are other advantages that could be had with the technology besides ultra-fast focusing?
I would say that controlling focus and the depth of field is the very tip of the iceberg with what is possible and with what you will see in light field. I would go back to the core tenant of the technology. We are able to fundamentally capture more photons on a given sensor than with conventional 2D sensors. It starts with getting more data – more light is equivalent to more data. Just like the differences between JPEG and RAW, it all starts there.
From there it extends to the fact that when you have this richer data and more photons you can physically redesign the main lenses in a way that you can’t contemplate today. Try to imagine a lens that goes from f/0.5 to f/28 but weighs 1/10th as much as anything that you can build today and has an extremely long zoom range. That is huge for photography. That is huge for applications such as drones or aerial photography. It is a big step for commercial and industrial applications – that is the next step.
That is the hardware, but we are also really excited when you start adding in the software. Focusing and depth of field in video would be a big deal, but it goes far beyond that. Imagine if you knew every pixel in a photograph; there are all kinds of things that we can do starting with things that people do every day in Photoshop. Imagine replacing the background of your image without having to do any segmentation. Just click on the background and replace it. Like instant green screen without requiring a green screen. These are all things we are working on.
To sum it all up, our belief is that in the same way digital replaced film, computational and light field technology will replace digital. In a relatively short period of time, it will be a fundamentally better way to take a photograph. It will create lighter weight and less expensive cameras – giving photographers more creative freedom.
Would you say that something like the Illum is more of a step towards what Lytro is trying to achieve than a final product?
We are product people and technologists, so no product is ever final. Illum is a huge step forward in light field over the first generation. I think that what you’ll see from us in the future will be similar, yet bigger, move forward.
Are you aware of recent announcements concerning Panasonic building their own interchangeable light-field cameras?
We are always excited when we see other people doing things and experimenting in light field technology. In the same way with digital, this revolution is going to take more than just one company, so we welcome innovation across the market.
In direct relation to Panasonic news, there wasn’t enough detail for me to understand what they are doing. But I hope that it is great, and I hope that it is significant and I hope that it advances the state of the art.
Lytro seems to have set out to create the single ultimate lens. For future cameras, is Lytro interested in exploring interchangeable lenses?
I would say that there is nothing to announce on that today, but stay tuned. Our view is that different lenses are good for different purposes, so we want to give people full creative freedom to have the right tool for the right job.
If you have someone who is not optimistic about the technology, maybe they tried it and didn’t see the relevance of it today, what would you say to them?
I would say that we are more critical of our products than everyone, so we are always looking at ways we can improve. We think that what we have done with Illum 2 and Lytro Desktop 4.3 is a huge step forward. We are nowhere near done, so we will keep innovating. If you go through the history of imaging and photography, during every innovation the traditionalists were incredibly skeptical to change.
With the invention of color, people thought black and white was so good, yet color was gimmicky. Digital would be another example of skepticism and we know where those two technologies turned out. My prediction and my commitment is that we believe it is big, if not bigger, than those two innovations and we are super excited about the technology.
