This is What Makes Glass Imaging’s Groundbreaking Photo Enhancing Tech Different

Glass Imaging is a software company that says it has developed a way to dramatically improve the poor quality of small sensor cameras — mainly smartphones — using a novel approach. While it has explained a bit about its process in the past, the company recently shared even greater detail to explain why its process is so much better than what is already out there.

Unlike other up-res software or other more conventional AI enhancement technologies, Glass says it uniquely combines its intelligent pipeline with actual visual information — “reversing the trend towards hallucinations.” That’s why, the company says, its results don’t produce artifacts or elements that don’t actually exist, which is a common problem with generative AI.

“Traditional smartphone camera software relies on an Image Signal Processing (ISP) pipeline, which combines tunable algorithms embedded in silicon with custom software blocks. A consequence of this approach is that large teams of image quality engineers are required to tune each of these blocks for each newly released phone, comparing the results across various settings to find the most visually pleasing results,” Glass’s Vishal Vinod writes in a new blog post.

While this process works, some quality tends to be lost through the interactions of those algorithms, Vinod explains. Combining noise reduction, multiple exposures, and sharpening often results in the muddy, mushy pixels that are indicative of smartphone cameras and other small sensor systems.

“Modern AI-based image enhancement and upsampling can counter some of these issues by post-processing the captured image and finding a visually plausible interpretation of the degraded image, creating apparent detail to fill in what was missed or distorted by the traditional processing. However, these post-processed details often contain hallucinations that are not authentic to the original scene,” Vinod says.

Taking one of these images, which already has poor detail, and upscaling it with traditional methods doesn’t solve any problem, but rather makes them worse.

“The technical imaging community has seen a recent surge in the use of diffusion models, which are designed to enhance image resolution by filling in missing details. However, these models often produce synthetic-looking images, lacking the nuanced textures of real scenes, or create meaningless character-like shapes in text,” he continues.

To get better results, Glass Imaging started over, focusing on the information that is captured in RAW and leveraging it from that point, minimizing its reliance on generative techniques and instead focusing on what is actually there.

“This results in high-resolution images that faithfully represent the original scene, comparable to the quality of professional SLR cameras. Glass AI enhances smartphone cameras, enabling photographers to capture clear and crisp zoomed-in images even in low-light conditions. This technology empowers photographers to capture the photographs they envision, bringing their artistic vision to life with precision and creativity by effectively utilizing the RAW captures,” Vinod says.

“What sets Glass AI’s approach apart is its novel methodology for accurately characterizing lens aberrations and reliably learning the sensor’s characteristics and noise behavior. These factors are crucial when training a network to demosaic and fuse multiple raw images, as they are interconnected and influence each other. For instance, failing to account for optical blur can hinder accurate demosaicing.”

What Vinod says here sounds great, but it’s easy to be skeptical as companies have made similar promises for years. That’s why PetaPixel visited Glass Imaging in person earlier this year to see the tech in action with photos it knows weren’t prepared ahead of time.

Standing outside Glass Imaging’s headquarters, the company sent out a DJI drone and snapped a photo. The result is an image that successfully captures the scene, but the details are weak: pixels look mushed, it lacks sharpness, and highlight and shadow detail is also poor. In short, it’s par for the course when it comes to a photo taken with a small sensor.

Glass Imaging then took the RAW photo and sent it through its ISP to generate a corrected photo, and the detail it recovers is downright astounding.

Side by side at 100%, the difference is enormous. Not only is the photo clearer and cleaner, but it looks like it was taken with a much better camera with a significantly bigger sensor. The dynamic range is improved, the pixels are cleaner, and the overall appearance of the photo is just better.

Looking at the finest details, what’s apparent is that Glass Imaging’s corrections aren’t generating new pixels — or at least it isn’t appearing to. Every detail from the wrinkles on faces to the bumps on the roof of the building is real, not “hallucinations” (as Glass Imaging puts it) of an AI.

“Another standout feature of Glass’s AI technology is the ISO-conditioned generation. By adjusting the processing based on the ISO sensitivity, Glass ensures optimal image quality under varying lighting conditions, a feat that most smartphones struggle to match,” Vinod says. “In low-light conditions, where high ISO settings are inevitable, Glass maintains a clear advantage.”

Glass Imaging is focusing its efforts on smartphones because that’s where the most work needs to be done, but its process is not limited to smartphones: it’ll work on anything that can capture a RAW file. That means it can correct lens aberrations and fix sensor issues on standalone photography equipment, too. The only limitation is training the system on the intricacies of each of those pieces of equipment. That’s going to take time, but the results look worth it.

“No lens is perfect. No image sensor is noise-free and therefore leveraging our unique method of training neural networks to reverse optical aberrations and correct sensor noise allows us to maximize detail preservation, resolution, and edge fidelity of any camera while suppressing noise and artifacts to form a sharp and clean image. Glass AI works with any device and is not limited to smartphone cameras,” Vinod explains.

PetaPixel is planning to task Glass Imaging with more difficult requests in the future by tasking it to take lenses that have huge, glaring flaws but incredible character to see if the algorithm can remove those flaws but retain what makes the photos they take appealing.

More examples of what Glass Imaging’s AI can do can be seen in the company’s most recent blog post.

Image credits: Glass Imaging