This Tiny Sensor is About to Make Smartphone Photography Way Better

spectricity

Semiconductor company Spectricity has developed a proprietary spectral imaging technology that is capable of greatly improving smartphone cameras by making them smarter and able to capture colors more consistently no matter the lighting condition.

The S1 multispectral image sensor, as Spectricity calls it, can see more light than current three-color RGB sensors. It is the world’s first truly miniaturized, mass-manufacturable mobile spectral image sensor and camera module. The S1 is so much more powerful than current conventional sensors that not only can it see more visible light, it can also look toward the near-infrared spectrum. The result is that it is able to reproduce more consistent colors in photos that look more natural and have better white balance than current sensors can.

“A standard camera integrated into a smartphone has an RGB sensor that sees red, green, and blue,” Spectricity’s CEO Vincent Mouret tells Digital Trends.

“We add filters to create up to 16 different images with different colors, different wavelengths of light, light coming from different sources, and the reflected light coming from the object of the scene. You can identify many different properties thanks to these different images compared to a standard RGB.”

This technology would solve a current problem: smartphone photos see the world very inconsistently. A photo taken with an Apple device will look very different from one taken with a Vivo or Samsung device, as each is “seeing” color differently. Some might argue that one looks more “real” than another, but there is a level of uncertainty there. By integrating a multispectral camera system like the S1, color accuracy can be fine-tuned and made to look more natural.

“Smartphone cameras are essentially color blind. Their automatic white balancing algorithms (AWB) often fail to resolve the correct white point, resulting in poor color fidelity,” the company says.

“This is especially problematic in the presence of multiple illuminants or challenging scenes. Spectricity’s spectral imaging AWB detects illuminants more accurately in each area of the scene through their spectral signature. This result in true colors: improved color photography [and] accurate unbiased skin tones.”

spectricity

“Incorporation of spectral camera modules into mobile devices is part of an ongoing trend to add advanced sensing functionality,” the company explains on its website.

“Spectral imagers will be able to capture image data across many spectral channels, beyond the conventional red, green, and blue color channels. This will allow measuring of objects’ spectral signatures. Among the applications that will benefit are image acquisition where more accurate auto white balancing is needed, personalized cosmetics and skincare, remote healthcare, and smart gardening/agriculture.”

The company tells Digital Trends that the technology is able to better represent skin tones, an issue that has been brought up multiple times in the past and is one that Google says it is actively trying to combat using software.

“You can see the skin tones are totally different depending on the lighting condition,” Mouret explains to Digital Trends. “The solution is to use a spectral imager to analyze the lighting conditions, to really give the right tone. This is the only way. You can put a lot of AI behind it, but it’s not enough. You need to have some additional hardware.”

The S1 isn’t quite a camera, although it could be used as one — it would only capture 800 x 600 pixel photos, though. Instead, the sensor is best put to use working alongside a dedicated camera system and would inform that camera what to do with the lighting conditions. Luckily, its sensor is very small and it would be able to easily fit alongside an existing RGB CMOS camera.

The company is so confident in its invention, it expects all smartphones on the market to be using its technology within the next couple of years. Spectricity believes that mobile photographers will have access to it sensors on high-end smartphones starting next year, but it will become far more widespread and common by 2026.


Image credits: Spectricity

Discussion