On-Sensor AI to be the Next Big Step for Smartphone Cameras

Samsung and SK Hynix, both South Korean-based sensor manufacturers, are reportedly putting heavy emphasis on integrating AI tech directly onto an image sensor.

While using a dedicated chip for AI image processing is not new — Sony has been utilizing them in its full-size cameras for some time now — literally closing the gap between the that chip and a sensor appears to be the next major step for two of South Korea’s most well-known sensor makers.

Last year, SK Hynix unveiled a camera sensor that successfully combines the AI chip and image sensor into one unit late year. The company says the sensor is capable of processing data at the sensor level instead of having to route it to a separate chip. There is not a lot of information explaining the benefits of building AI directly into a sensor instead of relying on an external chip since the concept is quite early on in development. However, by combining the chip and sensor, companies may be able to processes visual data better and faster than current methods which rely on two separate pieces of hardware.

That said, it is possible that the combination of tech would result in better image quality, reduced latency, lower power consumption, and improved performance of object recognition.

Inside Samsung, this advancement of its technology is being referred to as “Humanoid Sensors” as the goal is to attempt to develop sensors that are capable of replicating the way humans see, Korea Tech Today reports. The idea is that these sensors aren’t just capturing information but actively understanding what is being “seen,” hence the human aspect.

While details on this internal development are scarce, Samsung’s already released Zoom Anyplace technology found in its latest 200-megapixel ISOCELL image sensor is positioned as an early look at the concept. Zoom Anyplace allows the camera to track subjects in digital zoom, enables dual simultaneous recording at two separate digital focal lengths, and — as the name suggests — lets filmmakers digitally zoom in on subjects without modifying the angle of view.

Samsung seems to believe that a successful melding of the two technologies would result in a huge increase in output resolution that is on-par with human vision, another nod to the “Humanoid Sensor” name, and it is not out of the question for these new hybrid sensors to be able to produce images at 500 or even 600 megapixels.

This is, for now, theory as sensors with an integrated AI chip aren’t expected to come to market until 2027 at the earliest.

Image credits: Samsung