New Chip Can Process Nearly Two Billion Images Per Second

Engineers at Penn State University have created a chip that can process and classify nearly two billion images per second.

Penn Engineering Today reports that a team of researchers – including Farshid Ashtiani, Alexander J. Geers, and Firooz Aflatouni – developed the chip which is smaller than a square centimeter.

The chip can perform an entire image classification in roughly half a nanosecond and all without the need for a separate processor or memory unit.

In traditional artificial intelligence (AI) systems used for image recognition, the image of the target object is first formed on an image sensor, such as the digital camera in a smartphone.

Then, the image sensor converts light into electrical signals, and ultimately into binary data. Only then, can the system sufficiently “understand” the image to process, analyze, store, and classify it using computer chips.

While digital chips today can perform billions of computations per second, more sophisticated image classification such as identifying moving objects or 3D object identification are pushing the limits of even the most powerful technology.

The current speed limit of these technologies is set by the clock-based schedule of computation steps in a computer processor, where computations occur one after another on a linear schedule.

Penn State engineers have created the first scalable chip that classifies and recognizes images almost instantaneously — by designing a workaround that removes the most time-consuming aspects of traditional chip-based AI image processing.

Their 9.3-square-millimeter custom processor directly processes light received from an “object of interest” using what they call an “optical deep neural network.”

The researchers’ processor effectively uses “optical neurons” interconnected using optical wires, known as waveguides, to form a deep network of many layers.

Information passes through the layers, with each step helping to classify the input image into one of its learned categories.

Thanks to the chip’s ability to compute as light propagates through it to read and process optical signals directly, the researchers claim that the chip doesn’t need to store information and can perform an entire image classification in roughly half a nanosecond.

“We aren’t the first to come up with technology that reads optical signals directly but we are the first to create the complete system within a chip that is both compatible with existing technology and scalable to work with more complex data,” says Geers.

The team expects the work will have applications in automatically detecting text in photos, helping self-driving cars recognize obstacles and other computer vision tasks.

AI has been changing the world of camera technology in recent months. Earlier this year, scientists developed an AI camera that can shoot full color in total darkness.

This week, Camero-Tech announced its latest AI-powered detection system, the Xaver 1000, that allows soldiers to see through walls before attacking.

Image credits: Header photo licensed via Depositphotos.