These Earbuds Have Tiny Cameras That Take Photos and Let Users Talk to AI About What They See

Scientists have built tiny cameras into off-the-shelf wireless earbuds that capture low-resolution images to allow users to talk with an AI system about what is in front of them.

Researchers at the University of Washington have developed what they describe as the first system to place small cameras into off-the-shelf earbuds. The idea is to give users a way to interact with an AI model about the world in front of them, similar to what smart glasses aim to do. For example, someone could point their head toward a Korean food package and say, “Hey Vue, translate this for me,” and the system would respond with something like, “The visible text translates to ‘Cold Noodles’ in English.”

The prototype, called VueBuds, takes low-resolution black-and-white images using a tiny, low-power camera built into the earbuds. Those images are sent over Bluetooth to a nearby phone or device, where a small AI model processes them and responds to questions in about a second.

The processing all happens locally on the device, which the researchers say helps with privacy. A small light also turns on when images are being captured, and users can delete images right away if they want.

The system avoids streaming video altogether. That’s partly because Bluetooth can’t handle a constant flow of large data, and partly because higher-quality cameras would drain too much power. Instead, it relies on occasional still images, which keeps energy use down while still making the system usable.

A Camera That is the Size of a Grain of Rice

The camera itself is extremely small — roughly the size of a grain of rice. One of the main challenges was figuring out where to place it so it could actually “see” what the user sees.

“One big question we had was: Will your face obscure the view too much? Can earbud cameras capture the user’s view of the world reliably?” lead author Maruchi Kim, who completed the work as a doctoral student at the Paul G. Allen School of Computer Science & Engineering, says in a statement.

The team found that tilting the cameras slightly outward — about 5 to 10 degrees — gives a wide field of view of around 98 to 108 degrees. There is a small blind spot when objects are held very close, around 7.8 inches, but the researchers say that’s rarely an issue in everyday use.

Because the system only captures grayscale images, it can’t yet answer questions that depend on color. The researchers say future versions could add color cameras or more specialized AI models, but that would increase power demands.

“This study lets us glimpse what’s possible just using a general purpose language model and our wireless earbuds with cameras,” Kim explains. “But we’d like to study the system more rigorously for applications like reading a book — for people who have low vision or are blind, for instance — or translating text for travelers.”

The researchers say part of the motivation is that smart glasses and VR headsets haven’t really taken off with most people. They think earbuds might be a more practical way in, since people already wear them and they’re less noticeable.

“We haven’t seen most people adopt smart glasses or VR headsets, in part because a lot of people don’t like wearing glasses, and they often come with privacy concerns, such as recording high-resolution video and processing it in the cloud,” senior author Shyam Gollakota explains. “But almost everyone wears earbuds already, so we wanted to see if we could put visual intelligence into tiny, low-power earbuds, and also address privacy concerns in the process.”

The invention comes followings reports that Apple is preparing to launch camera-equipped AirPods as part of the company’s broader AI technology and device ambitions. Unlike wearable, camera-equipped devices like Meta’s smart glasses, these AirPods with cameras are not designed at all for actually capturing photos or videos. Instead, the cameras can capture low-resolution images of the world around the user, powering contextual AI.

The full paper on VueBuds can be read here.