AI Can Recreate Images From Human Brain Waves

A team of researchers demonstrated that they can decode human brain waves and show the image that a person is picturing in their mind.

Researchers from the National University of Singapore, the Chinese University of Hong Kong, and Stanford University, trained artificial intelligence (AI) to recreate photos seen by humans using only their brain waves with astonishing accuracy.

In a paper published in November last year, the team led by Zijiao Chen used brain scans of participants as they looked at more than 1,000 images while inside an fMRI (functional magnetic resonance imaging) machine.

Among the 1,000 pictures shown to the participants, there was an image of a red firetruck, a gray building, a baseball player, and a giraffe eating leaves.

The machine recorded the resulting brain signals over time. The researchers then sent those signals through an AI model to train it to associate specific brain patterns with certain images.

After this process, the participants were shown new images while in the fMRI machine. The system detected the patient’s brain waves and generated a shorthand description of what images the brain waves related to.

The system then used an AI image generator to produce the image that it deduced that the participant saw.

According to NBC News, the resulting generated image matched the attributes, such as color and shape, as well as the semantic meaning of the original image shown to the participant roughly 84 percent of the time.

A Future Without Smartphones, Only Thoughts

The researchers believe that the technology could be available for use by everyone in the next ten years.

However, currently, it takes approximately 20 hours to train the AI model on each individual’s brain activity before it can recreate the images from fMRI data.

“It might be able to help disabled patients to recover what they see, what they think,” Chen tells NBC News.

Chen believes that the model will mean that humans will no longer require smartphones to communicate: “We can just think.”

The study is the latest example of how scientists are attempting to discover how AI models can work with the human brain to recreate images.

In February, PetaPixel reported on a group of Japanese scientists who used people’s brain activity to generate images on the AI model Stable Diffusion.

And last year, researchers at Radboud University in the Netherlands were developing “mind-reading” technology that can translate a person’s brainwaves into photographic images.


Image credits: All photos sourced from “Seeing Beyond the Brain: Conditional Diffusion Model with Sparse Masked Modeling for Vision Decoding” by Zijiao Chen, Jiaxin Qing, Tiange Xiang, Wan Lin Yue, Juan Helen Zhou.

Discussion