Portrait Analysis Reveals That The Human Face Can Express At Least 21 Emotions


How many human emotions can you capture on camera? According to a study by researchers at Ohio State University, the number is at least 21.

Throughout history, it was believed that there are just a small handful of basic human emotions: happiness, sadness, anger, fear, surprise, and disgust. Aleix Martinez, an electrical and computer engineering professor at OSU, believed that there had to be more.

“Six seemed a small number given the rainbow of possibilities of feeling and expressing emotions,” he tells TIME.

To investigate this issue, Martinez turned to photography and some fancy computer vision software. The researchers recruited 230 volunteers to pose for simple portraits. Each photograph was captured after providing the subjects with scenarios to respond to emotionally — things like smelling bad odors or hearing surprising news.

After capturing 5,000 different portraits, the team fed them to a computer for analysis of facial muscles. This technique revealed that there exist 21 distinct complex emotions that are common to humanity.

They are: happy, sad, fearful, angry, surprised, disgusted, happily surprised, happily disgusted, sadly fearful, sadly angry, sadly surprised, sadly disgusted, fearfully angry, fearfully surprised, fearfully disgusted, angrily surprised, angrily disgusted, disgustedly surprised, appalled, hatred, and awed.

22 different emotional facial expressions. The first one in the upper left is "neutral."

22 different emotional facial expressions. The first one in the upper left is “neutral.”

Newer digital cameras often have “smile detection” for automatically snapping portraits when smiles are seen in the frame, but “happy” isn’t always the look photographers want. Perhaps in the future cameras will include advanced face detection features that can trip the shutter when you look “fearfully disgusted”.

Image credits: Photographs by Aleix M. Martinez

  • David Vaughn

    This chart from my 6th grade counselor’s office begs to differ.

  • RonT

    Interesting. My educational background is in psychology, not just photography (the two go well together). Photographers are probably more aware of facial emotion expressions than many and photographs of facial emotions have been the basis of this type of research since it began.

    This is actually very old social research and a lot of work has been done on this subject. Work on facial expression of emotion goes back to at least Darwin (he wrote a book over a hundred years ago, some years after his famous “Origin of the Species’) and probably farther. Modern research goes back at least 75 years.

    In that time, probably the most famous name in facial emotion expression is Paul Ekman. His published work on facial emotions with Freisen and others has led to significant understanding in the field of detecting deception (lying). His work forms the basis of training courses for lie detection for security and law enforcement (probably most famously in modern culture it is the basis of the TV series ‘Lie to Me’) and his research has led to the analysis of hundreds of thousands of portraits of expressions.

    The core of his results suggest that there is 7 Universal Emotions, not just 7 emotions. That is 7 emotions that are expressed by all faces all over the world, regardless of culture, society or other factors and are anger, disgust, fear, happiness, sadness and surprise. All other detected emotions are variations or modifications on the core universal emotions (such displayed by the list produced by this chap). There is also evidence by a variety of researchers that some facial emotions are culturally modified – that some expressions of emotion only exist in a specific culture.

    So while his list of 21 isn’t ‘wrong’ (lots of people have suggested lists of emotions even larger than that). It just isn’t necessary as most of his list are variations on the basics and not necessary.
    Ekman’s full deception course (you can do a ‘basic course online from memory) has broken each muscle movement of each expression down into a component and then each expression is expressed as a collection of these muscle movements (it gets pretty complicated, the picture ‘manual’ for learning the system is huge). So you can actually categorise the exact type of smile that a person is making between many many different options.

    Kind of suggests that the smile detecting shutter-fire system in camera phones is a lot smarter than most people think!

  • RonT


  • Darren D. Stoner

    There are seventeen different things a guy can do when he lies to give
    himself away. A guys got seventeen pantomimes. A woman’s got twenty, but
    a guy’s got seventeen… but, if you know them, like you know your own
    face, they beat lie detectors all to hell. ~Vincent Coccotti

  • Davor Pavlic

    I think I’ll go with Paul Ekman on this. The guy spent years of research in the field living with people and watching them, not looking at some photos. And even if this Martinez would have done the same, he only made combinations of the 6 basic feelings. Even with Paul Ekmans research you could go further into deconstructing the face. One can only show disgust and one can show disgust cmobined with intense anger, that doesn’t make it a separate feeling, but a combination of two.

  • one

    Much more.

  • Graf Almassy