The way our brains process information from various sensory channels is one of the biggest questions in psychology. How do the two most important senses, hearing and sight, interact with each other to interpret stimuli?
It is clear that humans rely heavily on representative symbols, i.e. words and language, to communicate meaning. This information enters through both visual and auditory channels, which often complement each other. For example, in a lecture it is helpful when a professor reinforces words on a slide with a verbal explanation. While language depends on the knowledge and recognition of words and grammar, a large part of human communication relies on a less representative medium: emotion.
The way we communicate emotion links visual and sensory channels. Many facial expressions are recognized globally for their emotional salience, to the point where they are decomposed to simple geometry (such as the smiley face, a universal symbol of happiness). Facial expression and speech transmit emotional information through tone, rhythm, number of pauses, and other vocalized nuances. It may follow, then, that we often pair these auditory and visual stimuli with each other to form a more coherent picture of a person’s emotional state. These simple observations have prompted many recent behavioural and electrophysiological studies.
Research has shown that music can have an effect on visual emotion (the emotional response elicited when you see a face, for example), as well as the post-stimulus time that it takes to recognize a word. Earlier this year, researchers Nidhya Logeswaran and Joydeep Bhattacharya from the University of London demonstrated the influence of music on visual emotion for non-simultaneous processes. They conducted behavioural experiments in which human adults rated the emotional content of a face on a scale of one to seven (where one is extremely sad and seven extremely happy). Prior to being shown the face, however, the subjects were played a short musical excerpt. They found that the face rating was often biased towards the emotion transmitted by the musical prime. Moreover, neutral face ratings had the highest variation based on the musical stimulus.
Music has a powerful ability to evoke emotions, though its non-representative nature can create some confusion across cultures as to what emotions a song is intended to draw forth. One way this can be addressed is to recognize a set of fundamental, basic emotions, which can be evoked without relying on a concrete meaning. For example, lullabies are sung in cultures all around the world for their ability to calm infants.
Some critics argue, however, that music as an art form appeals to the imagination and not necessarily to basic instinct emotions. A piece such as J.S. Bach’s “Crab Cannon” (composed of a line played backwards and forwards simultaneously) contains a geometric beauty, although most people wouldn’t recognize this aspect by simply listening to the piece.
Keith Oatley, Professor Emeritus at the department of applied psychology at U of T, believes that language evolved to satisfy a social function—to bring groups together in conversation—and that music has a social quality that facilitates connections between individuals. In this sense, basic emotions don’t need to be about anything in particular, they are similar to instinct, which draws a person towards comfort or away from pain.
Many people, however, feel complex, “object-oriented” emotions, such as love when they hear music. When asked if this reveals how we construct complex emotions, Oatley replied, “Yes, probably. It has to do with a person’s own experiences. As music is having its affect, people find themselves thinking different thoughts, and various memories come to mind. It is known that in different moods, your access to different kinds of memories is different.”
Does a person’s mood prior to assessing emotional face content affect their interpretation, and is suppression of unwanted emotions beneficial? “All the evidence [states] is that if you’re feeling bad for some reason, then someone comes in who is important to you, then really the way to deal with that isn’t really by trying to suppress your own emotion,” Oatley explains. “As you concentrate on [the new stimulus], then your own emotions in that interaction are likely to be more appropriate.” Although we are far from uncovering all of the brain’s mysteries, recent research has provided evidence that this cross-sensual emotion priming does occur.
