Music to my eyes: Cross-modal interactions in the perception of emotions in musical performance

@article{Vines2011MusicTM,
  title={Music to my eyes: Cross-modal interactions in the perception of emotions in musical performance},
  author={Bradley W. Vines and Carol Krumhansl and Marcelo M. Wanderley and Ioana Dalca and Daniel J. Levitin},
  journal={Cognition},
  year={2011},
  volume={118},
  pages={157-170}
}
View on Elsevier
escholarship.mcgill.ca
102 Citations
Highly Influential Citations
5
Background Citations
32
Methods Citations
4
Results Citations
9

Figures and Tables from this paper

102 Citations

Citation Type

Citation Type

Cross-Modal Perception of Emotion from Dynamic Point-Light Displays of Musical Performance
Cross-modal interactions in the perception of emotion in musical performance were investigated in this study. Although it has been widely assumed that the expressive qualities of music are…
Multisensory integration of musical emotion perception in singing
TLDR
It is demonstrated that long-term knowledge and task context affect multisensory integration in applied settings and even though singers’ orofacial movements are dominated by sound production, their facial expressions can communicate emotions composed into the music, and observes do not rely on audio information instead.
The effect of context and audio-visual modality on emotions elicited by a musical performance
TLDR
This study demonstrates the contribution of the performance setting and the performers’ appearance and nonverbal expression to emotion induction by music, encouraging further systematic research into the factors involved.
The mechanism and neural substrate of musical emotions in the audio-visual modality
Previous studies on multisensory integration (MSI) of musical emotions have yielded inconsistent results. The distinct features of the music materials and different musical expertise levels of…
Enhancing emotional experiences to dance through music: the role of valence and arousal in the cross-modal bias
TLDR
It was shown that the cross-modal bias was more pronounced for sad than for happy movements, whereas it was equivalent when contrasting high vs. low arousal movements; and that movement valence did not modulate participants’ GSR, while movement arousal did, such that GSR was potentiated in the case of low arousing movements with sad music and when high arousal movements were paired with happy music.
Crossmodal interactions in the perception of expressivity in musical performance
TLDR
Although both auditory and visual kinematic cues contribute significantly to the perception of overall expressivity, the effect of visual k Cinematic cues appears to be somewhat stronger, which provides preliminary evidence of crossmodal interactions in the perceived expressivity.
Gesture-sound causality from the audience’s perspective: : investigating the aesthetic experience of performances with digital musical instruments.
In contrast to their traditional, acoustic counterparts, digital musical instruments (DMIs) rarely feature a clear, causal relationship between the performer’s actions and the sounds produced. They…
Musicianship Enhances Perception But Not Feeling of Emotion From Others’ Social Interaction Through Speech Prosody
Music expertise has been shown to enhance emotion recognition from speech prosody. Yet, it is currently unclear whether music training enhances the recognition of emotions through other communicative…
The audio-visual integration effect on music emotion: Behavioral and physiological evidence
TLDR
The results suggest that when the music was positive, the effect of audio-visual integration was greater, and the modulation effect of the presentation durations of visual information on the music-induced emotion was more significant.
The influence of performing gesture type on interpersonal musical timing, and the role of visual contact and tempo.
...
...

References

SHOWING 1-10 OF 107 REFERENCES
Cross-modal interactions in the experience of musical performances: Physiological correlates
Cross-modal interactions in the perception of musical performance
An exploratory study of musical emotions and psychophysiology.
  • C. Krumhansl
  • Psychology
    Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale
  • 1997
TLDR
Physiological measures were recorded while listners heard two excerpts chosen to represent each of three emotions: sad, fear, and happy, and found significant differences among the excerpts.
A Cross-Cultural Investigation of the Perception of Emotion in Music: Psychophysical and Cultural Cues
Studies of the link between music and emotion have primarily focused on listeners' sensitivity to emotion in the music of their own culture. This sensitivity may reflect listeners' enculturation to…
Modeling Perceived Emotion With Continuous Musical Features
The relationship between musical features and perceived emotion was investigated by using continuous response methodology and time-series analysis. Sixty-seven participants responded to four pieces…
Audio-visual integration of emotional cues in song
We examined whether facial expressions of performers influence the emotional connotations of sung materials, and whether attention is implicated in audio-visual integration of affective cues. In…
Music: A Link Between Cognition and Emotion
Cognition and emotion are closely linked in music. The interplay between expectations and the sounded events is hypothesized to play a central role in creating musical tension and relaxation. The…
Music, movement and marimba: an investigation of the role of movement and gesture in communicating musical expression to an audience
The experiment reported in this article investigated the assumption that visual movement plays a role in musician-to-audience communication in marimba performance. Body movement is of particular…
Visual Perception of Performance Manner in the Movements of Solo Musicians
The study of music perception has focused almost exclusively on sound, ignoring the role of seeing the performer's body movements. Whilst anecdotes frequently refer to the importance of the…
Multidimensional scaling of emotional responses to music: The effect of musical expertise and of the duration of the excerpts
Musically trained and untrained listeners were required to listen to 27 musical excerpts and to group those that conveyed a similar emotional meaning (Experiment 1). The groupings were transformed…
...
...