Music Interfaces Based on Automatic Music Signal Analysis: New Ways to Create and Listen to Music

  title={Music Interfaces Based on Automatic Music Signal Analysis: New Ways to Create and Listen to Music},
  author={Masataka Goto and Roger B. Dannenberg},
  journal={IEEE Signal Processing Magazine},
Music analysis based on signal processing offers new ways of creating and listening to music. This article focuses on applications and interfaces that are enabled by advances in automatic music analysis. By using signal processing, some of these applications provide nonexperts the chance to enjoy music in their daily lives, while other applications apply signal processing to enhance professional music production and create new opportunities for composers and performers. Described in this… 

Figures from this paper

Intelligent User Interfaces for Music Discovery
This article argues that three major developments have transformed and shaped user interfaces during this period of MIR-driven intelligent user interfaces for music browsing and discovery, each connected to a phase of new listening practices.
Intelligent User Interfaces for Music Discovery: The Past 20 Years and What's to Come
It is argued that three major developments have transformed and shaped user interfaces during this period, each connected to a phase of new listening practices: first, connected to personal music collections, intelligent audio processing and content description algorithms that facilitate the automatic organization of repositories and finding music according to sound qualities.
Music Tower Blocks: Multi-Faceted Exploration Interface for Web-Scale Music Access
Music Tower Blocks offers the user to connect to their personal music streaming profiles and highlight on the landscape their favorite or recently-listened-to music, to support exploring parts of the landscape near to (or far-away from) their own taste.
Analysis of Music Teaching in Basic Education Integrating Scientific Computing Visualization and Computer Music Technology
  • Y. Zhao
  • Education
    Mathematical Problems in Engineering
  • 2022
In the current music education, the music teaching method is too simple, boring, relatively backward, unable to attract the attention of the students, and the students gradually lose their interest
Deep symbolic processing of human-performed musical sequences
This paper describes an ongoing research that uses cascading connected layers of symbolic processing as the core of a human-performed error identification and characterization module able to overcome the complexity of the studied open-ended domain.
Unveiling the Hierarchical Structure of Music by Multi-Resolution Community Detection
A novel methodology for the hierarchical analysis of music structure that is based on graph theory and multi-resolution community detection is proposed, which can perform both the tasks of boundary detection and structural grouping, without the need of particular constraints that would limit the resulting segmentation.
Deep Learning in Music Recommendation Systems
  • M. Schedl
  • Computer Science
    Front. Appl. Math. Stat.
  • 2019
This review article explains particularities of the music domain in RS research and gives an overview of the state of the art that employs deep learning for music recommendation, in particular in the context of the current research on deep learning.
Kiite Cafe: A Web Service for Getting Together Virtually to Listen to Music
By analyzing the behavior logs of 1,760 Kiite Cafe users over about five months, it is quantitatively show that these user experiences can generate various effects (e.g., users react to a more diverse range of songs on Kiite cafe than when listening alone).
Analysis of Audio Vocalizations in the Context of the Teaching and Learning of Singing
This project proposes the objective analysis of the audio signal of a set of vocalizations using the fundamental frequency as the essential feedback parameter and identifies 3 zones that correspond to the typical vibrato of the human voice.


Active Music Listening Interfaces Based on Signal Processing
  • Masataka Goto
  • Art
    2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07
  • 2007
Seven interfaces for active music listening are developed, such as interfaces for skipping sections of no interest within a musical piece while viewing a graphical overview of the entire song structure, for displaying virtual dancers or song lyrics synchronized with the music, for changing the timbre of instrument sounds in compact-disc recordings.
Frontiers of music information research based on signal processing
  • Masataka Goto
  • Art
    2014 12th International Conference on Signal Processing (ICSP)
  • 2014
This keynote paper will discuss how music technologies will open up new possibilities in both music appreciation and music creation.
Songle: A Web Service for Active Music Listening Improved by User Contributions
A public web service for active music listening, Songle, that enriches music listening experiences by using music-understanding technologies based on signal processing and proposes a mechanism of collaborative training for music- Understanding technologies, in which corrected errors will be used to improve the music- understanding performance through machine learning techniques.
Music Structure Analysis from Acoustic Signals
Music is full of structure, including sections, sequences of distinct musical textures, and the repetition of phrases or entire sections. The analysis of music audio relies upon feature vectors that
A chorus section detection method for musical audio signals and its application to a music listening station
  • Masataka Goto
  • Computer Science
    IEEE Transactions on Audio, Speech, and Language Processing
  • 2006
An application of the RefraiD method, a new music-playback interface for trial listening called SmartMusicKIOSK, which enables a listener to directly jump to and listen to the chorus section while viewing a graphical overview of the entire song structure.
Fundamentals of Music Processing: Audio, Analysis, Algorithms, Applications
This textbook provides both profound technological knowledge and a comprehensive treatment of essential topics in music processing and music information retrieval, offering detailed technological insights as well as a deep understanding of music processing applications.
Automatic music transcription: challenges and future directions
Limits of current transcription methods are analyzed and promising directions for future research are identified, including the integration of information from multiple algorithms and different musical aspects.
Classical Music on the Web - User Interfaces and Data Representations
A set of web-based user interfaces for explorative analysis and visualization of classical orchestral music and a web API that serves as a backend to those applications are presented and used to provide real-time information to the audience during a live concert of the RCO.
State of the Art Report: Audio-Based Music Structure Analysis
This paper gives an overview of state-of-theart methods for computational music structure analysis, where the general goal is to divide an audio recording into temporal segments corresponding to musical parts and to group these segments into musically meaningful categories.
Drumix: An Audio Player with Real-time Drum-part Rearrangement Functions for Active Music Listening
This paper presents a highly functional audio player, called Drumix, that allows a listener to control the volume, timbre, and rhythmic patterns (drum patterns)of bass and snare drums within existing