• Corpus ID: 15864347

A Voice Interface for Sound Generators: adaptive and automatic mapping of gestures to sound

  title={A Voice Interface for Sound Generators: adaptive and automatic mapping of gestures to sound},
  author={Stefano Fasciani and Lonce L. Wyse},
Sound generators and synthesis engines expose a large set of parameters, allowing run-time timbre morphing and exploration of sonic space. However, control over these high-dimensional interfaces is constrained by the physical limitations of performers. In this paper we propose the exploitation of vocal gesture as an extension or alternative to traditional physical controllers. The approach uses dynamic aspects of vocal sound to control variations in the timbre of the synthesized sound. The… 

Figures from this paper

Non-speech voice for sonic interaction: a catalogue
It is pointed out that while voice-based techniques are already being used proficiently in sound retrieval and sound synthesis, their use in sound design is still at an exploratory phase.
This paper proposes to use the voice as a source of realvalued and time-continuous control signals that can be employed to interact with any multidimensional human-controllable device in real-time.
SynthAssist: Querying an Audio Synthesizer by Vocal Imitation
The method presented in this paper can be used to search through large databases of previously existing “factory presets” or program a synthesizer using the data-driven approach to automatic synthesizer programming.
A Self-Organizing Gesture Map for a Voice-Controlled Instrument Interface
An unsupervised offline method that learns how to reduce and map the gestural data to a generic instrument parameter control space and applies this method to the existing Voice-Controlled Interface for musical instruments, obtaining sensible usability improvements.
Deep generative models for musical audio synthesis
Recent generative deep learning systems for audio synthesis are able to learn models that can traverse arbitrary spaces of sound defined by the data they train on.
Sketching sonic interactions by imitation-driven sound synthesis
The integration of these two software packages provides an environment in which sound designers can go from concepts, through exploration and mocking-up, to prototyping in sonic interaction design, taking advantage of all the possibilities offered by vocal and gestural imitations in every step of the process.
Real-valued parametric conditioning of an RNN for interactive sound synthesis
The focus of this paper is on conditioning data-driven synthesis models with real-valued parameters on the ability of the system to generalize and to be responsive to parameter values and sequences not seen during training.
Growing the practice of vocal sketching
A set of introductory exercises on vocal sketching, to probe the communication effectiveness of vocal imitations for design purposes, are presented and discussed, in the scope of the research-through-design workshop activities of the EU project SkAT-VG.
Body as controller
Why finding an answer to three questions about the mathematical form of the mapping function from the movement feature to the sound parameters is worthwhile is discussed and possible solutions that require further investigation are provided.
The Body Electric: A NIME designed through and with the somatic experience of singing
It is shown that NIME design using principles from soma design can offer creative opportunities in developing novel sensing mechanisms, which can in turn inform composition and further elicit curious engagements between performer and artefact, disrupting notions of performer-led control.


Making music through real-time voice timbre analysis: machine learning and timbral control
This thesis develops approaches that can be used with a wide variety of musical instruments by applying machine learning techniques to automatically derive the mappings between expressive audio input and control output, with a focus on timbral control.
Strategies of mapping between gesture data and synthesis model parameters using perceptual spaces
Several implications of the mapping strategies are discussed: the influence of chosen mapping limits onto performers' virtuosity, and the incidence of mapping on the learning process with virtual instruments and on improvisation possibilities.
The Singing Tree:: design of an interactive musical interface
The Singing Tree is a novel interactive musical intertlce which responds to vocai rnput with real-time aurai and visuai feedback, providing a reward-orientedrelationshipbetweenthe sounds one makes and the synthesized music one hears.
Multidimensional perceptual scaling of musical timbres.
  • J. Grey
  • Physics
    The Journal of the Acoustical Society of America
  • 1977
Two experiments were performed to evaluate the perceptual relationships between 16 music instrument tones, and a three‐dimensional scaling solution was found to be interpretable in terms of the spectral energy distribution.
A Gesture follower for performing arts
A gesture follower for performing arts is developed, which indicates in real-time the time correspondences between an observed gesture sequence and a fixed reference gesture sequence, to facilitate the computation of parameters related to interpretation and expression.
Singing-driven interfaces for sound synthesizers
Els instruments musicals digitals es descomponen usualment en dues parts: la interficie d'usuari i el motor de sintesi. Tradicionalment la interficie d'usuari pren el nom de controlador musical.
Real-time human interaction with supervised learning algorithms for music composition and performance
This thesis presents a general-purpose software system for applying standard supervised learning algorithms in music and other real-time problem domains, called the Wekinator, which supports human interaction throughout the entire supervised learning process, including the generation of training examples and the application of trained models to real- time inputs.
Gabor, Multi-Representation Real-Time Analysis/Synthesis
Gabor provides a unified framework for granular synthesis, PSOLA, phase vocoder, additive synthesis and other STFT techniques, combining various models, representations and timing paradigms from Max/MSP.
Re-Designing Principles for Computer Music Controllers: a Case Study of SqueezeVox Maggie
This paper revisits/extends “Principles for Designing Computer Music Controllers” (NIME 2001), subsequently updated in a NIME 2007 keynote address, and uses a redesign of SqueezeVox Maggie as an example of which principles have held fast over the years, and which have changed due to advances in technology.
On the Choice of Mappings Based On Geometric Properties
A method of comparison is applied to existing techniques, while a suggestion is offered on how to integrate and extend this work through a new implementation.