• Corpus ID: 16685437

Hand and Finger Motion-Controlled Audio Mixing Interface

@inproceedings{Ratcliffe2014HandAF,
  title={Hand and Finger Motion-Controlled Audio Mixing Interface},
  author={Jarrod Ratcliffe},
  booktitle={NIME},
  year={2014}
}
This paper presents a control interface for music mixing using real time computer vision. Two input sensors are considered: the Leap Motion and the Microsoft Kinect. The author presents predominant design considerations, including improvement of the user’s sense of depth and panorama, maintaining broad accessibility through integration of the system with Digital Audio Workstation (DAW) software, and implementing a system that is portable and affordable. To provide the user with a heightened… 

Figures from this paper

Haptic and Visual feedback in 3D Audio Mixing Interfaces
TLDR
Results of the evaluation support this claim, as users preferred the interaction paradigm using the Falcon with no visual feedback, and users disliked active haptic feedback for augmented perception of 3D space or for snapping to objects.
Audio Collage as an Instrument for Musical Expression: Combining Freehand and Tangible Controllers
TLDR
This paper details ongoing research of a four-channel performance tool based on the idea of an audio collage controlled by freehand gestures and a tangible button interface, and the challenges of connecting gestural data to human proprioception in 3D space.
Force dynamics as a design framework for mid-air musical interfaces
In this paper we adopt the theory of force dynamics in human cognition as a fundamental design principle for the development of mid-air musical interfaces. We argue that this principle can provide
Stage- vs. channel-strip metaphor - comparing performance when adjusting volume and panning of a single channel in a stereo mix
TLDR
Qualitative data however, suggests that the stage metaphor is largely favoured for its intuitive interaction - confirming earlier studies.
Cinemacraft: exploring fidelity cues in collaborative virtual world interactions
TLDR
The research contributes to the field of immersive, collaborative interaction by making transparent the platform, methodology, instruments and code accessible for team members with less technological expertise, as well as developers aspiring to use interactive 3D media to promote further experimentation and conceptual discussions.
The design of audio mixing software displays to support critical listening
TLDR
It is suggested that within the context of a youth-services agency such as this, where children are involved in any kind of crisis, the use of these services is a good idea.
A portfolio of original compositions and an investigation into the use of gestures and controllers in computer-based electroacoustic music /Law Vanissa Wing Lun.
Most musical instruments have undergone evolution over several hundred years. The pursuit of excellence in instrument building helps instruments in meticulously translating musicians’s movement and

References

SHOWING 1-10 OF 18 REFERENCES
Interactive Mixing Using Wii Controller
TLDR
The design, construction and analysis of an interactive gesture-controlled audio mixing system by means of a wireless video game controller, which allows full, live control of gains, stereo panning, equalization, dynamic range compression and a variety of other effects for multichannel audio.
Multitouch Interface for Audio Mixing
TLDR
This paper proposes a new set of controls that might be used to simplify and/or improve the performance of audio mixing tasks, taking into account the spatial characteristics of modern mixing technologies such as surround and 3D audio and making use of multitouch interface technologies.
Towards an Interface for Music Mixing based on Smart Tangibles and Multitouch
TLDR
The paper presents the design of the mixing interface including the smart tangible as well as a preliminary user study involving a hands-on focus group session where 5 different control technologies are contrasted and discussed.
AWOL : Control Surfaces and Visualization for Surround Creation
This paper describes the creation of a surround sound manipulation tool and the development of pieces of sound and music where this tool, rather than a mixer or traditional music input instrument, is
ON-THE-FLY MULTI TRACK MIXING
TLDR
The concept of on-the-fly mixing is introduced, which precisely addresses these two issues: ensuring that the sound engineer/composers intentions are always preserved in the music reproduction and allowing some degree of user control.
Gestures to Operate DAW Software
TLDR
This paper reports on a user survey on mappings between 22 DAW functions and 30 single-point as well as multi-point gestures and describes design strategies to create a consistent set of gesture commands.
Multi-modal instrument: towards a platform for comparative controller evaluation
TLDR
Multi-Modal Instrument (MMI) is a developing platform for experimentation with controller mapping in a physical (modal) modelling synthesis setting, allowing a range of contemporary and traditional affordable controllers to be combined and applied to synthesis model parameters.
Gesture Control of Sound Spatialization for Live Musical Performance
TLDR
This paper provides a comparison of seven popular software spatialization systems from a control point of view, and examines human-factors issues relevant to gesture control, and draws useful conclusions regarding likely successful mapping strategies.
Cuebert: A New Mixing Board Concept for Musical Theatre
TLDR
Cuebert, a mixing board concept for musical theatre, is presented, a design that retains the physical controls demanded by sound engineers while taking advantage of multitouch display technology to allow for flexible display of dynamic and context-sensitive content.
Towards a more flexible and creative music mixing interface
TLDR
An initial prototype, which implements a stage metaphor control structure, is presented and two initial evaluation activities are briefly presented and discussed, one of which compares multi touch to two different tangible user interface interaction schemes.
...
...