Soloist: Generating Mixed-Initiative Tutorials from Existing Guitar Instructional Videos Through Audio Processing

  title={Soloist: Generating Mixed-Initiative Tutorials from Existing Guitar Instructional Videos Through Audio Processing},
  author={Bryan Wang and Mengyu Yang and Tovi Grossman},
  journal={Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems},
Learning musical instruments using online instructional videos has become increasingly prevalent. However, pre-recorded videos lack the instantaneous feedback and personal tailoring that human tutors provide. In addition, existing video navigations are not optimized for instrument learning, making the learning experience encumbered. Guided by our formative interviews with guitar players and prior literature, we designed Soloist, a mixed-initiative learning framework that automatically generates… Expand
1 Citations

Figures and Tables from this paper

Educational Technologies and Their Application to Music Education: An Action-Research Study in an Ecuadorian University
Information and Communication Technologies (ICT) are tools that are part of the process of teaching and learning music. These didactic/pedagogical resources are widely used by teachers. TheyExpand


Data-driven interaction techniques for improving navigation of educational videos
A set of techniques that augment existing video interface widgets are introduced, including: a 2D video timeline with an embedded visualization of collective navigation traces; dynamic and non-linear timeline scrubbing; data-enhanced transcript search and keyword summary; automatic display of relevant still frames next to the video; and a visual summary representing points with high learner activity. Expand
TutoriVR: A Video-Based Tutorial System for Design Applications in Virtual Reality
A VR-embedded tutorial system is developed that supplements video tutorials with 3D and contextual aids directly in the user's VR environment and users were able to use the proposed system to recreate painting tasks in VR. Expand
A Stylus-Driven Intelligent Tutoring System for Music Education Instruction
A stylus-driven intelligent tutoring system for music education instruction that aims to combine the benefits while addressing the limitations of existing instructional resources for teaching music theory, and provides an accessible educational application with an intelligent sketch user interface designed for novice students interested in learning music theory. Expand
DemoCut: generating concise instructional videos for physical demonstrations
DemoCut is introduced, a semi-automatic video editing system that improves the quality of amateur instructional videos for physical tasks and asks users to mark key moments in a recorded demonstration using a set of marker types derived from a formative study. Expand
Strummer: An interactive guitar chord practice system
The Strummer system is developed, an interactive multimedial system for guitar practice that provides data-driven and personalized practice for learners in order to identify important and easy-to-learn chords and songs. Expand
Maestoso: An Intelligent Educational Sketching Tool for Learning Music Theory
Maestoso first automatically recognizes students' sketched input of quizzed concepts, then relies on existing sketch and gesture recognition techniques to automatically recognize the input, and finally generates instructor-emulated feedback. Expand
LoopMaker: Automatic Creation of Music Loops from Pre-recorded Music
LoopMaker is presented, an interactive system that assists users in creating and exploring music loops from pre-recorded music and suggests that the quality of these loops are comparable to manually created loops by experts. Expand
MixT: automatic generation of step-by-step mixed media tutorials
MixT is presented, which automatically generates mixed media tutorials that combine the strengths of these tutorial types and ground the design in a formative study that shows that mixed-media tutorials outperform both static and video tutorials. Expand
How to Design Voice Based Navigation for How-To Videos
It is found that users' navigation objectives and intents affect the choice of referent type and referencing approach in command utterances, and recommends to support conversational strategies like sequence expansions and command queues. Expand
Exploring the Potential of an Intelligent Tutoring System for Sketching Fundamentals
An intelligent tutoring system for sketching fundamentals called Sketchtivity is designed and developed, and deployed in to six existing courses at the high school and university level during the 2017-2018 school year. Expand