Learn Piano with BACh: An Adaptive Learning Interface that Adjusts Task Difficulty Based on Brain State

@article{Yuksel2016LearnPW,
  title={Learn Piano with BACh: An Adaptive Learning Interface that Adjusts Task Difficulty Based on Brain State},
  author={Beste F. Yuksel and Kurt B. Oleson and Lane Harrison and Evan M. Peck and Daniel Afergan and Remco Chang and Robert J. K. Jacob},
  journal={Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems},
  year={2016}
}
We present Brain Automated Chorales (BACh), an adaptive brain-computer system that dynamically increases the levels of difficulty in a musical learning task based on pianists' cognitive workload measured by functional near-infrared spectroscopy. As users' cognitive workload fell below a certain threshold, suggesting that they had mastered the material and could handle more cognitive information, BACh automatically increased the difficulty of the learning task. We found that learners played with… 
Hand motor learning in a musical context and prefrontal cortex hemodynamic response: a functional near-infrared spectroscopy (fNIRS) study
TLDR
Findings suggest an initial executive function engagement followed by facilitation of motor execution over time in response to training a piano chord progression in an ecological experimental setting.
Update of fNIRS as an Input to Brain–Computer Interfaces: A Review of Research from the Tufts Human–Computer Interaction Laboratory
Over the past decade, the Human–Computer Interaction (HCI) Lab at Tufts University has been developing real-time, implicit Brain–Computer Interfaces (BCIs) using functional near-infrared spectroscopy
EEG Correlates of Difficulty Levels in Dynamical Transitions of Simulated Flying and Mapping Tasks
TLDR
The results suggest that the latency of changes in the user's compound state between different tasks is a factor that should be taken into account when building adaptive HMI systems.
Neurofeedback for Personalized Adaptive Training
TLDR
Preliminary results suggest task specific brain areas are coupled with the behavioral performance in a new neurofeedback based training paradigm using the wearable and portable fNIRS.
Cognitive and affective probing: A tutorial and review of active learning for neuroadaptive technology.
TLDR
This tutorial provides guidelines to both implement, and critically discuss potential ethical implications of, novel cognitive probing applications and research endeavours.
Detecting and Adapting to Users' Cognitive and Affective State to Develop Intelligent Musical Interfaces
TLDR
This chapter considers intelligent musical interfaces that can measure cognitive or affective states implicitly in real-time to allow musically appropriate adaptations by the system without conscious effort on the part of the user.
The Cognitive Eye: Indexing Oculomotor Functions for Mental Workload Assessment in Cognition-Aware Systems
TLDR
A comparison of both parameters with regard to the capacity to differentiate task load in an auditory n-back task is provided, suggesting pupil diameter to display the level of mental workload.
Adaptive learning based on cognitive load using artificial intelligence and electroencephalography
TLDR
In this project a driver has been utilized to sample raw sensor-data from the Emotiv Epoc EEG-headset, and a recurrent neural network has been constructed to classify fifteen different emotions from raw data provided by Swartz Center for Computational Neuroscience.
Towards Robust Neuroadaptive HCI: Exploring Modern Machine Learning Methods to Estimate Mental Workload From EEG Signals
TLDR
This paper studies promising modern machine learning algorithms, including Riemannian geometry-based methods and Deep Learning, to estimate workload from EEG signals, with both user-specific and user-independent calibration, to go towards calibration-free systems.
Entropic Brain-computer Interfaces - Using fNIRS and EEG to Measure Attentional States in a Bayesian Framework
TLDR
This work considers new avenues for implicit BCI based on recent discoveries in cognitive neuroscience and conducts a series of experiments using BCI’s principal non-invasive brain sensors, fNIRS and EEG to show how Bayesian and systems neuroscience formulations explain the difference in performance of machine learning algorithms trained on brain data in different conditions.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 73 REFERENCES
Braahms: a novel adaptive musical interface based on users' cognitive state
TLDR
A new type of real-time BCI that assists users in musical improvisation by adapting to users' measured cognitive workload implicitly and building a real- time, implicit system using this brain signal that musically adapts to what users are playing.
Dynamic difficulty using brain metrics of workload
Dynamic difficulty adjustments can be used in human-computer systems in order to improve user engagement and performance. In this paper, we use functional near-infrared spectroscopy (fNIRS) to obtain
Task-evoked pupillary response to mental workload in human-computer interaction
TLDR
To provide a measure of mental workload for interactive tasks, the use of task-evoked pupillary response is investigated to show that a more difficult task demands longer processing time, induces higher subjective ratings ofmental workload, and reliably evokes greater pupillaryresponse at salient subtasks.
Topographical changes in the ongoing EEG related to the difficulty of mental tasks
TLDR
An experiment was carried out to investigate the hypothesis that task difficulty is reflected in changes in the topographical distribution of the ongoing EEG, and found that higher task difficulty resulted in EEG changes that led to the identification of two factors.
Assessing Cognitive Load in Adaptive Hypermedia Systems: Physiological and Behavioral Methods
TLDR
Pupil diameter and event-related brain potentials were measured while subjects read texts of different levels of difficulty and results indicate that pupil size may not be suitable as an index of cognitive load for adaptive hypermedia systems.
High-resolution EEG mapping of cortical activation related to working memory: effects of task difficulty, type of processing, and practice.
TLDR
The results serve to dissociate the effects of task difficulty and practice, to differentiate the involvement of posterior cortex in spatial versus verbal tasks, to localize frontal midline theta to the anteromedial cortex, and to demonstrate the feasibility of using anatomical MRIs to remove the blurring effect of the skull and scalp from the ongoing EEG.
Cognitive Load Measurement as a Means to Advance Cognitive Load Theory
In this article, we discuss cognitive load measurement techniques with regard to their contribution to cognitive load theory (CLT). CLT is concerned with the design of instructional methods that
Monitoring Working Memory Load during Computer-Based Tasks with EEG Pattern Recognition Methods
TLDR
The results support the feasibility of using EEG-based methods for monitoring cognitive load during human-computer interaction with neural network pattern recognition applied to EEG spectral features.
...
1
2
3
4
5
...