Learn Piano with BACh: An Adaptive Learning Interface that Adjusts Task Difficulty Based on Brain State
@article{Yuksel2016LearnPW, title={Learn Piano with BACh: An Adaptive Learning Interface that Adjusts Task Difficulty Based on Brain State}, author={Beste F. Yuksel and Kurt B. Oleson and Lane Harrison and Evan M. Peck and Daniel Afergan and Remco Chang and Robert J. K. Jacob}, journal={Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems}, year={2016} }
We present Brain Automated Chorales (BACh), an adaptive brain-computer system that dynamically increases the levels of difficulty in a musical learning task based on pianists' cognitive workload measured by functional near-infrared spectroscopy. As users' cognitive workload fell below a certain threshold, suggesting that they had mastered the material and could handle more cognitive information, BACh automatically increased the difficulty of the learning task. We found that learners played with…
103 Citations
Hand motor learning in a musical context and prefrontal cortex hemodynamic response: a functional near-infrared spectroscopy (fNIRS) study
- Psychology, BiologyCognitive Processing
- 2019
Findings suggest an initial executive function engagement followed by facilitation of motor execution over time in response to training a piano chord progression in an ecological experimental setting.
Update of fNIRS as an Input to Brain–Computer Interfaces: A Review of Research from the Tufts Human–Computer Interaction Laboratory
- Computer SciencePhotonics
- 2019
Over the past decade, the Human–Computer Interaction (HCI) Lab at Tufts University has been developing real-time, implicit Brain–Computer Interfaces (BCIs) using functional near-infrared spectroscopy…
EEG Correlates of Difficulty Levels in Dynamical Transitions of Simulated Flying and Mapping Tasks
- Computer Science, PsychologyIEEE Transactions on Human-Machine Systems
- 2021
The results suggest that the latency of changes in the user's compound state between different tasks is a factor that should be taken into account when building adaptive HMI systems.
Neurofeedback for Personalized Adaptive Training
- PsychologyAHFE
- 2017
Preliminary results suggest task specific brain areas are coupled with the behavioral performance in a new neurofeedback based training paradigm using the wearable and portable fNIRS.
Cognitive and affective probing: A tutorial and review of active learning for neuroadaptive technology.
- Computer Science, BiologyJournal of neural engineering
- 2019
This tutorial provides guidelines to both implement, and critically discuss potential ethical implications of, novel cognitive probing applications and research endeavours.
Detecting and Adapting to Users' Cognitive and Affective State to Develop Intelligent Musical Interfaces
- ArtNew Directions in Music and Human-Computer Interaction
- 2019
This chapter considers intelligent musical interfaces that can measure cognitive or affective states implicitly in real-time to allow musically appropriate adaptations by the system without conscious effort on the part of the user.
The Cognitive Eye: Indexing Oculomotor Functions for Mental Workload Assessment in Cognition-Aware Systems
- PsychologyCHI Extended Abstracts
- 2021
A comparison of both parameters with regard to the capacity to differentiate task load in an auditory n-back task is provided, suggesting pupil diameter to display the level of mental workload.
Adaptive learning based on cognitive load using artificial intelligence and electroencephalography
- Computer Science
- 2017
In this project a driver has been utilized to sample raw sensor-data from the Emotiv Epoc EEG-headset, and a recurrent neural network has been constructed to classify fifteen different emotions from raw data provided by Swartz Center for Computational Neuroscience.
Towards Robust Neuroadaptive HCI: Exploring Modern Machine Learning Methods to Estimate Mental Workload From EEG Signals
- Computer ScienceCHI Extended Abstracts
- 2018
This paper studies promising modern machine learning algorithms, including Riemannian geometry-based methods and Deep Learning, to estimate workload from EEG signals, with both user-specific and user-independent calibration, to go towards calibration-free systems.
Entropic Brain-computer Interfaces - Using fNIRS and EEG to Measure Attentional States in a Bayesian Framework
- Computer SciencePhyCS
- 2017
This work considers new avenues for implicit BCI based on recent discoveries in cognitive neuroscience and conducts a series of experiments using BCI’s principal non-invasive brain sensors, fNIRS and EEG to show how Bayesian and systems neuroscience formulations explain the difference in performance of machine learning algorithms trained on brain data in different conditions.
References
SHOWING 1-10 OF 73 REFERENCES
Braahms: a novel adaptive musical interface based on users' cognitive state
- Computer ScienceNIME
- 2015
A new type of real-time BCI that assists users in musical improvisation by adapting to users' measured cognitive workload implicitly and building a real- time, implicit system using this brain signal that musically adapts to what users are playing.
Dynamic difficulty using brain metrics of workload
- Computer ScienceCHI
- 2014
Dynamic difficulty adjustments can be used in human-computer systems in order to improve user engagement and performance. In this paper, we use functional near-infrared spectroscopy (fNIRS) to obtain…
Task-evoked pupillary response to mental workload in human-computer interaction
- Psychology, Computer ScienceCHI EA '04
- 2004
To provide a measure of mental workload for interactive tasks, the use of task-evoked pupillary response is investigated to show that a more difficult task demands longer processing time, induces higher subjective ratings ofmental workload, and reliably evokes greater pupillaryresponse at salient subtasks.
Topographical changes in the ongoing EEG related to the difficulty of mental tasks
- Psychology, BiologyBrain Topography
- 2005
An experiment was carried out to investigate the hypothesis that task difficulty is reflected in changes in the topographical distribution of the ongoing EEG, and found that higher task difficulty resulted in EEG changes that led to the identification of two factors.
Assessing Cognitive Load in Adaptive Hypermedia Systems: Physiological and Behavioral Methods
- EducationAH
- 2004
Pupil diameter and event-related brain potentials were measured while subjects read texts of different levels of difficulty and results indicate that pupil size may not be suitable as an index of cognitive load for adaptive hypermedia systems.
High-resolution EEG mapping of cortical activation related to working memory: effects of task difficulty, type of processing, and practice.
- Psychology, BiologyCerebral cortex
- 1997
The results serve to dissociate the effects of task difficulty and practice, to differentiate the involvement of posterior cortex in spatial versus verbal tasks, to localize frontal midline theta to the anteromedial cortex, and to demonstrate the feasibility of using anatomical MRIs to remove the blurring effect of the skull and scalp from the ongoing EEG.
Dynamic functional changes associated with cognitive skill learning of an adapted version of the Tower of London task
- Psychology, BiologyNeuroImage
- 2003
Optical brain monitoring for operator training and mental workload assessment
- PsychologyNeuroImage
- 2012
Cognitive Load Measurement as a Means to Advance Cognitive Load Theory
- Psychology
- 2003
In this article, we discuss cognitive load measurement techniques with regard to their contribution to cognitive load theory (CLT). CLT is concerned with the design of instructional methods that…
Monitoring Working Memory Load during Computer-Based Tasks with EEG Pattern Recognition Methods
- Psychology, Computer ScienceHum. Factors
- 1998
The results support the feasibility of using EEG-based methods for monitoring cognitive load during human-computer interaction with neural network pattern recognition applied to EEG spectral features.