Benchmarking Brain-Computer Interfaces Outside the Laboratory: The Cybathlon 2016

  title={Benchmarking Brain-Computer Interfaces Outside the Laboratory: The Cybathlon 2016},
  author={Domen Novak and Roland Sigrist and Nicolas Gerig and Dario Wyss and Ren{\'e} Bauer and Ulrich G{\"o}tz and Robert Riener},
  journal={Frontiers in Neuroscience},
This paper presents a new approach to benchmarking brain-computer interfaces (BCIs) outside the lab. A computer game was created that mimics a real-world application of assistive BCIs, with the main outcome metric being the time needed to complete the game. This approach was used at the Cybathlon 2016, a competition for people with disabilities who use assistive technology to achieve tasks. The paper summarizes the technical challenges of BCIs, describes the design of the benchmarking game… 

Figures and Tables from this paper

Design Considerations for Long Term Non-invasive Brain Computer Interface Training With Tetraplegic CYBATHLON Pilot
It is demonstrated that BCI-based calibration paradigms with active user-engagement, such as with real-time feedback, could help in achieving better user acceptability and performance.
Long-Term Mutual Training for the CYBATHLON BCI Race With a Tetraplegic Pilot: A Case Study on Inter-Session Transfer and Intra-Session Adaptation
This long-term study demonstrates that regular training helped the pilot to significantly increase the distance between task-specific patterns, which resulted in an improvement of performance, both with respect to class separability in the calibration data, and withrespect to the game.
The Cybathlon BCI race: Successful longitudinal mutual learning with two tetraplegic users
This work aims at corroborating the importance and efficacy of mutual learning in motor imagery (MI) brain–computer interface (BCI) by leveraging the insights obtained through our participation in
Competing at the Cybathlon Championship for Athletes With Disabilities: Long-Term Motor Imagery Brain-Computer Interface Training of a Tetraplegic Cybathlete
Background: The brain-computer interface (BCI) race at the Cybathlon championship for athletes with disabilities challenges teams (BCI researchers, developers and pilots with spinal cord injury) to
A Motor Imagery-based Brain-Computer Interface Scheme for a Spinal Muscular Atrophy Subject in CYBATHLON Race
A four-class MI-BCI system was developed for an SMA subject to participate in the CYBATHLON competition and the system's feasibility was demonstrated by introducing the general designs, training procedures, offline processing and online BCI-game setup.
Long-Term BCI Training of a Tetraplegic User: Adaptive Riemannian Classifiers and User Training
This work proposes and evaluates the design of a multi-class Mental Task (MT)-based BCI for longitudinal training of a tetraplegic user for the CYBATHLON BCI series 2019 and reports on the evolution of the user's neurophysiological patterns and user experience throughout the BCI training and competition.
A Transfer Learning Algorithm to Reduce Brain-Computer Interface Calibration Time for Long-Term Users
The proposed r-KLwDSA algorithm was particularly successful in improving the BCI accuracy of the sessions that had initial session-specific accuracy below 60%, with an average improvement of around 10% in the accuracy, leading to more stroke patients having meaningful BCI rehabilitation.
Consistency of Motor-Imagery Frequency Band is Associated with the Performance of Real-Time Brain Computer Interface
The results showed that the efficacy of the BCI system highly relied on the stability of the individual frequency pattern, and the classification rate was associated with the consistency of theindividual ERD/ERS frequency bands across different runs.
Current Status, Challenges, and Possible Solutions of EEG-Based Brain-Computer Interface: A Comprehensive Review
This article provides a comprehensive review of the state-of-the-art of a complete BCI system and a considerable number of popular BCI applications are reviewed in terms of electrophysiological control signals, feature extraction, classification algorithms, and performance evaluation metrics.
The CYBATHLON - Bionic Olympics to Benchmark Assistive Technologies
The Cybathlon is a unique championship in which people with physical disabilities compete against each other to complete everyday tasks using latest robotic technology and offers a platform to drive forward research and challenge the usability of assistive robots.


Towards Noninvasive Hybrid Brain–Computer Interfaces: Framework, Practice, Clinical Application, and Beyond
This article presents an hBCI framework, which was used in studies with nonimpaired as well as end users with motor impairments, and review the work of a large scale integrated project funded by the European commission which was dedicated to develop practical hybrid BCIs and introduce them in various fields of applications.
Flaws in current human training protocols for spontaneous Brain-Computer Interfaces: lessons learned from instructional design
This literature study highlights that current spontaneous BCI user training procedures satisfy very few of these requirements and hence are likely to be suboptimal and proposes new research directions that are theoretically expected to address some of these flaws and to help users learn the BCI skill more efficiently.
A closed-loop human simulator for investigating the role of feedback control in brain-machine interfaces.
This online prosthesis simulator (OPS) is used to optimize "online" decode performance based on a key parameter of a current state-of-the-art decode algorithm, the bin width of a Kalman filter, and shows that offline and online analyses indeed suggest different parameter choices.
Brain Computer Interfaces, a Review
The state-of-the-art of BCIs are reviewed, looking at the different steps that form a standard BCI: signal acquisition, preprocessing or signal enhancement, feature extraction, classification and the control interface.
Psychological predictors of SMR-BCI performance
An SSVEP BCI to Control a Hand Orthosis for Persons With Tetraplegia
This study demonstrates a BCI system for orthosis control that was asynchronous, meaning that subjects could move the orthosis whenever they wanted, instead of pacing themselves to external cues.
Flight simulation using a Brain-Computer Interface: A pilot, pilot study
Current trends in hardware and software for brain-computer interfaces (BCIs).
The results indicate that BCI technology is in transition from isolated demonstrations to systematic research and commercial development, including the development of better integrated and more robust BCI hardware and software, the definition of standardized interfaces, and theDevelopment of certification, dissemination and reimbursement procedures.
The BCI competition 2003: progress and perspectives in detection and discrimination of EEG single trials
The BCI Competition 2003 was organized to evaluate the current state of the art of signal processing and classification methods and the results and function of the most successful algorithms were described.