Publications
2026
3.
Felix Grün; Muhammad Saif-ur-Rehman; Tobias Glasmachers; Ioannis Iossifidis
Invariance to~Quantile Selection in~Distributional Continuous Control Inproceedings
In: Nicosia, Giuseppe; Ojha, Varun; Giesselbach, Sven; Pardalos, M. Panos; Umeton, Renato; Emanuele, La Malfa; Gabriele, La Malfa (Ed.): Machine Learning, Optimization, and Data Science, pp. 175–190, Springer Nature Switzerland, Cham, 2026, ISBN: 978-3-032-21477-5.
Abstract | Links | BibTeX | Tags: Actor-critic, BCI, Continuous control, Distributional reinforcement learning, Machine Learning, Quantile regression, reinforcement learning (RL)
@inproceedings{grunInvarianceQuantileSelection2026,
title = {Invariance to~Quantile Selection in~Distributional Continuous Control},
author = {Felix Grün and Muhammad Saif-ur-Rehman and Tobias Glasmachers and Ioannis Iossifidis},
editor = {Giuseppe Nicosia and Varun Ojha and Sven Giesselbach and M. Panos Pardalos and Renato Umeton and La Malfa Emanuele and La Malfa Gabriele},
doi = {10.1007/978-3-032-21477-5_12},
isbn = {978-3-032-21477-5},
year = {2026},
date = {2026-06-01},
urldate = {2026-06-01},
booktitle = {Machine Learning, Optimization, and Data Science},
pages = {175–190},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {In recent years, distributional reinforcement learning has produced many state-of-the-art results in typical reinforcement learning benchmarks, such as the suite of Atari games. Increasingly sample-efficient distributional algorithms for the discrete action domain have been developed over time, which vary primarily in the way they parameterize their approximations of value distributions, and how they quantify the differences between those distributions. In this work, we transfer three of those algorithms - Quantile Regression Deep Q-Network (QR-DQN), Implicit Quantile Networks (IQN) and Fully Parameterized Quantile Function (FQF) - to the continuous action domain by extending two powerful actor-critic algorithms - Twin Delayed Deep Deterministic policy gradient (TD3) and Soft Actor-Critic (SAC) - with distributional critics. We investigate whether the relative performance of the methods for the discrete action space translates to the continuous case. To that end, we compare them empirically on a set of continuous control tasks (Ant, HalfCheetah, Hopper, Humanoid and Walker2D). Our results indicate qualitative invariance regarding the number and placement of distributional atoms in the deterministic, continuous action setting.},
keywords = {Actor-critic, BCI, Continuous control, Distributional reinforcement learning, Machine Learning, Quantile regression, reinforcement learning (RL)},
pubstate = {published},
tppubtype = {inproceedings}
}
In recent years, distributional reinforcement learning has produced many state-of-the-art results in typical reinforcement learning benchmarks, such as the suite of Atari games. Increasingly sample-efficient distributional algorithms for the discrete action domain have been developed over time, which vary primarily in the way they parameterize their approximations of value distributions, and how they quantify the differences between those distributions. In this work, we transfer three of those algorithms - Quantile Regression Deep Q-Network (QR-DQN), Implicit Quantile Networks (IQN) and Fully Parameterized Quantile Function (FQF) - to the continuous action domain by extending two powerful actor-critic algorithms - Twin Delayed Deep Deterministic policy gradient (TD3) and Soft Actor-Critic (SAC) - with distributional critics. We investigate whether the relative performance of the methods for the discrete action space translates to the continuous case. To that end, we compare them empirically on a set of continuous control tasks (Ant, HalfCheetah, Hopper, Humanoid and Walker2D). Our results indicate qualitative invariance regarding the number and placement of distributional atoms in the deterministic, continuous action setting.
2.
Aline Xavier Fidêncio; Felix Grün; Christian Klaes; Ioannis Iossifidis
Performance Boundaries for~Brain-Computer Interfaces Using Error-Related Potentials and~Reinforcement Learning Inproceedings
In: Nicosia, Giuseppe; Ojha, Varun; Giesselbach, Sven; Pardalos, M. Panos; Umeton, Renato; Emanuele, La Malfa; Gabriele, La Malfa (Ed.): Machine Learning, Optimization, and Data Science, pp. 335–349, Springer Nature Switzerland, Cham, 2026, ISBN: 978-3-032-21480-5.
Abstract | Links | BibTeX | Tags: adaptive brain-computer interface, BCI, electroencephalography (EEG), error-related potentials (ErrPs), Machine Learning, motor imagery (MI), reinforcement learning (RL)
@inproceedings{xavierfidencioPerformanceBoundariesBrainComputer2026,
title = {Performance Boundaries for~Brain-Computer Interfaces Using Error-Related Potentials and~Reinforcement Learning},
author = {Aline Xavier Fidêncio and Felix Grün and Christian Klaes and Ioannis Iossifidis},
editor = {Giuseppe Nicosia and Varun Ojha and Sven Giesselbach and M. Panos Pardalos and Renato Umeton and La Malfa Emanuele and La Malfa Gabriele},
doi = {10.1007/978-3-032-21480-5_23},
isbn = {978-3-032-21480-5},
year = {2026},
date = {2026-06-01},
urldate = {2026-06-01},
booktitle = {Machine Learning, Optimization, and Data Science},
pages = {335–349},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {Non-invasive brain-computer interfaces (BCIs) can improve quality of life for individuals with motor disabilities. However, input data non-stationarity often leads to performance degradation over time. Adaptive BCIs aim to address this challenge. Recent studies have proposed leveraging error-related potentials (ErrPs) - neural signals elicited during self-made or observed errors - as a natural feedback mechanism for closed-loop systems. While these approaches demonstrate potential performance gains, their effectiveness relies heavily on accurate ErrP detection, which remains challenging. In a previous study, we introduced a novel reinforcement learning-based BCI framework incorporating ErrPs as reward signals. We systematically examine how misclassification rates in ErrP detection in terms of false positives (FPs) and false negatives (FNs) influence closed-loop performance. We use both synthetic and real datasets to evaluate two contextual bandit algorithms (LinUCB and NeuralUCB), trained to map motor imagery-related time-frequency modulations to actions in a binary task. Firstly, our findings show that the sensitivity of agents to FPs and FNs depends on baseline accuracies. In some conditions, such as insufficient exploration, false negatives might be more detrimental than false positives, but this needs further investigation. Proper agent parametrization and enough data samples can compensate the negative effects to some extent.},
keywords = {adaptive brain-computer interface, BCI, electroencephalography (EEG), error-related potentials (ErrPs), Machine Learning, motor imagery (MI), reinforcement learning (RL)},
pubstate = {published},
tppubtype = {inproceedings}
}
Non-invasive brain-computer interfaces (BCIs) can improve quality of life for individuals with motor disabilities. However, input data non-stationarity often leads to performance degradation over time. Adaptive BCIs aim to address this challenge. Recent studies have proposed leveraging error-related potentials (ErrPs) - neural signals elicited during self-made or observed errors - as a natural feedback mechanism for closed-loop systems. While these approaches demonstrate potential performance gains, their effectiveness relies heavily on accurate ErrP detection, which remains challenging. In a previous study, we introduced a novel reinforcement learning-based BCI framework incorporating ErrPs as reward signals. We systematically examine how misclassification rates in ErrP detection in terms of false positives (FPs) and false negatives (FNs) influence closed-loop performance. We use both synthetic and real datasets to evaluate two contextual bandit algorithms (LinUCB and NeuralUCB), trained to map motor imagery-related time-frequency modulations to actions in a binary task. Firstly, our findings show that the sensitivity of agents to FPs and FNs depends on baseline accuracies. In some conditions, such as insufficient exploration, false negatives might be more detrimental than false positives, but this needs further investigation. Proper agent parametrization and enough data samples can compensate the negative effects to some extent.
2025
1.
Aline Xavier Fidêncio; Felix Grün; Christian Klaes; Ioannis Iossifidis
Hybrid Brain-Computer Interface Using Error-Related Potential and Reinforcement Learning Journal Article
In: Frontiers in Human Neuroscience, vol. 19, 2025, ISSN: 1662-5161.
Abstract | Links | BibTeX | Tags: adaptive brain-computer interface, BCI, EEG, error-related potentials (ErrPs), Machine Learning, motor imagery (MI), reinforcement learning (RL)
@article{xavierfidencioHybridBraincomputerInterface2025,
title = {Hybrid Brain-Computer Interface Using Error-Related Potential and Reinforcement Learning},
author = {Aline Xavier Fidêncio and Felix Grün and Christian Klaes and Ioannis Iossifidis},
editor = {Frontiers},
url = {https://www.frontiersin.org/journals/human-neuroscience/articles/10.3389/fnhum.2025.1569411/full},
doi = {10.3389/fnhum.2025.1569411},
issn = {1662-5161},
year = {2025},
date = {2025-06-04},
urldate = {2025-06-04},
journal = {Frontiers in Human Neuroscience},
volume = {19},
publisher = {Frontiers},
abstract = {Brain-computer interfaces (BCIs) offer alternative communication methods for individuals with motor disabilities, aiming to improve their quality of life through external device control. However, non-invasive BCIs using electroencephalography (EEG) often suffer from performance limitations due to non-stationarities arising from changes in mental state or device characteristics. Addressing these challenges motivates the development of adaptive systems capable of real-time adjustment. This study investigates a novel approach for creating an adaptive, error-related potential (ErrP)-based BCI using reinforcement learning (RL) to dynamically adapt to EEG signal variations. The framework was validated through experiments on a publicly available motor imagery dataset and a novel fast-paced protocol designed to enhance user engagement. Results showed that RL agents effectively learned control policies from user interactions, maintaining robust performance across datasets. However, findings from the game-based protocol revealed that fast-paced motor imagery tasks were ineffective for most participants, highlighting critical challenges in real-time BCI task design. Overall, the results demonstrate the potential of RL for enhancing BCI adaptability while identifying practical constraints in task complexity and user responsiveness.},
keywords = {adaptive brain-computer interface, BCI, EEG, error-related potentials (ErrPs), Machine Learning, motor imagery (MI), reinforcement learning (RL)},
pubstate = {published},
tppubtype = {article}
}
Brain-computer interfaces (BCIs) offer alternative communication methods for individuals with motor disabilities, aiming to improve their quality of life through external device control. However, non-invasive BCIs using electroencephalography (EEG) often suffer from performance limitations due to non-stationarities arising from changes in mental state or device characteristics. Addressing these challenges motivates the development of adaptive systems capable of real-time adjustment. This study investigates a novel approach for creating an adaptive, error-related potential (ErrP)-based BCI using reinforcement learning (RL) to dynamically adapt to EEG signal variations. The framework was validated through experiments on a publicly available motor imagery dataset and a novel fast-paced protocol designed to enhance user engagement. Results showed that RL agents effectively learned control policies from user interactions, maintaining robust performance across datasets. However, findings from the game-based protocol revealed that fast-paced motor imagery tasks were ineffective for most participants, highlighting critical challenges in real-time BCI task design. Overall, the results demonstrate the potential of RL for enhancing BCI adaptability while identifying practical constraints in task complexity and user responsiveness.