Multimodal and Multivariate Machine Learning Methods for Nonlinearly Coupled Oscillatory Systems

• Applicant: Professor Dr. Klaus-Robert Müller
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing ; Human Cognitive and Systems Neuroscience ; Software Engineering and Programming Languages ; Theoretical Computer Science
• Term: from 2013 to 2016
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 236447838

• Final Report Year: 2017

Final Report Abstract

In the course of this project we have contributed a set of decomposition (or factor-) methods, specifically designed for the extraction of amplitude modulated oscillatory components from highdimensional uni- and multimodal neuroimaging recordings. We have studied the strengths and weaknesses of the proposed methods using theoretical considerations as well as numerical simulations. Additionally, we have demonstrated the practical utility of our methods in a multitude of different real-world neuroimaging datasets recorded with a total of four measurement modalities (EEG, MEG, fNIRS, and fMRI). The results obtained on the real-world datasets have been in line with what was expected beforehand or with findings confirmed by the literature which demonstrates the validity of our methodological developments. Ongoing and Future Work. The presented compelling evidence for the correctness of our approach, makes it an attractive choice for future studies of amplitude modulated neural sources, their relation to other aspects of brain activity in within-subject analysis or to corresponding activity from other brains in across-subject analysis, studied in academical, commercial, or clinical settings. In particular, it is currently applied to new paradigms in cognitive and auditory neuroscience. Concerning the future work of extending the SPoC approach, we are investigating multivariate target functions further. In applications this allows for multimodal user feedback. On top of that, we would like to explore spiky noise elements within correlated components or sensor data.

Publications

• “Finding brain oscillations with power dependencies in neuroimaging data,” NeuroImage, vol. 96, pp. 334–348, 2014
  S. Dähne, V. V. Nikulin, D. Ramírez, P. J. Schreier, K.-R. Müller, and S. Haufe
  
• “Spoc: A novel framework for relating the amplitude of neuronal oscillations to behaviorally relevant parameters,” NeuroImage, vol. 86, pp. 111–122, 2014
  S. Dähne, F. C. Meinecke, S. Haufe, J. Höhne, M. Tangermann, K.-R. Müller, and V. V. Nikulin
  
• “Identifying granger causal relationships between neural power dynamics and variables of interest,” NeuroImage, vol. 111, pp. 489–504, 2015
  I. Winkler, S. Haufe, A. K. Porbadnigk, K.-R. Müller, and S. Dähne
  
• “Multivariate machine learning methods for fusing multimodal functional neuroimaging data,” Proceedings of the IEEE, vol. 103, no. 9, pp. 1507–1530, 2015
  S. Dähne, F. Bießmann, W. Samek, S. Haufe, D. Goltz, C. Gundlach, A. Villringer, S. Fazli, and K.-R. Mü ller
  
• “The berlin brain-computer interface: Progress beyond communication and control,” Frontiers in neuroscience, vol. 10, 2016
  B. Blankertz, L. Acqualagna, S. Dähne, S. Haufe, M. Schultze-Kraft, I. Sturm, M. Ušćmlic, M. A. Wenzel, G. Curio, and K.-R. Mü ller
  
• “Unsupervised classification of operator workload from brain signals,” Journal of neural engineering, vol. 13, no. 3, p. 036 008, 2016
  M. Schultze-Kraft, S. Dähne, M. Gugler, G. Curio, and B. Blankertz