Developing Multimodal Biomarkers and Computational Models for Objective and Subjective Impairments in Altered Cognitive States Using fNIRS, EEG, and Eye-Tracking

Background

Altered cognitive states such as mental fatigue, drowsiness, and brain fog have significant impact on human performance, quality of life, and long-term health. Subjectively, individuals report mental sluggishness, memory difficulties, poor concentration, or reduced mental clarity. Objectively, these are linked to reduced task performance and lapses. Understanding and detecting these altered cognitive states is critical for performance in high-risk environments and for personalized intervention strategies in order to minimize cognitive decline and optimize individual health outcomes. Identifying biomarkers for these cognitive states will help in early detection and intervention and in managing conditions like long-covid, neurodegenerative diseases, chronic fatigue syndromes, and post-concussions syndrome. Despite its prevalence, the precise mechanisms underlying these altered cognitive states remain unclear and clear easily acquired individual biomarkers are lacking. We recently identified measures of input-driven brain complexity, i.e. the profile of the harmonic responses of high-bandwidth steady-state visual evoked potentials as measured by electroencephalography (EEG), that were predictive of fluctuations of cognitive states (attention vs. inattention and mental clarity vs. drowsiness) within individual participants (Wang, Marcucci, Peters, Braidotti, Muckli, & Faccio, 2024, Nature Communications, 15:6393), . These results suggest a path to identify biomarkers as a modulation of input/tasks-driven neural responses.

Aims

This project expands our previous findings and aims to establish individual biomarkers of altered cognitive states (mental fatigue) with novel mobile multi-modal neuroimaging, that can be obtained easily and in mobile settings: 

1. Utilize multimodal neuroimaging to identify biomarkers of altered cognitive states: The project will use a novel head-mounted device combining functional near-infrared spectroscopy (fNIRS), and EEG with concurrent mobile eye-tracking will be utilized to measure input-driven neural responses in a mobile setting. EEG and fNIRS capture complementary aspects of neural activity, making their combination particularly powerful for studying cognitive. Eye-tracking will be employed to distinguish neural responses caused by eye movements from those genuine to the brain state, enhancing the accuracy of neural measurements. 

2. Longitudinal measurements of input-driven neural responses for stratification: The project will assess input-driven neural responses over time to track changes in cognitive states. Healthy participants will complete visual (flicker) and cognitive tasks targeting attention, working memory, and executive function during mobile multimodal neuroimaging. Self-reported assessments of mental fatigue and clarity will capture subjective experiences. By collecting data across multiple sessions, the study aims to identify consistent neural patterns linked to subjective experiences and objective performance. This will enable stratification of individuals based on their cognitive state trajectories and neural response profiles. 

3. Develop advanced machine learning models for multimodal data integration: Using the collected multimodal data, the project will build joint embedding models (e.g., deep state-space models) of EEG, fNIRS, and eye-tracking data into a shared feature space. This model will leverage task conditions and self-reported mental clarity to detect neural patterns associated with mental fatigue/cognitive sluggishness. By integrating data across multiple sessions and tasks, the model aims to contribute to the creation of individualized diagnostic tools and enhance understanding of the neural mechanisms underlying altered cognitive states.

Training Outcomes

The student undertaking this project will receive interdisciplinary training in: 

1. Quantitative methods: Including advanced signal processing for fNIRS, EEG, and eye tracking data. 

2. Machine learning and computational modeling: Building and applying advanced machine learning / generative models to large-scale multimodal data. 

3. Precision medicine: Developing diagnostic tools that can be individualized based on a person’s unique cognitive and neural profile. 

4. Clinical neuroscience: Understanding the neural underpinnings of altered cognitive states and cognitive impairments with prospect to translating into clinical contexts. 

This training will provide the student with a comprehensive skill set, preparing them for future leadership in academia, healthcare, or the biomedical industry.