curiousbrutus / nirs-vis

NIRS-VIS is a Master Thesis Project for decoding visual stimuli from fNIRS brain data with transformers and autoencoders via Pytorch

fNIRS-Vise: Decoding Visual Experiences from fNIRS Brain Signals

fNIRS-Vise is an ambitious research project that pushes the boundaries of brain-computer interfaces (BCIs) by decoding and reconstructing visual experiences directly from fNIRS (functional near-infrared spectroscopy) brain signals.

🎯 Key Objectives Decode Visual Stimuli: Develop and refine deep learning models (transformers, autoencoders) to accurately interpret fNIRS data and extract meaningful visual representations. Reconstruct Visual Experiences: Utilize state-of-the-art generative models like Stable Diffusion to transform decoded neural patterns into visual imagery, effectively "seeing" through the mind's eye. Advance fNIRS Applications: Contribute to the growing field of fNIRS research by demonstrating the feasibility of visual decoding and paving the way for novel BCI applications.

🧠 Data & Methodology Proprietary fNIRS Data: High-quality fNIRS recordings collected during visual stimulation tasks, providing a unique resource for model training and validation. Open-Source fNIRS Datasets: Leveraging publicly available datasets (fNIRS2MW, etc.) to enhance model generalizability and robustness. Hybrid Deep Learning Architecture: Integrating the strengths of fNIRS-T, fNIRSNet, and MinD-Vis into a novel architecture optimized for fNIRS vision decoding. Transfer Learning: Utilizing knowledge from fMRI-based visual decoding (MinD-Vis) to accelerate model development and improve performance.

💡 Inspired By MinD-Vis: A groundbreaking fMRI-based visual reconstruction framework that serves as a key inspiration and reference point for NIRS-Vis.

🌐 Broader Impact fNIRS-Vise has the potential to revolutionize our understanding of the human visual system and unlock new possibilities in:

Brain-Computer Interfaces: Enabling more intuitive and immersive communication and control systems. Neurological Research: Providing insights into the neural mechanisms of visual perception and disorders. Clinical Applications: Developing diagnostic and therapeutic tools for visual impairments.

🤝 Get Involved We welcome collaborations and contributions from researchers, developers, and enthusiasts passionate about brain decoding and BCIs. Contact us to explore potential partnerships!

📜 License This project is licensed under the Apache License, Version 2.0. See the LICENSE file for details.