kylemath / eeg-notebooks_v0.1

A collection of classic EEG experiments implemented with Python and Jupyter notebooks

EEG Notebooks

A collection of classic EEG experiments implemented in Python and Jupyter notebooks. This repo is a work in progress with the goal of making it easy to perform classical EEG experiments and automatically analyze data.

Currently, all experiments are implemented for the Muse EEG device and based on work done by Alexandre Barachant and Hubert Banville for the muse-lsl library.

Please see the documentation for advanced installation instructions and complete info about the project.

Getting Started

Installation

If you are a Mac user, follow the installation instructions here

You will need a Muse 2016 and Python installed on your computer. Psychopy, the stimulus presentation library that underlies most of the experiments, officially only supports Python 2. However, some users, especially those on Linux, have been able to work entirely in Python 3 without any issues.

git clone https://github.com/neurotechx/eeg-notebooks

Install all requirements.

pip install -r requirements.txt

See here for more detailed setup instructions for windows operating systems.

Running Experiments

Open the experiment you are interested in running in notebooks folder. Notebooks can be opened either with the Jupyter Notebook browser environment (run jupyter notebook) or in the nteract desktop application.

All experiments should be able to performed entirely within the notebook environment. On Windows 10, you will want to skip the bluetooth connection step and start an EEG data stream through the BlueMuse GUI.

*Note: if errors are encountered during viewing of the eeg data, try starting the viewer directly from the command line (muselsl view). Version 2 of the viewer may work better on Windows computers (muselsl view -v 2)

The basic steps of each experiment are as follows:

1. Open an LSL stream of EEG data.
2. Ensure that EEG signal quality is excellent and that there is very little noise. The standard deviation of the signal (displayed next to the raw traces) should ideally be below 10 for all channels of interest.
3. Define subject and session ID, as well as trial duration. Note: sessions are analyzed independently. Each session can contain multiple trials or 'run-throughs' of the experiments.
4. Simultaneously run stimulus presentation and recording processes to create a data file with both EEG and event marker data.
5. Repeat step 4 to collect as many trials as needed (4-6 trials of two minutes each are recommended in order to see the clearest results)
6. Load experimental data into an MNE Raw object.
7. Apply a band-pass filter to remove noise
8. Epoch the data, removing epochs where amplitude of the signal exceeded a given threshold (removes eye blinks)
9. Generate averaged waveforms from all channels for each type of stimulus presented

Notebooks in the old_notebooks folder only contain the data analysis steps (6-9). They can be used by using the run_experiments.py script (e.g python run_eeg_experiment.py Auditory_P300 15 1)

Currently available experiments:

• N170 (Faces & Houses)
• SSVEP
• Visual P300
• Cueing (Kyle Mathewson)
• Baseline (Kyle, Eye's Open vs. Closed, needs notebook made)

eeg-notebooks_v0.1's Issues

And you can implement early_stopping as well, to make sure that your model doesn't start getting worse on the validation:

https://keras.io/callbacks/#earlystopping

Gotta get that test error going down.

Wrap your plotting code into a function.

Stratify

3. Might want to use stratify on the train_test_split

Scaling model

you will want to fit a scaler (mean / standard deviation) https://scikit-learn.org/stable/modules/preprocessing.html --- on the training data, and then use that scaling model on the test/validation
Because you will not have test data available at training time... but you are using the data in the scaling currently.

I have no notion that it should be able to differentiate the two classes.

Can you do any sort of unsupervised learning to show that there are significant differences in the distribution? Plot the distributions, scatter the PCA's in 2d, do something that shows that there is a separation line... At this point,you can fiddle and futz with hyperparameters all day long, but there is nothing better than random performance for those features.

Worth trying xgboost on this problem, it is super fast / tree based (descriptive) / and powerful

Batch Decisions

So, batches... in an ideal world, you would do gradient descent on each data point. But this would take a long time, and by batching things up you can save time by doing things in parallel.
So, a batch is a collection of data points that you run through in parallel.
and this makes all the calculations pool over a set of data.
Often you use a small number for the batch size, as this gets you closer to the single data point technique.
But there is a lot of research now to use really really big batches.
the bigger the batch, the more variance you wash out.
And the less learning signal you get... so it is a bias (to the error in the batch) versus variance tradeoff
So, ideal would be batch_size = 1 but that means that the learning is slow...
As you increase (and you want to increase in powers of 2 (.. 4, 8, 16, ... for memory layout efficiency))
The ideal batch size would be as small as you can go training time wise

Either plot log(loss) or use a log scale...

http://wiki.fast.ai/index.php/Log_Loss it is what is often used and helps to compare against my mental model, might be easier to interpret.

Allow different number of sessions per subject (array of number of sessions), and allow for more than 2 sessions loaded

Normalization should be an option... you are still normalizing before splitting your train/val/test

Unbalanced classes. Stratified sampling. k-fold stratified cross validation helps

Should start using logging instead of print, it will help in the long run.

Random input

1. substitute totally random data for your input data or

2. Also, you can try np.random.shuffle(y_train) and see how it performs on randomly shuffled input data.

How does number of features relate to number of data points relate to number of parameters?

link

Rather than having a text cell before a code cell, code cells should have titles inline (and then you can hide code):

#@title SETUP

Done TODO on deep-eeg-notebooks

Done

1. this line is confusing?sub = isub

2. some long lines of code make it less readable... try to keep code within the dotted line

3. There are moments when raw sizes are printed, try to give them some context with

     `print( ' right cue data size: {}'format(tfr.data.shape))` 
      (1063, 2, 49, 49) (1063, 49, 49, 2) 
   

4. regarding epochs. Don't redefine epochs = 100 it makes co rerunning hard.

5. to_categorical() before the train test split

6. not sure what I am seeing in the 2 subplots... there is a y-axis label missing for top / bottom

• Changed Label above, multi instances

9. all your imports should go at the top of the file and not in the cell it is used in
10. for instance, there is a plot at the end of : raw.plot_psd(fmin=1, fmax=30 ); that i missed because it was below 100 of the same status line

• removed it not important

24. categorical_crossentropy was right for your categorical mod

25. loss function can be binary_crossentropy i think

    model.compile(loss='categorical_crossentropy',
          optimizer=opt,
          metrics=['accuracy'])
    

26. Looks like it is over fitting on random noise and then performing about the same as a random baseline? Two balanced classes on output would give 50% accuracy. What is the output classes distritbution of training and test sets?

• About Random 50/50

16. Looks like there is some signal found but not much. How long does training take? Time per epcoh/step
17. Plot you learning rate over time. Does it decay to zero?

• These are both in code output

18. What is your batch size? Can you try with smaller batches? Bigger batches? Are batches shuffles after each epoch?

• Tried these, small and large, no effect

23. I use Adam almost consistently for optimizer
    opt = keras.optimizers.Adam(lr=0.001, beta_1=0.9, beta_2=0.999,
    epsilon=None, decay=0.0, amsgrad=False)
24. Start with a very small model. Conv layers only. No lstm for instance. This could be a function of vanichsing gradients in the recurrent layers.
25. What is the data shape? Can you show the whole input to the model rather than feeding in recurrent?

• This is just a CNN right now, withonly two layers

17. Also, you always want to set the random seed explicitly for ML research, np.random.seed(1017) random_state in train_test_split. this will help with reproducibility testing.
18. Long output logs are confusing as fuck. Can you use tqdm or something similar to show progress rather than listing a bit meaningless status?
19. add a model.summary() somewhere, it gives you a sense of the model in plaintext. (Trainable params: 174,592) 11)
20. you use two different definitions of the word epoch in the file.
21. each cell (eg. 2) should have a status message if it succeded or failed.
22. mark each image with the answer and see if your CNN visual features can detect.
23. You can use np.ones_like and np.zeros_like to make easy to discern training data. For instance, here is a sanity check, make all the left cues ones, and all the right cues zeros... then make a model that fits on that.
24. split into train/test/validation It will help with framing the problem as a model learning and then using that model for a downstream task

Multilevel Model over subjects

You can include other data outside

so input could be a 64x64 image and a vector of real numbers. Those numbers could be fed into a layer 'after' the convolution. This is common for multi-sensor setups.

So, learning to compress EEG data to a latent space and then regenerate it would give you good features for downstream tasks. I wonder if a word vectors method has been used for EEG.

Create notebook for baseline eyes open eyes closed task (baseline_task.py)

Revert to time domain to make less assumptions about features

input into LSTM over time

May want to try this: glorot_normal https://keras.io/initializers/ for the intiialization

Brain Vision problem how to solve this?

Loading data for subject 001
Loading data for subject number: 001
Extracting parameters from gdrive/My Drive/data/P3/001_P3_ActiveDry.vhdr...

FileNotFoundError Traceback (most recent call last)
in ()
12 for session in sessions:
13 #Load Data
---> 14 raw = LoadBVData(sub,session,data_dir,exp)
15 #Pre-process EEG data
16 temp_epochs = PreProcess(raw,event_id,

7 frames
/usr/local/lib/python3.6/dist-packages/mne/io/brainvision/brainvision.py in _aux_vhdr_info(vhdr_fname)
555 def _aux_vhdr_info(vhdr_fname):
556 """Aux function for _get_vhdr_info."""
--> 557 with open(vhdr_fname, 'rb') as f:
558 # extract the first section to resemble a cfg
559 header = f.readline()

FileNotFoundError: [Errno 2] No such file or directory: '/content/gdrive/My Drive/data/P3/001_P3_ActiveDry.vhdr'

And you might want to start thinking about monitoring parameters during training with Tensorboard.

https://keras.io/callbacks/#tensorboard

Similarly, should make the model builder a function which you can call with a flag... keep the if switch but wrap it all up.