Epoch Rejection with t-start about mne-python

[dengemann]

good you mention this. i also have similar use cases. we often do experiments that combine block- + event-related design features where you have events which occur against the background of a condition instead of baseline. E.g. a face that performs 10 randomly timed gaze shifts to random directions. As certain features of the face are condition specific i can't use the pre-event period for baselining but would rather use little periods of rest between the blocks which i have to do manually. I would be happy if i could do it with a (@agramfort : optionally) smarter baseline / reject functionality. This would involve also being able to pass arbitrary time segments as baseline. case 1 : one resting-state base line for all events; case 2 shifting resting state baselines one for the first 10 epochs, another one for the next 10 and so forth. This could be done in a way that it dosent interfer with the existing options. Wdyt?

[agramfort]

could you maybe think of a subclass on your own to play with this and see how it flies? I just don't to see too much complexity coming so quickly.

[dengemann]

Yes, good idea. Let's do it this way, then we can see how complex it would get.

[christianbrodbeck]

To actually create my baseline, I simply create a second Epochs object. I then "align" the two Epochs objects so that every pair of epochs with the same index correspond to the same trial, and then it's a simple subtraction. I think that might be a simpler solution than adding it to the Epochs initialization.

[agramfort]

To actually create my baseline, I simply create a second Epochs object. I then "align" the two Epochs objects so that every pair of epochs with the same index correspond to the same trial, and then it's a simple subtraction. I think that might be a simpler solution than adding it to the Epochs initialization.

let's KISS :)

[dengemann]

The stuff we don't add to the API for KISS reasons we should expose as examples in public gists linked from the DOC / the webpage. I'm sure many users would like that. What you describe could be a good candidate, but also the function I use to measure button press latencies from epochs to reject epochs in which participants pressed to early, to late or just the wrong button: https://gist.github.com/4215754

[christianbrodbeck]

let's KISS :)

well...

@dengemann
I am using my factors to keep track of which events correspond in the two Epochs objects (which makes the surface procedure rather simple, but since those are not in mne (yet) it does not make for a good public example :) ). Does the way you manage events make it harder? I can give you an example using my factors if that would help.

[dengemann]

Ok, true. But sure, I'd be happy to see how you do it. We could also consider to do some experimenting on this together using a subclass on a separate branch as @agramfort proposed and then see where to divide between mne-python source and mne-python sophistacted, outstanding, smart, (etc. ...) usage examples... Then we could feed back the solutions considered simple and general enough to mne-python.

[christianbrodbeck]

@dengemann Sure, would you mind installing/forking my https://github.com/christianmbrodbeck/Eelbrain package? That would save me some surgery of transplanting classes into a new mne-python branch before we know what exactly to transplant :)

[dengemann]

Sure, sounds good ;-) Let me first finish event_id saving #238

On Sat, Dec 8, 2012 at 7:23 PM, Christian Brodbeck <[email protected]

wrote:

@dengemann https://github.com/dengemann Sure, would you mind
installing/forking my https://github.com/christianmbrodbeck/Eelbrainpackage? That would save me some surgery of transplanting classes into a
new mne-python branch before we know what exactly to transplant :)

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/240#issuecomment-11161725.

[dengemann]

ok, cloned. If @agramfort, @mluessi, @Eric89GXL are fine with it let's
create a smarter epochs testing branch on mne-python. Otherwise let's do
the same on Eelbrain -- btw. where does the name come from?

On Sat, Dec 8, 2012 at 7:28 PM, Denis-Alexander Engemann <
[email protected]> wrote:

Sure, sounds good ;-) Let me first finish event_id saving #238

On Sat, Dec 8, 2012 at 7:23 PM, Christian Brodbeck <
[email protected]> wrote:

@dengemann https://github.com/dengemann Sure, would you mind
installing/forking my https://github.com/christianmbrodbeck/Eelbrainpackage? That would save me some surgery of transplanting classes into a
new mne-python branch before we know what exactly to transplant :)

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/240#issuecomment-11161725.

[christianbrodbeck]

smarter epochs

The way I've been using them is actually the opposite way: I use a dataset that contains the epochs object. That way we could keep the modifications on the Epochs object minimal and the use of the "advanced" classes optional.

How should we go about this? I could create an example using the mne sample data? In the repo? or in a dropbox folder? How can we specify the sample data path so that it works for you and me? Make the example in an mne-branch?

where does the name come from

Uh, that's somewhat complicated :) I originally started using Python for skin conductance analysis, so I called my first script collection eel-meter (any similarity to the name of an existing device is pure coincidence). When I started EEG I changed to -brain.

[dengemann]

On Sat, Dec 8, 2012 at 8:22 PM, Christian Brodbeck <[email protected]

wrote:

smarter epochs

The way I've been using them is actually the opposite way: I use a dataset
that contains the epochs object. That way we could keep the modifications
on the Epochs object minimal and the use of the "advanced" classes
optional.

can you elaborate this, I'm somehow stuck with 'the opposite way'.

How should we go about this? I could create an example using the mne
sample data? In the repo? or in a dropbox folder? How can we specify the
sample data path so that it works for you and me? Make the example in an
mne-branch?

Yes, if no one protests lets make an mne-branch and create an example
folder there for smart-stuff. Otherwise let's use one of our private forked
mne-repos.

where does the name come from

Uh, that's somewhat complicated :) I originally started using Python for
skin conductance analysis, so I called my first script collection eel-meter
(any similarity to the name of an existing device is pure coincidence).
When I started EEG I changed to -brain.

The reason I'm asking this is much funnier, I was reading 'Edelbrain' ---
top-down priming --- all the time. Edel means noble / precious / deluxe in
German and there's some Bavarian connotation to it ;-)

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/240#issuecomment-11162393.

[christianbrodbeck]

can you elaborate this, I'm somehow stuck with 'the opposite way'.

well, let's make an example

lets make an mne-branch and create an example
folder there for smart-stuff. Otherwise let's use one of our private forked
mne-repos.

I don't think I have write access. I can make a branch in my fork? Will that be easy for you to checkout?

[dengemann]

I have. But would go easy on it. Let's wait and listen what the rest of us
suggests.

On Sat, Dec 8, 2012 at 9:01 PM, Christian Brodbeck <[email protected]

wrote:

can you elaborate this, I'm somehow stuck with 'the opposite way'.

well, let's make an example

lets make an mne-branch and create an example
folder there for smart-stuff. Otherwise let's use one of our private forked
mne-repos.

I don't think I have write access. I can make a branch in my fork? Will
that be easy for you to checkout?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/240#issuecomment-11162858.

[dengemann]

@agramfort @mluessi @EricGXL89, @christianmbrodbeck and I would like to
conduct some smart epochs experiments on a related upstream branch. Is this
fine with you or shall we go private for this?

[mluessi]

I think it's fine to have an experimental feature branch hosted in the main repo, but let's see what @agramfort has to say before we make a branch. Also, I think the goal should be to come up with an advanced, but easy to use (especially for people who are new to Python) Epochs object, which will be included in 0.6.

[dengemann]

That's why I thought a real upstream branch would be nice because it
enforces proper commits and development rather than private coding.
@christianmbrodbeck let's then start with a branch in the private forks
where we can checkout Eelbrain features and then commit to the proposed dev
branch as we proceed.

On Sat, Dec 8, 2012 at 11:11 PM, Martin Luessi [email protected]:

I think it's fine to have an experimental feature branch hosted in the
main repo, but let's see what @agramfort https://github.com/agramforthas to say before we make a branch. Also, I think the goal should be to
come up with an advanced, but easy to use (especially for people who are
new to Python) Epochs object, which will be included in 0.6.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/240#issuecomment-11164396.

[christianbrodbeck]

I added 3 basic examples of how I have been using events to a dev-events branch. At various stages check the repr of the objects, they should be useful. Information about the classes is also here. Let me know of anything that is unclear or that you would like more information on. These are the three new examples in the examples/events directory:

• load-events.py demonstrates a sort of high level interface to load events and how epochs could be contained in datasets. Let me know what you think of this approach - it's an alternative to having events inside of Epochs object that might actually be more flexible (at the same time as keeping the Epochs object simple :) ). Say, you want to look at stc in comination with sensor data: both objects can be in the same event-dataset and you can perform common indexing operations.
• loader.py illustrates a function that can be made for an experimental paradigm to automatically label events
• align.py shows what you actually asked for -- how to load epochs for two different events and align the trials -- and uses loader.py

[dengemann]

Great, but as @mluessi suggested let' wait for feedback from @agramfort before further proceeding.

On 09.12.2012, at 01:38, Christian Brodbeck [email protected] wrote:

I added 3 basic examples of how I have been using events to a dev-events branch. At various stages check the repr of the objects, they should be useful. Information about the classes is also here. Let me know of anything that is unclear or that you would like more information on. These are the three new examples in the examples/events directory:

load-events.py demonstrates a sort of high level interface to load events and how epochs could be contained in datasets. Let me know what you think of this approach - it's an alternative to having events inside of Epochs object that might actually be more flexible (at the same time as keeping the Epochs object simple :) ). Say, you want to look at stc in comination with sensor data: both objects can be in the same event-dataset and you can perform common indexing operations.
loader.py illustrates a function that can be made for an experimental paradigm to automatically label events
align.py shows what you actually asked for -- how to load epochs for two different events and align the trials -- and uses loader.py
—
Reply to this email directly or view it on GitHub.

[christianbrodbeck]

Definitely -- but if you need a solution for your baseline correction you can check out what my function does.

By the way, what do people think on the original 'tmin' and 'tmax' proposal? :)

[dengemann]

On 09.12.2012, at 01:57, Christian Brodbeck [email protected] wrote:

Definitely -- but if you need a solution for your baseline correction you can check out what my function does.

Sure, i appreciate this. will look into it tomorrow.

By the way, what do people think on the original 'tmin' and 'tmax' proposal? :)

my gut feeling would be to not add it to reject just because it is a dict. then you would end up with 2 times tmin / max. anyways, i would like to keep the point for after 0.5 and see where and how to get it.
—
Reply to this email directly or view it on GitHub.

[agramfort]

we can make a branch on mne-tools/mne-python but usually for such private work I give write access to my repo so we can be 2 pushing on my branch.

I am also not convinced that tmin and tmax to reject is the cleanest way to achieve what we want.

[dengemann]

What I would then suggest is to have a private branch for doing experiments
and exchanging and a more serious 0.6 / smart_epochs branch we treat with
same commitment and respect as we do treat our master, that means, proper
reviewed PRs with the distant goal of being merged with the master. makes
sense?

[agramfort]

What I would then suggest is to have a private branch for doing experiments
and exchanging and a more serious 0.6 / smart_epochs branch we treat with
same commitment and respect as we do treat our master, that means, proper
reviewed PRs with the distant goal of being merged with the master. makes
sense?

fine with me. Just do it with car as a git push upstream master is
quickly done :)

[dengemann]

The more I think about it .. -- if the goal is to merge it on the long run
we can just go private until it is ready for PRs, maybe we don't need an
extra branch: if we keep up our schedule we will be developing 0.6 in about
7 days..

On Sun, Dec 9, 2012 at 3:33 PM, Alexandre Gramfort <[email protected]

wrote:

What I would then suggest is to have a private branch for doing
experiments
and exchanging and a more serious 0.6 / smart_epochs branch we treat with
same commitment and respect as we do treat our master, that means, proper
reviewed PRs with the distant goal of being merged with the master. makes
sense?

fine with me. Just do it with car as a git push upstream master is
quickly done :)

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/240#issuecomment-11171495.

[dengemann]

What about his guy? I'll prepend 0.6 ....

[agramfort]

keep the discussion for 0.6 yes.

[larsoner]

Cool. Set the milestone as 0.6.

[christianbrodbeck]

To come back to the original issue:

I have a use case where my baseline is 700 to 600 ms before my actual epoch starts. I would still like to load my actual epoch starting at -100 ms for visualization, but I would not want to reject trials because of something that occurs in that unused baseline time period. Since the reject Epochs kwarg is already a dict, how would you feel about adding an optional 'tmin' and 'tmax' entry to that dictionary?

As @agramfort said

I am also not convinced that tmin and tmax to reject is the cleanest way to achieve what we want.

Would you prefer to add kwargs to Epochs? reject_tmin and reject_tmax? or reject_window=(tmin, tmax)? Or is there yet a better option?

[dengemann]

On 05.01.2013, at 15:52, Christian Brodbeck [email protected] wrote:

To come back to the original issue:

I have a use case where my baseline is 700 to 600 ms before my actual epoch starts. I would still like to load my actual epoch starting at -100 ms for visualization, but I would not want to reject trials because of something that occurs in that unused baseline time period. Since the reject Epochs kwarg is already a dict, how would you feel about adding an optional 'tmin' and 'tmax' entry to that dictionary?

As @agramfort said

I am also not convinced that tmin and tmax to reject is the cleanest way to achieve what we want.

Would you prefer to add kwargs to Epochs? reject_tmin and reject_tmax? or reject_window=(tmin, tmax)?

for consistency, i would do it along the lines of tmin. so reject_min / max
Or is there yet a better option?

—
Reply to this email directly or view it on GitHub.

[christianbrodbeck]

for consistency, i would do it along the lines of tmin. so reject_min / max

I was thinking a tuple like reject_window=(tmin, tmax) would be consistent with the baseline parameter

[dengemann]

that's also nice, i find it quite intuitive.
actually i like both new proposals.

On 05.01.2013, at 16:09, Christian Brodbeck [email protected] wrote:

for consistency, i would do it along the lines of tmin. so reject_min / max

I was thinking a tuple like reject_window=(tmin, tmax) would be consistent with the baseline parameter

—
Reply to this email directly or view it on GitHub.

[agramfort]

I like reject_tmin / tmax better

[agramfort]

Both are fine with me

[agramfort]

Thinking more. Wouldn't it be simpler to allow the baseline to be out of tmin tmax and reject based on the union of baseline and tmin/tmax ?

[christianbrodbeck]

Thinking more. Wouldn't it be simpler to allow the baseline to be out of tmin tmax and reject based on the union of baseline and tmin/tmax ?

That would not allow for what I originally wanted to do:

• a baseline at (-700, -600)
• I would want an Epochs with data (-100, 600), just to see the baseline
• but I would not want to lose epochs due to the (-100, 0) interval, so I would want to reject based on (0, 600)

As for specifying a baseline outside of the epoch, I think it would complicate the Epochs object considerably, and still not provide a general solution (say, the temporal distance between the baseline event the event for Epochs is variable). IMO this could be handled through events and a data object that allows baseline correction (with #309 it could be as simple as epochs -= bl_epochs.summary('time')).

[dengemann]

very elegant. sounds like a must-have!

On 05.01.2013, at 16:53, Christian Brodbeck [email protected] wrote:

Thinking more. Wouldn't it be simpler to allow the baseline to be out of tmin tmax and reject based on the union of baseline and tmin/tmax ?

That would not allow for what I originally wanted to do:

a baseline at (-700, -600)
I would want an Epochs with data (-100, 600), just to see the baseline
but I would not want to lose epochs due to the (-100, 0) interval, so I would want to reject based on (0, 600)
As for specifying a baseline outside of the epoch, I think it would complicate the Epochs object considerably, and still not provide a general solution (say, the temporal distance between the baseline event the event for Epochs is variable). IMO this could be handled through events and a data object that allows baseline correction (with #309 it could be as simple as epochs -= bl_epochs.summary('time')).

—
Reply to this email directly or view it on GitHub.

[larsoner]

A functionally equivalent solution would be to have the user load tmin=-700
to tmax=600 with bmin=-700 to bmax=-600 and reject_tmin/max=0-600, then
provide a crop(-100, 600) method, if we don't already. We have that method
for other instances, and it should be easy. Thus does have the drawback of
loading -600 to -100 for no reason, but I could live with that. I'll have
to think about how this would work with peels added versus non preloaded
data, though.

This would avoid, I think, complicating the code with a new baseline epochs
object. I'm not a fan of that, since any changes to the main object would
have to now be done twice, once to the main object and again to the
baseline one. Think about changing projectors, modifying events, and
anything else we currently have or will add that can be applied on
non-preloaded data...

[larsoner]

Actually, I just realized this can probably all be done with two simple changes, preloading or not:

• Add reject_tmin and reject_tmax as proposed.
• Change the function that acually loads each epoch individually. All that needs to be done here is:
  1. load (min(tmin, bmin), tmax) instead of (tmin, tmax) (and for generality probably (min(tmin, bmin), max(tmax,bmax)).
  2. Change the rejection range appropriately.
  3. After baseline, crop epoch back down before returning it.

Does that make sense and cover your use case @christianmbrodbeck? The only drawback here is temporarily loading, in your example, (-500, -100) one epoch at a time, which will just mean it will be a little slower (not meaningfully any higher memory requirements). I prefer this tradeoff to the additional complications of sub-instancing the baseline epochs.

[christianbrodbeck]

@Eric89GXL

• Add reject_tmin and reject_tmax as proposed.

Already covers everything I need, I will send a PR with that. If you want I can look into including the other proposal, but personally I prefer to handle the baseline correction using a separate epochs object since it's more flexible and does not load more data than necessary.

[larsoner]

@christianmbrodbeck, if efficiency is the concern, it would also be possible (and actually not that difficult) to make the loading function I talked about smart enough to only load the necessary portions (instead of the wasted -500 to -100). It would just require dealing with three use cases: 1) no overlap, 2) partial overlap, and 3) full overlap (current behavior). This shouldn't be too difficult to code.

I think that would be just as flexible and in the end less complex than a second object, as it would avoid the issues I mentioned having to do with needing to change things in two places for various functions by having two epochs objects. Consider the use case where a user doesn't preload data, and then modifies epochs.info in some way, say to change the projectors. The baseline epochs object wouldn't know about it (!), or we'd have to re-copy the base Epochs instance's info object back into the baseline Epochs instance whenever we wanted to do something. Also, any other functions we have or may add we will need to remember to call the same function on two different instances, which is generally not a safe programming practice, if it can otherwise be avoided.

I might be missing something about having the second (baseline) epochs object, though -- in what way(s) would it be more flexible? If it's so that baselines and epochs can be manipulated independently (by perhaps having different projectors or something), then I think it makes more sense to have the user themselves do this by actually having two separate epochs objects...

[larsoner]

Nevermind that last comment about actually having two separate epochs objects -- realized that won't work for rejection purposes.

[christianbrodbeck]

@Eric89GXL
What I had in mind concerning flexibility was mainly being able to actually use a different event rather than a rigid time interval.

As for manipulating info, wouldn't that be better done on the Raw? I prefer having two objects (and have already implemented that in my pipeline apart from the rejection time window issue) but if that seems to be a general concern I can look into adding a disjoint baseline to the Epochs object.

[larsoner]

Yeah, it is probably better to do it in Raw, but there could be cases where users might want to do it (testing or whatever). You never know what they'll try :)

I hadn't considered the possibility of another event. I see how that could be useful, too. However, it seems like that would add another layer of complication if we tried to incorporate it directly. I do think adding an arbitrary, user-supplied disjoint baseline (as basically a generator for or array of scalars for each epoch) would be the most flexible, and the least "disruptive" to the current Epochs instances. I think that's what you're talking about -- and I think that would that cover your use case?

[dengemann]

On 05.01.2013, at 18:52, Eric89GXL [email protected] wrote:

Yeah, it is probably better to do it in Raw, but there could be cases where users might want to do it (testing or whatever). You never know what they'll try :)

I hadn't considered the possibility of another event. I see how that could be useful, too. However, it seems like that would add another layer of complication if we tried to incorporate it directly. I do think adding an arbitrary, user-supplied disjoint baseline (as basically a generator for or array of scalars for each epoch) would be the most flexible, and the least "disruptive" to the current Epochs instances. I think that's what you're talking about -- and I think that would that cover your use case?

i woul like that. for a current mixed block and event related design i applied eric's scheme and created an epoch of pre-block baselines to be applied to each of the events in the block. some generator solution / mapping tool for customized match between trials and baselines would be neat though

—
Reply to this email directly or view it on GitHub.

[larsoner]

Alright people, how do you think on this:

1. Add reject_tmin and reject_tmax as proposed, with defaults to None, meaning use all time points, as currently done.
2. Change the function that acually loads each epoch individually. Deal with cases where (bmin, bmax) 1) don't overlap, 2) partially overlap, and 3) fully overlap (current behavior). This way no extra data is loaded than necessary. You will also presumably need to incorporate reject_tmin and reject_tmax here since I think that's where the rejection is done.
3. Add new behavior for baseline argument. Use current behavior if it's an instance of tuple or None. Add new behavior otherwise, assuming it's something that can be accessed as baseline[0], baseline[1], ..., be it a generator or array or whatever.

I like (2) because I think it's something that should be there anyway -- a user might assume that (bmin, bmax) doesn't need to be in (tmin, tmax), and I don't see a reason to require it if we don't need to. I like (3) because it extends the functionality to cover what @dengemann did, and what you want to do, among other things, without complicating the Epochs instance itself.

[dengemann]

sounds like a good plan. 3) might then be implemented independently, so we can hack into it to see, how to besu get there. baseline might also be a dict in which events are mapped to their according (custom) baseline segments

On 05.01.2013, at 20:44, Eric89GXL [email protected] wrote:

Alright people, how do you think on this:

Add reject_tmin and reject_tmax as proposed, with defaults to None, meaning use all time points, as currently done.
Change the function that acually loads each epoch individually. Deal with cases where (bmin, bmax) 1) don't overlap, 2) partially overlap, and 3) fully overlap (current behavior). This way no extra data is loaded than necessary. You will also presumably need to incorporate reject_tmin and reject_tmax here since I think that's where the rejection is done.
Add new behavior for baseline argument. Use current behavior if it's an instance of tuple or None. Add new behavior otherwise, assuming it's something that can be accessed as baseline[0], baseline[1], ..., be it a generator or array or whatever.
I like (2) because I think it's something that should be there anyway -- a user might assume that (bmin, bmax) doesn't need to be in (tmin, tmax), and I don't see a reason to require it if we don't need to. I like (3) because it extends the functionality to cover what @dengemann did, and what you want to do, among other things, without complicating the Epochs instance itself.

—
Reply to this email directly or view it on GitHub.

[christianbrodbeck]

Actually what I did not consider was that I baseline correct in source space. This is no issue with two separate epochs objects, but I don't see a simple way to do that Epochs-internally(?).

[larsoner]

So maybe just doing (1) and (2) above for now (and doing (3) later, maybe when @dengemann feels like doing it for block designs) makes more sense? Is that what your new PR covers?

[christianbrodbeck]

The PR only covers 1. 2 would make baseline correction for sensor space and source space different (whereas using 2 objects the two cases could be handled in a consistent way). Is the simpler sensor space bl correction worth making it inconsistent with source space bl correction?

[larsoner]

I think that most users would expect the functionality suggested in (2), namely, being able to have your baseline be outside tmin/tmax. I know I did when I starting working with the data, even though I haven't hit a use case for it yet.

That being said, we can always add that later, if/when someone actually requests it. I think baseline correcting in source space is a bit of a different beast, and it probably makes the most sense (like you say) to just let the user have two epochs objects, take them to source space, then do the correction themselves.

[agramfort]

I am fine with the PR sent (see my tiny comments)

Regarding baseline correction is source space be careful as any inverse
solver assumes a zero mean baseline and re-baseline correcting can seem
weird.

[larsoner]

@christianmbrodbeck okay to close this one?

[christianbrodbeck]

@Eric89GXL
Unless we want to keep it open for the baseline correction. I could look into it (I agree that it would make logical sense). This would make sense especially if baseline correction in source space is not normally done.

@agramfort
I've been baseline correcting in source space because for free orientation solutions usually the whole brain is active at any point. When looking at significant deviation from zero for a visual response this seems to look reasonable. So when comparing two conditions is it preferable to not baseline correct in source space?

[dengemann]

Were we done here? Can I close this?

[agramfort]

@christianmbrodbeck can we close? or are you still willing to improve things here?

[christianbrodbeck]

I don't have this issue at the moment so unless it gets asked for I have other things to work on