Comments (10)
I think, overall, we have to figure out clearly what FOOOF is supposed to be doing, and from there, what is a bug, what is simply outside the scope of (current) FOOOF. There has to be decision points on what we solve, when. Currently, FOOOF assumes, and will only work, given data with a single background slope.
@srcole: the LFP data (plotted in log-log) seems to violate this assumption, and so FOOOF fails. It does seem to work well applied on smaller ranges.
Although not by design, practically, it appears the scope of current FOOOF simply doesn't cover broad freq ranges (such as 1-100), as a consequence of them basically always violating the single slope assumption. Given this multiple slope issue, it seems likely, current FOOOF will simply never, in a single fit, be appropriate for a wide range, such as 1-100 Hz, potentially limiting some of it's use with ECoG & LFP (although it can be applied piece-wise). This is a pretty annoying limitation, and limits current FOOOF more than we might have wanted. Whatever we do, as in the original point of this issue, we need to be super clear about this limitation for current FOOOF (in so far as we keep this version)
With this all in mind, @voytek suggestions:
0) Yes, if we want to generalize to broader frequency ranges - effectively broadening the scope of FOOOF.
- I vote no. Maybe the user could preset which fit to use, given their data, but allowing the algorithm to choose heuristically would make meaningful comparisons across PSDs / between fits impossible.
- Maybe. This is also broadening the scope of FOOOF, given what's noted above that broad range seems likely to never meet the assumptions.
Ultimately, there are many expansions and possible updates to FOOOF. Our current question is: what is v0.1 for, and given that, what should it do? I understand v0.1 to be dealing with the single slope case - that may or may not be good enough. Deciding if it is or not, is the first required decision point.
- If we stick to the current scope (assume single background slope), I think it's working.
- If v0.1 has to deal with multiple slopes, all bets are off and it's back to the drawing board with the algorithm, at the very least for slope fitting, potentially for more than that.
A note on exponential fits:
Given that the issue here seems to be slope changes, wouldn't fitting a single exponential have all the same pitfalls? It's not clear to me that offers a fix, but it's potentially worth trying. However, if the real problem is changes in slope, then we would then need to figure out how to algorithmically find knees.
from fooof.
We have a few options.
- Find a better generalized background fit than the current quadratic.
- Fit several different 1/f backgrounds (power, exp, linear, quadratic) and choose the one with smallest RMSE right from the get go.
- Flexibly update fitting models based on input frequency range (maybe for 2-30 Hz, quadratic is best, but for 2-100 exponential is best?)
Then return the fitting model type and equation. Also return the 1/f slope as estimated from a linear fit of the returned background in log-log space (regardless of the resultant best fit equation) across the 30-50 Hz range (per Gao's paper) to get a 1/f slash EI estimate.
from fooof.
@srcole can you attach the data for this one? I haven't been testing on such broad ranges (or any LFP) - it may be that our presets or algorithm are inappropriate.
from fooof.
Another example: standard CA1 LFP with hippocampal theta, which looks like a classic PSD with a flat slope and some bumps. However, the background fit is too low, so there are extra bumps fitted.
@rdgao said he would implement an exponential fit that might work better.
from fooof.
For the LFP data, attached are log-log, full spectrum (terrible fit, can see the slope violates the assumption though). Picking ranges which are approximately single-sloped works.
Param note: min_amp set to 0.2.
from fooof.
from fooof.
But scope creep is where the fun is, Erik! Publishing is awful. (But yes, I agree. I believe we all agree on this point, now.)
from fooof.
@parenthetical-e : I very much agree, and yes, I think we're all on the same page now.
@voytek @srcole @rdgao : Let's consider v0.1 locked in as far as algorithmic changes go (unless anyone wants to try and argue otherwise), and stick to that, short of any newly discovered major issues. I'm going to wrap up the last small things, and add better notes to docs & tutorials and assumptions and limitations, then tag a release.
Once v0.1 is locked in and put out, we can all start fresh on the exciting new world of v0.2 - for better or worse, we haven't entirely finished characterizing PSDs - and we can continue onwards from there.
from fooof.
HAHAHAHAHAH at August 31st "okay we're ready to lock in v0.1".....
from fooof.
This issue / discussion has been addressed by adding knee fitting options. Closing.
from fooof.
Related Issues (20)
- Notes on linear vs log power & units of different parameters / settings HOT 1
- [ENH]: Sliding window: Use previous params as guesses for next fit
- Run modes HOT 4
- Consider checking / changing default values HOT 1
- Developing a system for benchmarking
- bug report: AttributeError: 'FOOOF' object has no attribute 'to_df' HOT 2
- units/scaling for PW and min_peak _height? HOT 3
- Weird fitting when upper bound frequency is higher than 100 Hz. HOT 2
- Bug report: tutorial about Example Power Spectra and Models HOT 2
- Alternative Knee Model HOT 1
- get_params not working due to inhomogeneities in FOOOFGroup HOT 10
- fooof v.1.1 not yet updated on conda forge HOT 7
- Problem on retrieving aperiodic and peak data from Group object with get_data HOT 1
- [Question] ERROR regarding Lower-bound peak width limit is < or ~= the frequency resolution: 1.00 <= 1.00 HOT 2
- specparam - FOOOF confusion HOT 3
- [ENH] to_df to also return the gaussian parameters HOT 6
- Parts of the original spectrum are below the aperiodic fit HOT 1
- Finalizing 2.0 release plan
- Top of the Specpram 2.0.0rc2 documentation page not adjusting properly HOT 6
- Specparam2rc doc buggy example HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fooof.