Comments (16)
Good point. If you extract features for different kind of time series this should be easily implemented.
from tsfresh.
Can it at least be a log friendly progress indicator and not a some spinnings forward slashes, pipes and backslashes or some incrementally filling up progress bar which are visually pleasing but can be hell to logs.
Although we must think about the humans sitting there wondering how long it will take and if they have time to read a comic, we must also think of the machines that are conducting their work and calculating programmatically as well. A progress indicator may not be of interest to them as they are not wondering about comics, but just trying to do the job assigned to them and calculate some features.
Hopefully it is possible to find a happy middle ground here that provides appropriate info to all executing parties, whether they be people or machines.
from tsfresh.
Just try clarify, this point.
Universe Sandbox2 just presented me with an example of a progress indicator implementation that could be classified and classed as person_friendly and also not_great_for_a_machine_or_log. It seemed relevant to share and I ironically choose Universe Sandbox2 over having to further contemplate the 50 / 3 Python paradox/dilemma (with which I have recently been confronted with) after further discussing and explaining it in detail to with my wife and dog on our evening walk. She got it and agrees, our dog Dude has not weighed in his opinion yet, but I am not waiting on it.
an example of a not so machine friendly progress implementation
by Steam (although it is not meant to be consumed programmatically), this is in no way a criticism of Steam's progress indicator, I find it quite informative and visually detailed and I like that, but I am person.
from tsfresh.
@earthgecko : Yeah, I've had my logs wrecked by some of those terrible progress indicators you're talking about. They also seem like unnecessary complexity at this stage. I'm just hoping for something simple, that's also reassuring for first-time users as they sit wondering whether things are working on their dataset or not :)
from tsfresh.
I could take care of this, but I would wait until the parallelization is implemented
from tsfresh.
@nils-braun what about: https://pypi.python.org/pypi/tqdm
I am tinkering with it around right now.
from tsfresh.
@nils-braun or do you already have a solution in mind?
from tsfresh.
The question it comes down is: what should be the "minimal step" in a progress? Should each calculator function be one step in the progress? Or each kind/simple (depending on the parallelization)?
from tsfresh.
I think a progress for the chunks is sufficient.
With chunks I refer to the squares (sample x kind) out of the original data frame.
from tsfresh.
so based on the parallelisation - ok. This would however mean that if you have one single time series with one kind, you only have one step in the progress bar (just to make this clear).
from tsfresh.
lets say
df =
id val1 val2
1 4 4
2 3 4
3 4 1
4 1 4
and you extract the features in parallel for those 4 "chunks"
id val1
1 4
2 3
id val2
1 4
2 4
id val1
3 4
4 1
id val2
3 1
4 4
after the completion of any one of those "chunks" the progress bar should increase by 1/4.
from tsfresh.
... actually we do not have any parallelization mode for this, aren't we? We do have parallelisation per kind (which would be 2 in this example) or per sample (with would be 4 in this case, with the ids 1, 2, 3, 4 for both kinds). Or did I miss something?
from tsfresh.
Ah sorry, I got it. Do not bother with my last comment...
from tsfresh.
Actually, the "per_sample" parallelization is parallelizing per kind and per sample
from tsfresh.
The progress bar for future extraction is now includes. I will include another one for feature selection soon.
from tsfresh.
@moritzgelb implemented this in #152
from tsfresh.
Related Issues (20)
- shuffle_transfer failed during shuffle c1992000bcde9deee65a2332ab4614a2 HOT 3
- Can Not Rolling With Binary Feature HOT 2
- tsfresh 0.20.0: TypeError exception with Pandas 2.0.0 HOT 3
- support of python 3.10 HOT 2
- Debugging tsfresh methods from Notebook freezes whole process HOT 4
- Example Provided in doc doesn't work HOT 1
- Quick start example fails due to TypeError in Setting MultiIndex HOT 1
- During the feature extraction process, how could I set the param from a subset of another columns. HOT 1
- TypeError: acf() got an unexpected keyword argument 'unbiased' HOT 11
- 05 Timeseries Forecasting (multiple ids).ipynb doesn't work as written HOT 2
- Problem with the function extract_features HOT 7
- Feature calculator return type documentation HOT 2
- extract_relevant_features hangs on specific data HOT 1
- number_of_maxima only returns maxima of last key in the dictionary. HOT 2
- Feature selection taking magnitutes longer than it should HOT 8
- Look-Ahead Bias in Generated Features HOT 4
- id column in the result of make_forecasting_frame have only (id, ?) as identifier HOT 4
- roll time series only give "step by step window" for max_timeshift window, but have left alone windows from 1 to max_timeshift window? HOT 1
- Cannot run dask_feature_extraction_on_chunk
- AttributeError: `np.PINF` was removed in the NumPy 2.0 release. Use `np.inf` instead. HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tsfresh.