Comments (15)
Duplicate of #937
@giova86, thanks for your workaround.
@bfdykstra, @seanlaw, @vanbenschoten: The matrixprofile project certainly is interesting and I had approved the original pull request. On the other hand, I have not seen any of the matrix profiles features as statistically significant features in one of my projects. Therefore, I am going to remove the dependency on matrix profiles.
from tsfresh.
Fixed in https://github.com/blue-yonder/tsfresh/tree/v0.20.0.
Many thanks for your patience and perseverance.
Merry Christmas and a Happy New Year!
from tsfresh.
I'm using an M1 chip and had the same issues when trying to install it under python 3.9.
However, with Python 3.9.7 pip install worked.
You might need to
brew install llvm
from tsfresh.
@bfdykstra I don't know anything regarding the matrixprofile
API but if you'd like to submit a PR to this repo with the stumpy
replacement parts then I can help review it.
Alternatively, would it make sense to simply remove the matrixprofile
parts instead of replacing it given the lack of activity in that package? My concern is that we may be forcing stumpy
to support matrixprofile
features that are absent in stumpy
and so we should be open to removing the matrixprofile
bits if necessary.
from tsfresh.
My preferred solution is to refactor the matrix profile features into a separate module, e.g. tsfresh-matrixprofiles, which can be installed on top of tsfresh.
@kempa-liehr that's probably a good idea. Having said that, it appears that users are unable to install tsfresh
right now and the solution provided above addresses this issue while preserving some level of backwards compatibility. If the goal is to split it out then I think we should simply spend the time to remove the matrixprofile
package (which doesn't seem to be supported anymore) parts altogether for this PR, which would also resolve the installation issues. My suggestion would be to spend this time fixing the current issue, which is an installation problem. What do you think?
from tsfresh.
I tried to use Rosetta and it seems to work. I followed these steps:
- open terminal with Rosetta
- create and start venv (I'm using Python 3.9.13)
- install tsfresh (pip install tsfresh)
- since there is a bug with statsmodels 0.13.0 I have downgraded it to 0.12.1
- use if name == "main"
I checked with this very simple code:
from tsfresh import extract_relevant_features
from tsfresh.examples.robot_execution_failures import load_robot_execution_failures
timeseries, y = load_robot_execution_failures()
if __name__ == "__main__":
features = extract_relevant_features(timeseries, y, column_id="id", column_sort="time")
print(features)
from tsfresh.
I'm no expert with all this stuff. Just saw a youtube thing on matrix profiles. It looks like it's using /usr/bin/clang instead of the brew-installed /usr/local/bin/gcc which has the -fopenmp flag working. Is there any way to force it to use gcc?
from tsfresh.
I'm running into the same issue. brew install llvm
did not help on Python 3.10.6
from tsfresh.
I have the same problem...brew install llvm
did not help. I'm using Python 3.9.13
from tsfresh.
I used what @JoachimSchaeffer did and set my python version to exactly 3.9.7 and pip install worked.
I started taking a crack at doing what the matrixprofile author said and replacing matrix profile with stumpy
. However stumpy does not provide an implementation of maximum_subsequence
, but we could probably roll our own implementation of that based on what matrix profile already has.
I'm also not entirely sure that there is a 1-1 mapping between the APIs of the two packages:
https://matrixprofile.docs.matrixprofile.org/api.html#matrixprofile-compute
https://stumpy.readthedocs.io/en/latest/api.html#stumpy.stump
from tsfresh.
@seanlaw Thank you weighing in!
Not sure how critical the matrix profile stats are as features, maybe @nils-braun could provide some insight as to what would be lost if we removed them?
From the documentation of tsfresh it says:
Calculates the 1-D Matrix Profile[1] and returns Tukey’s Five Number Set plus the mean of that Matrix Profile.
Matrix profile's api documentation says it computes the exact or approximate matrix profile based on sample percent.
I see with stumpy
that we can use either stumpy.stump
or stumpy.scrump
for an exact or approximate matrix profile.
@seanlaw Does stumpy support calculations based off of a sample percent?
Looking at the tests, might shed some light on the desired usage:
def test_matrix_profile_window(self):
# Test matrix profile output with specified window
np.random.seed(9999)
ts = np.random.uniform(size=2 ** 10)
w = 2 ** 5
subq = ts[0:w]
ts[0:w] = subq
ts[w + 100 : w + 100 + w] = subq
param = [
{"threshold": 0.98, "windows": 36, "feature": "min"},
{"threshold": 0.98, "windows": 36, "feature": "max"},
{"threshold": 0.98, "windows": 36, "feature": "mean"},
{"threshold": 0.98, "windows": 36, "feature": "median"},
{"threshold": 0.98, "windows": 36, "feature": "25"},
{"threshold": 0.98, "windows": 36, "feature": "75"},
]
self.assertAlmostEqual(matrix_profile(ts, param=param)[0][1], 2.825786727580335)
from tsfresh.
Not sure how critical the matrix profile stats are as features, maybe @nils-braun could provide some insight as to what would be lost if we removed them?
Objectively, I don't think these stats features are useful at all (save for the min
of the matrix profile) as they aren't necessarily informative as an ML feature. Hence, why I suggested removing this part completely.
@seanlaw Does stumpy support calculations based off of a sample percent?
I am not too familiar with what is happening in matrixprofile
but I believe that you can use stumpy.scrump
and specify percentage=1.0
to obtain the exact matrix profile (see API docs).
Perhaps something like this would be equivalent for allowing the test to pass:
def test_matrix_profile_window(self):
# Test matrix profile output with specified window
np.random.seed(9999)
ts = np.random.uniform(size=2 ** 10)
w = 2 ** 5
subq = ts[0:w]
ts[0:w] = subq
ts[w + 100 : w + 100 + w] = subq
# mp = stumpy.stump(ts, m=36)[:, 0]
mp = stumpy.scrump(ts, m=36, percentage=1.0)[:, 0]
self.assertAlmostEqual(mp.min(), 2.825786727580335)
Then, to obtain the other numbers, you'd simply do:
mp.max()
mp.mean()
np.median(mp)
np.percentile(mp, 25)
np.percentile(mp, 75)
I think this is the easiest way forward without attempting to force a one-to-one match between the packages. We'll need to remove the threshold
parameters or simply ignore them when passed in. Please feel free to ask more questions if I've been to vague. Does that help?
from tsfresh.
My preferred solution is to refactor the matrix profile features into a separate module, e.g. tsfresh-matrixprofiles, which can be installed on top of tsfresh.
from tsfresh.
I'm using an M1 chip and had the same issues when trying to install it under python 3.9. However, with Python 3.9.7 pip install worked. You might need to
brew install llvm
Unfortunately this didn't work for me, but it sounds like the matrixprofile dependency is going to be removed so I will wait.
from tsfresh.
The solution I've found was to downgrade the Python version to 3.10.8 using pyenv in order to easily manage multiple Python versions (useful resource: https://opensource.com/article/19/5/python-3-default-mac).
from tsfresh.
Related Issues (20)
- shuffle_transfer failed during shuffle c1992000bcde9deee65a2332ab4614a2 HOT 3
- Can Not Rolling With Binary Feature HOT 2
- tsfresh 0.20.0: TypeError exception with Pandas 2.0.0 HOT 3
- support of python 3.10 HOT 2
- Debugging tsfresh methods from Notebook freezes whole process HOT 4
- Example Provided in doc doesn't work HOT 1
- Quick start example fails due to TypeError in Setting MultiIndex HOT 1
- During the feature extraction process, how could I set the param from a subset of another columns. HOT 1
- TypeError: acf() got an unexpected keyword argument 'unbiased' HOT 11
- 05 Timeseries Forecasting (multiple ids).ipynb doesn't work as written HOT 2
- Problem with the function extract_features HOT 7
- Feature calculator return type documentation HOT 2
- extract_relevant_features hangs on specific data HOT 1
- number_of_maxima only returns maxima of last key in the dictionary. HOT 2
- Feature selection taking magnitutes longer than it should HOT 8
- Look-Ahead Bias in Generated Features HOT 4
- id column in the result of make_forecasting_frame have only (id, ?) as identifier HOT 4
- roll time series only give "step by step window" for max_timeshift window, but have left alone windows from 1 to max_timeshift window? HOT 1
- Cannot run dask_feature_extraction_on_chunk
- AttributeError: `np.PINF` was removed in the NumPy 2.0 release. Use `np.inf` instead. HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tsfresh.