Dealing with missing values about matrixprofile-ts

[vanbenschoten]

Yeah, in this case you'll probably have to do some custom handling of the gaps. I'd recommend concatenation of the data where possible.

[vanbenschoten]

@Modestas96 the code should be able to handle missing data, so long as the gaps are small relative to the subsequence length. Can you give some more details about the issue?

[Modestas96]

Understood, in my example the gaps are relatively big, so I was wondering if I have to deal with them specifically myself or the algorithm will be able to handle it. So what happens in my case is that algorithm returns matrix profile with only nan or inf values.

[previsualconsent]

I'm curious about how to specify missing data. I used nan's but this broke the results.

import numpy as np
import matplotlib.pyplot as plt
from matrixprofile import *

x = np.linspace(0,100*np.pi,1001)
y = np.sin(x)
mp,ind = matrixProfile.stomp(y,25)
print("No NaNs:", mp[:10], ind[:10])

y[100] = np.nan
mp,ind = matrixProfile.stomp(y,25)
print("One NaN:", mp[:10], ind[:10])```

Output

No NaNs: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] [20. 21. 22. 23. 24. 25. 26. 27. 28. 29.]
One NaN: [nan nan nan nan nan nan nan nan nan nan] [inf inf inf inf inf inf inf inf inf inf]

Is there a different way to handle missing data?

[tylerwmarrs]

@vanbenschoten This issue should not be closed. The current stomp implementation does not correctly handle missing values.

[vanbenschoten]

@tylerwmarrs agreed

[vanbenschoten]

The latest code version (0.0.7) takes care of this issue.