Comments (7)
Hi @juliusandretti,
thanks for the message. Could you provide me with a minimal example? Let's work out a test so that we can be confident about the implementation of Crespo's algorithm.
Thank you for your help.
Greg
from pyactigraphy.
Here's a minimal example. Hope it helps.
`from datetime import datetime, timedelta
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from pyActigraphy.filters import _create_inactivity_mask
Time index for the arrays
start = datetime.today().replace(second=0)
dt = timedelta(minutes=1)
time = [start + i*dt for i in range(30)]
Estimate of wake/rest
ye = np.ones(30)
ye[10:20] = 0 # 1/3 wake, 1/3 rest, 1/3 wake
ye = pd.Series(ye, index=time)
Activity counts
act = np.ones(30)
act[[8,9,12,13,14,15,16,20,21]] = 0
act = pd.Series(act, index=time)
Applying the filter
mask_rest = _create_inactivity_mask(act[ye < 1], 11, 1)
mask_actv = _create_inactivity_mask(act[ye > 0], 4, 1)
mask_ar = pd.concat([mask_actv, mask_rest], verify_integrity=True)
There are no invalid zeroes
expected = pd.Series(np.ones(30), index=time)
Plots
plt.plot(1.2ye,label='1.2ye')
plt.plot(1.1expected,label='1.1expected mask')
plt.plot(mask_ar.sort_index(),label='mask')
plt.legend()
plt.show()`
from pyactigraphy.
I know this is an old issue, but I believe that this works as expected.
The code provided as example splits the active and rest phases to apply different minimal durations for the creation of the inactivity mask.
While this sounds like a resonable desire, the approach is flawed.
To prove that, one has to inspect the series returned by this selection call:
# full line of code: mask_actv = _create_inactivity_mask(act[ye > 0], 4, 1)
act[ye > 0]
Based on the generated input, this yields the following result:
2023-08-03 10:05:00.212904 1.0
2023-08-03 10:06:00.212904 1.0
2023-08-03 10:07:00.212904 1.0
2023-08-03 10:08:00.212904 1.0
2023-08-03 10:09:00.212904 1.0
2023-08-03 10:10:00.212904 1.0
2023-08-03 10:11:00.212904 1.0
2023-08-03 10:12:00.212904 1.0
2023-08-03 10:13:00.212904 0.0
2023-08-03 10:14:00.212904 0.0 # here we have a time gap afterwards because of activity and rest phases!!!!!
2023-08-03 10:25:00.212904 0.0
2023-08-03 10:26:00.212904 0.0
2023-08-03 10:27:00.212904 1.0
2023-08-03 10:28:00.212904 1.0
2023-08-03 10:29:00.212904 1.0
2023-08-03 10:30:00.212904 1.0
2023-08-03 10:31:00.212904 1.0
2023-08-03 10:32:00.212904 1.0
2023-08-03 10:33:00.212904 1.0
2023-08-03 10:34:00.212904 1.0
Now, when passing this to create_inactivity_mask()
, respectively _create_inactivity_mask()
we run into a problem.
The index is not checked!
Have a look at the following line from _create_inactivity_mask()
https://github.com/ghammad/pyActigraphy/blob/master/pyActigraphy/filters/filters.py#L13:
mask = np.ones_like(data)
This shows that the function is ignorant of what kind of time series is passed in, and gaps are consequently not handled.
So, we get the following output:
>>> _create_inactivity_mask(act[ye > 0], 4, 1)
2023-08-03 10:05:00.212904 1.0
2023-08-03 10:06:00.212904 1.0
2023-08-03 10:07:00.212904 1.0
2023-08-03 10:08:00.212904 1.0
2023-08-03 10:09:00.212904 1.0
2023-08-03 10:10:00.212904 1.0
2023-08-03 10:11:00.212904 1.0
2023-08-03 10:12:00.212904 1.0
2023-08-03 10:13:00.212904 0.0
2023-08-03 10:14:00.212904 0.0
2023-08-03 10:25:00.212904 0.0
2023-08-03 10:26:00.212904 0.0
2023-08-03 10:27:00.212904 1.0
2023-08-03 10:28:00.212904 1.0
2023-08-03 10:29:00.212904 1.0
2023-08-03 10:30:00.212904 1.0
2023-08-03 10:31:00.212904 1.0
2023-08-03 10:32:00.212904 1.0
2023-08-03 10:33:00.212904 1.0
2023-08-03 10:34:00.212904 1.0
After all, I still believe the function works as intended.
However, to prevent confusion in the future, one could implement a simple check in the function to verify that the data
is continous.
I'm not familiar with the whole code base and if there exists a general support for non-continuing time series in PyActigraphy.
Thus, I gladly leave the decision for the next actions gladly to somebody else. I just hope, the analysis is sparing others some time.
from pyactigraphy.
Hello @ugGit
Indeed, the mask creation function does not accept (and more generally, there is no support in pyActigraphy for) non-continuous time series. That's why masked periods are substituted with NaNs instead of simply chopping the chunk off. I considered that being easier for users and easier to handle from the code point of view.
That being said, adding a check in the mask creation function would be good. Would you be up for a PR?
Thank you.
Greg
from pyactigraphy.
To be honest. I'm not convinced anymore if it is reasonable catching this during the mask creation.
I rather suggest to make the check for continuous data at the highest level possible, which is the instantiation of the BaseRaw
object.
This would also prevent any other functions from producing outputs based on non-continuous input data.
What do you think?
As a check, I suggest during the init of BaseRaw
something like:
if not data.equals(data.resample(frequency)):
raise ValueError('smart message')
And yes, I'll make a PR once you agree on the implementation :)
from pyactigraphy.
@ugGit Well, I haven't tested the check you propose but it could be implemented, provided it is efficient (in terms of CPU timing). Nonetheless, the original proposal still looks good to me because it is a requirement of the function itself and this function is used in several parts of the package and might be re-used again in future developments.
To summarize, both modifications do not have to be exclusive.
WDYT?
from pyactigraphy.
Good point regarding the exclusivity!
I will already prepare a PR for the check in _create_inactivity_mask()
.
Regarding the second option, where the continuity is checked during the initialization of the BaseRaw
object. Here it should only be implemented if all analyses are supposed to work on continuous data.
If you think, that this is the case, then I would implement the check there as well.
Regarding the implementation detail, this should be more performant:
expected_nbr_of_epochs = (data.index[-1] - data.index[0]) / freq
present_nbr_of_epochs = len(data)
if present_nbr_of_epochs > 1 AND present_nbr_of_epochs != expected_nbr_of_epochs:
raise ValueError('smart message')
from pyactigraphy.
Related Issues (20)
- .mtn error HOT 3
- Visualization of Actigraphy data HOT 1
- Add support for different model of Actiwatch-L HOT 1
- Read actilife data HOT 2
- Sleep diary file format bug HOT 5
- Define Threshold as Parameter in create_inactivity_mask() HOT 1
- How Can You Read Actiwatch Spectrum Plus CSV Files? HOT 5
- Support different file encodings for RPX files
- Add support for RPX files in Italian language
- The package installation does not work in anaconda/spyder HOT 5
- Add basic sleep parameters HOT 1
- For Empatica E4 HOT 4
- Error - UnboundLocalError: local variable 'uuid' referenced before assignment HOT 2
- Pyactigrpahy native actigrpahy data CWA file HOT 1
- Package not supported for Python 3.11 or newer HOT 1
- .IS and _interdaily_stability possibly giving different results with resampling
- ActivityReport function
- Packages version conflict
- Mismatch between documentation and code for the computation of non parametric variables IS and IV
- New format for ActTrust Watches not compatible w/ import
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pyactigraphy.