Evaluate the influence of NaN values on event detection about hydrotools

[jarq6c]

I do think missing values ought to be handled in a preprocessing step and the README/docs recommend tips to run event detection, including filling missing data. However, this burden is on the caller. There are too many types of signals and data to account for all the possible ways a given time series could be filled. In some cases, it comes down to personal preference.

I would certainly question the events returned, so a warning may be justified, but I don't think event detection should fill values on the behalf of the user.

[jarq6c]

The filters seem robust again NaN values. The algorithm completes and returns data if NaNs are present. I ran the example below using a 7-day (larger than the gap) and a 30-minute (smaller than the gap) window. The NaNs will definitely influence the deviation from the median signal and effect where the start and end event time land. It's still not recommended to run event detection on time series with missing data, but it won't shut down the process and at least in this case return consistent (if not sensible) "events."

# Import tools to retrieve data and detect events
from hydrotools.nwis_client.iv import IVDataService
from hydrotools.events.event_detection import decomposition as ev
import matplotlib.pyplot as plt

# Retrieve streamflow observations for two sites
service = IVDataService(
    value_time_label="value_time"
    )
observations = service.get(
    sites='01360640', 
    startDT='2021-07-01', 
    endDT='2021-08-01'
    )

# Drop extra columns to be more efficient
observations = observations[[
    'value_time', 
    'value'
    ]]

# Check for duplicate time series, keep first by default
observations = observations.drop_duplicates(
    subset=['value_time']
    )

# Resample to hourly, keep first measurement in each 1-hour bin
observations = observations.set_index("value_time")
observations = observations.resample("15min").nearest(limit=1)

# Detect events
events = ev.list_events(
    observations['value'],
    halflife='6H', 
    window='7D',
    start_radius="24H"
    )

# Print event list    
print(events)

# Plot
observations.plot(logy=True)
observations.loc[events.start, "value"].plot(style="o", ax=plt.gca())
observations.loc[events.end, "value"].plot(style="o", ax=plt.gca())
plt.show()

[aaraney]

I agree with your decision here (not that it matters). You should be able to use the tool improperly. I think there is sufficient documentation and citations to feel comfortable about this issue.