whoigit / ott-utils

IFCB utilities for writing CF compliant files

Utilities for working with IFCB summary data

Installation via Anaconda

It is recommended to use Anaconda to install ott-utils.

conda env create -f environment.yml
source activate ott-utils

Installation via Anaconda for development purposes

ott-utils depends on pyifcb which is currently under development. To enable yourself to update pyifcb and still use it, build the ott-utils-dev environment following these steps:

conda env create -f environment-dev.yml
source activate ott-utils-dev
git clone https://github.com/joefutrelle/pyifcb
cd pyifcb
python setup.py develop

Then, when you want to update pyifcb and use it in ott-utils, change directories to pyifcb and run

git pull

ott-utils's Issues

drop NaNs in binned output?

binned output contains NaNs (for time periods that are empty); should these be dropped?

when representing in NetCDF these become NaNs and would have to be replaced with the appropriate fill value; it seems more useful to all downstream tools to just drop the NaNs and have discontinuous time indices instead.

failing on dataframe column name being an ndarray

Under certain versions of netCDF4, this error occurs indicating that the units parameter is being set to an ndarray instead of a string or dict:

Traceback (most recent call last):
  File "classsummary2netcdf.py", line 40, in <module>
    process_dir(in_dir, out_dir)
  File "classsummary2netcdf.py", line 34, in process_dir
    csdir2netcdf(in_dir, out_dir, frequency=freq, threshold=thresh, metadata=md)
  File "/ifcb_code/ott-utils/ott/netcdf.py", line 77, in csdir2netcdf
    cs2netcdf(cs_path, nc_path, frequency=frequency, threshold=threshold, metadata=metadata)
  File "/ifcb_code/ott-utils/ott/netcdf.py", line 65, in cs2netcdf
    instrument_attributes=i_attrs)
  File "netcdf-cf-utils/nccf/timeseries.py", line 66, in from_dataframe
    self.create_obs_vars(df, ('timeseries','time'), units)
  File "netcdf-cf-utils/nccf/cf.py", line 122, in create_obs_vars
    u = units if type(units) is str else units.get(varname)
TypeError: unhashable type: 'numpy.ndarray'

Software package versions:
numpy v1.12.0
munch v2.1.0
scipy v0.18.1
pandas v0.19.2
netCDF4 v 1.0
yaml v3.10