robfirth / pycoco

Python Tools for CoCo

pycoco

---

v0.9.17

---

---

This is the development repo for the python frontend for the core-collapse SNe template code 'CoCo': https://github.com/UoS-SNe/CoCo (my fork is currently https://github.com/RobFirth/CoCo).

CoCo was originally started by Natasha Karpenka, and is currently being updated and maintained by S. Prajs (https://github.com/SzymonPrajs).

A paper, Firth et. al. 2017, is currently in prep.

---

• Extending spectra with "donor" spectra

• Implemented a raft of changes that make it easier to interact with the spectral fits and the final templates

• fixed a particularly nasty bug (#28) that was screwing up specphot (via filter resampling)

• Updated notebook tarball - (again 17/12/17)

• Now available on PyPi and via pip - package available to install as pycocosn

• Extending templates now possible with Black Body spectrum, flat and linear

• calling CoCo (or reproduced CoCo functions) now more straightforward

• Added more test cases to test_pycoco

• Travis-CI implemented and Master and dev are passing. Fixed bug with ENVIRONS.

• Dependancies now handled in setup.py

• Calling all parts of CoCo from pycoco now operational

• Migrated to better, and clearer code structure and sub-modules

• Mangling now done within python, rather than C++ in CoCo

• File I/O and interaction with CoCo LCfit output now operational - 11/01/17

• Usage of a SN class now solid

• Implementing calls to CoCo LCfit now

• Calls to CoCo specfit now implemented

• Mangling now stable

• Adding tools for calculating magnitude offsets to make it easier to import new data

• regeneration of filter list file now possible

• added colours and bandpasses for LSST filters

• Less dependence on environment variables

• installation through setup.py now possible

• better handing of CoCo sim outputs

• SN position and mu now stored in infofile ./testdata/info/info.dat

• dark sky calculations and integration with LSST Throughputs now done

• Improved stability

• input from astropy tables now more straightforward - better integration with coco.simulate

• can now batch fit light curves and spectra from within python

---

To install:

pip install pycocosn

To install from source:

git clone https://github.com/RobFirth/verbose-enigma.git

then:

cd verbose-enigma
python setup.py install --user

(The --user argument only installs current user only, omitting flag will install for all users on the system if there are appropriate permissions)

NOTE: make sure that the python used to install is the one that you will use with pycoco

---

Ideally set the following environment variables:

COCO_ROOT_DIR (my default is ~/Code/CoCo/) PYCOCO_FILTER_DIR(my default is ~/Code/CoCo/data/filters/) PYCOCO_DATA_DIR (my default is ~/Code/CoCo/data/) SFD_DIR (my default is ~/data/Dust/sfddata-master/; see below) LSST_THROUGHPUTS (my default is ${HOME}/projects/LSST/throughputs) LSST_THROUGHPUTS_BASELINE (my default is ${LSST_THROUGHPUTS}/baseline)

also pycoco/ and CoCo need to be in your path and pythonpath, i.e.:

setenv PATH /Users/berto/Code/pycoco/:$PATH

setenv PYTHONPATH "/Users/berto/Code/pycoco/:$PYTHONPATH

---

Requirements

python packages

• matplotlib
• numpy
• scipy
• astropy
• lmfit
• sfdmap (https://github.com/kbarbary/sfdmap)

additionally

• lsst throughputs (https://github.com/lsst/throughputs; $LSST_THROUGHPUTS points to this directory)

---

for sfdmap, the environment variable SFD_DIR needs to point at the path to the parent directory of the appropriate dust map files. See the installation instructions here: https://github.com/kbarbary/sfdmap

---

Known Problems -

If using in an environment (i.e. through (ana)conda) on Mac and you see the following:

~/anaconda3/lib/python3.6/site-packages/pycocosn-0.9.6-py3.6.egg/pycoco/__init__.py in <module>()
     24 from . import extinction
     25 from . import colours
---> 26 from . import utils
     27 from . import errors
     28 from . import kcorr

~/anaconda3/lib/python3.6/site-packages/pycocosn-0.9.6-py3.6.egg/pycoco/utils.py in <module>()
     12 import warnings
     13
---> 14 import matplotlib.pyplot as plt
     15 from astropy import units as u
     16 from astropy.table import Table, Column

~/anaconda3/lib/python3.6/site-packages/matplotlib/pyplot.py in <module>()
    111 ## Global ##
    112
--> 113 _backend_mod, new_figure_manager, draw_if_interactive, _show = pylab_setup()
    114
    115 _IP_REGISTERED = None

~/anaconda3/lib/python3.6/site-packages/matplotlib/backends/__init__.py in pylab_setup(name)
     58     # imports. 0 means only perform absolute imports.
     59     backend_mod = __import__(backend_name, globals(), locals(),
---> 60                              [backend_name], 0)
     61
     62     # Things we pull in from all backends

~/anaconda3/lib/python3.6/site-packages/matplotlib/backends/backend_macosx.py in <module>()
     17
     18 import matplotlib
---> 19 from matplotlib.backends import _macosx
     20
     21 from .backend_agg import RendererAgg, FigureCanvasAgg

RuntimeError: Python is not installed as a framework. The Mac OS X backend will not be able to function correctly if Python is not installed as a framework. See the Python documentation for more information on installing Python as a framework on Mac OS X. Please either reinstall Python as a framework, or try one of the other backends. If you are using (Ana)Conda please install python.app and replace the use of 'python' with 'pythonw'. See 'Working with Matplotlib on OSX' in the Matplotlib FAQ for more information.

you need to swap the default backend.

If you have installed the pip matplotlib, there is a directory in your root called ``~/.matplotlib`.

Create a file ``~/.matplotlib/matplotlibrcthere and add the following:backend: TkAgg`

pycoco's Issues

SNClass.get_simplespecphot() gets everything twice in photdata

Seems to fail on the first attempt for SN1998bw

Reorganise the Package

Too much currently lives in the top level __init__.py

Bad specphot in ./recon and specphased spectra

This came up in #28. The first and 10th spectra seem to be erroneous.

Double check coverage to see if this is an issue that needs to be addressed or just lazy scripting by me.

specfit finds nans and falls over for 1994I_8.69.txt

/Users/berto/anaconda3/lib/python3.6/site-packages/scipy/integrate/quadrature.py:324: RuntimeWarning: divide by zero encountered in true_divide
  h0divh1 = h0 / h1
/Users/berto/anaconda3/lib/python3.6/site-packages/scipy/integrate/quadrature.py:326: RuntimeWarning: divide by zero encountered in true_divide
  y[slice1]*hsum*hsum/hprod +
/Users/berto/anaconda3/lib/python3.6/site-packages/scipy/integrate/quadrature.py:327: RuntimeWarning: invalid value encountered in add
  y[slice2]*(2-h0divh1))
/Users/berto/anaconda3/lib/python3.6/site-packages/scipy/integrate/quadrature.py:326: RuntimeWarning: invalid value encountered in true_divide
  y[slice1]*hsum*hsum/hprod +
/Users/berto/anaconda3/lib/python3.6/site-packages/scipy/integrate/quadrature.py:327: RuntimeWarning: invalid value encountered in multiply
  y[slice2]*(2-h0divh1))
 filter  fitflux spec_filterflux mangledspec_filterflux ... mask knot_colours   lambda_eff 
-------- ------- --------------- ---------------------- ... ---- ------------ -------------
BessellB     1.0             nan                    nan ... True      #0000ff 4354.41054934
BessellI 1.34748             nan                    nan ... True      #8e44ad 8011.65864198
BessellR 1.82889             nan                    nan ... True      #c0392b 6416.11806798
BessellV 1.92963             nan                    nan ... True      #2ca02c  5436.8698117
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-3-849228af5cdb> in <module>()
----> 1 pcc.coco.specfit_all()

~/Code/verbose-enigma/pycoco/coco.py in specfit_all(dirpath, overwrite, verbose)
    322 
    323         # run_specfit(path, plot=plot, verbose=verbose)
--> 324         specfit_sn(snname=snname, listpath=listpath, overwrite=overwrite, verbose=verbose)
    325         if verbose: print("Done")
    326 

~/Code/verbose-enigma/pycoco/coco.py in specfit_sn(SNobject, snname, listpath, photpath, fitpath, anchor_distance, save, plot, coco_dir, verbose, overwrite)
    421 
    422     run_specfit(SNObject=SNobject, wantedfilters=manglefilters, save=save, anchor_distance=anchor_distance, plot=plot,
--> 423                 coco_dir=coco_dir, overwrite=overwrite, verbose=verbose)
    424     pass
    425 

~/Code/verbose-enigma/pycoco/coco.py in run_specfit(SNObject, wantedfilters, anchor_distance, save, plot, coco_dir, overwrite, verbose)
    236 
    237                 S = copy.deepcopy(mS)
--> 238                 fit_dict = kcorr.mangle(SNObject, mS, mS.mjd_obs, wantedfilters, anchor_distance=anchor_distance)
    239                 if plot:
    240                     functions.plot_mangledata(S, fit_dict["data_table"], mS=fit_dict["SpectrumObject"], spl=fit_dict["final_spl"],

~/Code/verbose-enigma/pycoco/kcorr.py in mangle(sn, S, spec_mjd, filters, staticfilter, anchor_distance, verbose)
    493         wanted_filters = data_table[data_table["mask"]]["filter_object"].data
    494 
--> 495         fit_dict = manglespec3(S, spec_mjd, wanted_filters, wanted_flux, data_table)
    496 
    497     return fit_dict

~/Code/verbose-enigma/pycoco/kcorr.py in manglespec3(SpectrumObject, spec_mjd, wanted_filters, wanted_flux, data_table, verbose)
    531 
    532     ## Do the fit
--> 533     out = minimize(manglemin, params, args=(SpectrumObject, data_table), kws=({"verbose": verbose}))
    534     # out = minimize(manglemin, params, args=(SpectrumObject, data_table), epsfcn=1e-5)
    535     if verbose: print(fit_report(out))

~/anaconda3/lib/python3.6/site-packages/lmfit/minimizer.py in minimize(fcn, params, method, args, kws, scale_covar, iter_cb, reduce_fcn, **fit_kws)
   1954                        iter_cb=iter_cb, scale_covar=scale_covar,
   1955                        reduce_fcn=reduce_fcn, **fit_kws)
-> 1956     return fitter.minimize(method=method)

~/anaconda3/lib/python3.6/site-packages/lmfit/minimizer.py in minimize(self, method, params, **kws)
   1647                         val.lower().startswith(user_method)):
   1648                     kwargs['method'] = val
-> 1649         return function(**kwargs)
   1650 
   1651 

~/anaconda3/lib/python3.6/site-packages/lmfit/minimizer.py in leastsq(self, params, **kws)
   1300         np.seterr(all='ignore')
   1301 
-> 1302         lsout = scipy_leastsq(self.__residual, vars, **lskws)
   1303         _best, _cov, infodict, errmsg, ier = lsout
   1304         result.aborted = self._abort

~/anaconda3/lib/python3.6/site-packages/scipy/optimize/minpack.py in leastsq(func, x0, args, Dfun, full_output, col_deriv, ftol, xtol, gtol, maxfev, epsfcn, factor, diag)
    375     if not isinstance(args, tuple):
    376         args = (args,)
--> 377     shape, dtype = _check_func('leastsq', 'func', func, x0, args, n)
    378     m = shape[0]
    379     if n > m:

~/anaconda3/lib/python3.6/site-packages/scipy/optimize/minpack.py in _check_func(checker, argname, thefunc, x0, args, numinputs, output_shape)
     24 def _check_func(checker, argname, thefunc, x0, args, numinputs,
     25                 output_shape=None):
---> 26     res = atleast_1d(thefunc(*((x0[:numinputs],) + args)))
     27     if (output_shape is not None) and (shape(res) != output_shape):
     28         if (output_shape[0] != 1):

~/anaconda3/lib/python3.6/site-packages/lmfit/minimizer.py in __residual(self, fvars, apply_bounds_transformation)
    486         self.result.nfev += 1
    487 
--> 488         out = self.userfcn(params, *self.userargs, **self.userkws)
    489         out = _nan_policy(out, nan_policy=self.nan_policy)
    490 

~/Code/verbose-enigma/pycoco/kcorr.py in manglemin(params, SpectrumObject, data_table, verbose, clamped, *args, **kwargs)
    686         SplObj = interpolate.CubicSpline(data_table["lambda_eff"], weights, bc_type = "clamped")
    687     else:
--> 688         SplObj = interpolate.CubicSpline(data_table["lambda_eff"], weights)
    689 
    690     MangledSpectrumObject.flux = MangledSpectrumObject.flux * SplObj(MangledSpectrumObject.wavelength)

~/anaconda3/lib/python3.6/site-packages/scipy/interpolate/_cubic.py in __init__(self, x, y, axis, bc_type, extrapolate)
    537             raise ValueError("`x` must contain only finite values.")
    538         if not np.all(np.isfinite(y)):
--> 539             raise ValueError("`y` must contain only finite values.")
    540 
    541         dx = np.diff(x)

ValueError: `y` must contain only finite values.

Spectrum looks normal:

Likewise, lightcurve looks normal:

Plotting Optimised for Jupyter Notebooks

Plotting is a little rough when running from command line (interactive or otherwise). Plots use loc > 1.0 to push things like legends off to the side - this doesn't work.

Implement extinction module instead of my own scripts

These look more robust and cythoned so they should be rapid.
http://extinction.readthedocs.io/en/latest/

Load filter by name, as well as by path

in pycoco.FilterClass i.e. pcc.load_filter()

Check that the reading of mrt files is ok - don't need data_start?

Need Header in Phot Currently - Create Defaults?

need to fix bug with needing a header line for some load operations.

If no header, get the following:

>>>P_new = pcc.load('/Users/berto/Code/CoCo/data/lc/SN2011dh.dat')
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-3-2ef8fe586f28> in <module>()
----> 1 P_new = pcc.load('/Users/berto/Code/CoCo/data/lc/SN2011dh.dat')

/Users/berto/Code/verbose-enigma/pycoco/__init__.py in load(path, format, verbose)
    956 def load(path, format = "ascii", verbose = True):
    957     pc = PhotometryClass()
--> 958     pc.phot = load_formatted_phot(path, format = "ascii", verbose = verbose)
    959     pc.unpack(verbose = verbose)
    960     return pc

/Users/berto/Code/verbose-enigma/pycoco/__init__.py in load_formatted_phot(path, format, verbose)
    947     phot_table.meta = {"filename" : path}
    948 
--> 949     phot_table["MJD"].unit = u.day
    950     phot_table["flux"].unit = u.cgs.erg / u.si.angstrom / u.si.cm ** 2 / u.si.s
    951     phot_table["flux_err"].unit =  phot_table["flux"].unit

/Users/berto/anaconda2/lib/python2.7/site-packages/astropy/table/table.pyc in __getitem__(self, item)
   1157     def __getitem__(self, item):
   1158         if isinstance(item, six.string_types):
-> 1159             return self.columns[item]
   1160         elif isinstance(item, (int, np.integer)):
   1161             return self.Row(self, item)

/Users/berto/anaconda2/lib/python2.7/site-packages/astropy/table/table.pyc in __getitem__(self, item)
     96         """
     97         if isinstance(item, six.string_types):
---> 98             return OrderedDict.__getitem__(self, item)
     99         elif isinstance(item, (int, np.integer)):
    100             return self.values()[item]

KeyError: 'MJD'

Non-uniformity in the High-Z simulation specphot

Smooth behaviour in the specphot for within range simulations -

But everything gets a bit noisy when the extensions come into play.

Suspect this has something to do with the spectral extensions being incorrectly flux-calibrated (for want of a better term). I have attempted to fix this in recent commit - 2b859ea

A number (confusingly not all) of the spectra drop to 0 flux at their final point -

edit This appears to be a problem with some of the spectra before I sent them to @SzymonPrajs , so his method is working 100% fine. See

\edit

The merging used was not robust to this, and was extending using a donor multiplied by a factor derived from the final flux, in this case 0. As this appears to be uncommon, and just the end row, I have added to the subroutine in kcorr

    wmax = np.nanmax(S.wavelength)
    wmax_flux = S.flux[np.where(S.wavelength == wmax)][0]
    if verbose: print(wmax, wmax_flux)

    while wmax_flux == 0:
        if verbose:
            print("still 0!")
        S.data = S.data[:-1]
        S.wavelength = S.wavelength[:-1]
        S.flux = S.flux[:-1]

        wmax = np.nanmax(S.wavelength)
        wmax_flux = S.flux[np.where(S.wavelength == wmax)][0]


    if verbose:
        print("Not 0!")
        print(wmax, wmax_flux)

Will need to run some further tests - i.e. re-run the merge, specfits and specphase.

Black Body not good for all extensions

See - #31 .

Ideas:

• use simple linear model, decaying exponential?
• use donor spectra (need to be correct phase and subtype)
• use wavelength cut to limit fits.

Need to symbolically link SN####/ and SN####_m/ directories in `chains/` for specfit_sn

Need to symbolically link SN####/ and SN####_m/ directories in chains/ for specfit_sn

If a filter name is passed to SNClass.plot_lc(...) then it is treated as a list of single char filternames

Calling CoCo

Use a .so shared object library, or just spawn a command?

SpectrumClass.get_specphot() doesn't work for single overlapping filter.

As above.

Last Spectrum Missing from SN.plot_spec()?

>SN.spec
OrderedDict([('SN10as_100320_DUP_WF_b01.dat',
              <pycoco.classes.SpectrumClass at 0x11d262ef0>),
             ('SN10as_100321_GEM_GM_r01.dat',
              <pycoco.classes.SpectrumClass at 0x11cfd5780>),
             ('SN10as_100322_DUP_WF_b01.dat',
              <pycoco.classes.SpectrumClass at 0x1220c2898>),
             ('SN10as_100323_DUP_WF_b01.dat',
              <pycoco.classes.SpectrumClass at 0x123406be0>),
             ('SN10as_100324_DUP_WF_b01.dat',
              <pycoco.classes.SpectrumClass at 0x123443b00>),
             ('SN10as_100325_DUP_WF_b01.dat',
              <pycoco.classes.SpectrumClass at 0x11ba51550>),
             ('SN10as_100325_VLT_XS_combined.dat',
              <pycoco.classes.SpectrumClass at 0x11cfd5f98>),
             ('SN10as_100326_VLT_XS_combined.dat',
              <pycoco.classes.SpectrumClass at 0x122994fd0>),
             ('SN10as_100328_VLT_XS_combined.dat',
              <pycoco.classes.SpectrumClass at 0x1229a4240>),
             ('SN10as_100329_VLT_XS_combined.dat',
              <pycoco.classes.SpectrumClass at 0x11d080e48>),
             ('SN10as_100409_SOA_GO_g01.dat',
              <pycoco.classes.SpectrumClass at 0x11d074ba8>),
             ('SN10as_100422_GEM_GM_c01.dat',
              <pycoco.classes.SpectrumClass at 0x11bd35208>),
             ('SN10as_100708_GEM_GM_c01.dat',
              <pycoco.classes.SpectrumClass at 0x122faffd0>),
             ('SN10as_100723_SOA_GO_g01.dat',
              <pycoco.classes.SpectrumClass at 0x11d40f2b0>),
             ('SN10as_100802_GEM_GM_c01.dat',
              <pycoco.classes.SpectrumClass at 0x11b796ba8>),
             ('SN10as_100805_SOA_GO_g01.dat',
              <pycoco.classes.SpectrumClass at 0x11bc1bcc0>),
             ('SN10as_110205_BAA_IM_b01.dat',
              <pycoco.classes.SpectrumClass at 0x11bd41240>),
             ('SN10as_110205_BAA_IM_r01.dat',
              <pycoco.classes.SpectrumClass at 0x11bd2c588>)])

note final file - 'SN10as_110205_BAA_IM_r01.dat'.

Double Check Redshift Propagation

kcorr_data path goes wrong when built

when runnning from egg rather than just having module in the path, the default_kcorr_dir goes to /Users/bertiepwhite/.local/lib/python3.6/site-packages/pycoco-0.7.0-py3.6.egg/pycoco/kcorr_data
Solution to this is to have a data directory and manifest set up in the setup.py

1994I throws nan for specphot

As title, problem occurs when running get_specphot such as in scripts/extend_all_spectra.py

pycoco.coco.test_specfit doesn't

It tests lcfit, need to sort that.

Use SNClass.check_overlaps(...) in the SNClass.plot_lc(...) instead of the hacky way it is now

SNClass.check_overlaps(...) makees a list of overlapping filters. At the moment, SNClass.plot_lc(...) looks like:

if mark_spectra:

    for spec_key in self.spec:
        if verbose: print(np.nanmin(self.spec[spec_key].wavelength) - filter_uncertainty, self.phot.data_filters[filter_key]._lower_edge)
        if verbose: print(np.nanmax(self.spec[spec_key].wavelength) + filter_uncertainty, self.phot.data_filters[filter_key]._upper_edge)

        if verbose: print(self.spec[spec_key].data.meta["filename"] )
        too_blue =  self.phot.data_filters[filter_key]._lower_edge < np.nanmin(self.spec[spec_key].wavelength) - filter_uncertainty
        too_red = self.phot.data_filters[filter_key]._upper_edge > np.nanmax(self.spec[spec_key].wavelength) + filter_uncertainty
        # if self.spec[spec_key]. self.phot.data_filters[filter_key]._upper_edge and self.phot.data_filters[filter_key]._lower_edge
        if verbose: print(too_blue, too_red)
        if not too_red and not too_blue:
            ax1.plot([self.spec[spec_key].mjd_obs, self.spec[spec_key].mjd_obs],
                     [0.0, np.nanmax(self.phot.phot['flux'])*1.5],
                     ls = ':', color = hex['batman'], zorder = 0)

Should be able to replace it with:

for spectrum in self.spec:
    if filter in self.spec. _overlapping_filter_list:
        plot spec

SpectrumClass.get_specphot doesn't work when there is only one filter.

As title. e.g. when running scripts/extend_all_spectra.py

SN1993J
Traceback (most recent call last):
  File "/Users/berto/Code/verbose-enigma/scripts/extend_all_spectra.py", line 26, in <module>
    S.get_specphot(sn.phot.data_filters, verbose=True)
  File "/Users/berto/Code/verbose-enigma/pycoco/classes.py", line 558, in get_specphot
    for j, filter_name in enumerate(self._overlapping_filter_list):
TypeError: iteration over a 0-d array

How are we doing dust values? Need to have EBV_MW for SNe

use https://github.com/kbarbary/sfdmap?

Removing missing filters from light curve files

There are a few places within CoCo where it doesn't have the ability to verify if there is are response files present for every filter used in the input. Light curves passed to CoCo have to be checked for this to avoid crashes.

simplespecphot for Fig 2 in paper

I could do this by hand, but it might not be a bad idea to just code it up, in case it turns out to be useful. Will run in new branch.

pycoco.coco.specfit_sn no longer needs to generate lc + '_m.dat' files

Roman Machine - FAIL: test_PhotometryClass_get_data_dir_returns_default (__main__.TestClass)

Fail on Roman when running test scripts.
Traceback:

======================================================================
ERROR: test_PhotometryClass_get_data_dir_returns_default (__main__.TestClass)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/berto/Code/verbose-enigma/pycoco/classes.py", line 564, in set_filter_directory
    if not os.path.isdir(self.filter_directory):
AttributeError: 'PhotometryClass' object has no attribute 'filter_directory'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/berto/Code/verbose-enigma/scripts/test_pycoco.py", line 148, in test_PhotometryClass_get_data_dir_returns_default
    x = pcc.classes.PhotometryClass()
  File "/Users/berto/Code/verbose-enigma/pycoco/classes.py", line 1218, in __init__
    self.set_filter_directory(self._get_filter_directory())
  File "/Users/berto/Code/verbose-enigma/pycoco/classes.py", line 574, in set_filter_directory
    + " you messed with _default_filter_dir_path?")
pycoco.errors.PathError: The default filter directory '/Users/berto/Code/CoCo/data/filters/' doesn't exist. Or isn't a directory. Or can't be located. Have you messed with _default_filter_dir_path?

Some issue with the filter directory?

RuntimeError: Python is not installed as a framework

/Users/berto/anaconda3/envs/pycoco/bin/python
Python 3.6.3 :: Anaconda, Inc.
To switch back to your default Python environment, type 'source deactivate.csh'
(pycoco)[uos-12948:~] berto% ipython
Python 3.6.3 |Anaconda, Inc.| (default, Nov  8 2017, 18:10:31) 
Type 'copyright', 'credits' or 'license' for more information
IPython 6.2.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import pycoco as pcc
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-1-368bbd834de0> in <module>()
----> 1 import pycoco as pcc

~/Code/verbose-enigma/pycoco/__init__.py in <module>()
     24 from . import extinction
     25 from . import colours
---> 26 from . import utils
     27 from . import errors
     28 from . import kcorr

~/Code/verbose-enigma/pycoco/utils.py in <module>()
     12 import warnings
     13 
---> 14 import matplotlib.pyplot as plt
     15 from astropy import units as u
     16 from astropy.table import Table, Column

~/anaconda3/envs/pycoco/lib/python3.6/site-packages/matplotlib/pyplot.py in <module>()
    111 ## Global ##
    112 
--> 113 _backend_mod, new_figure_manager, draw_if_interactive, _show = pylab_setup()
    114 
    115 _IP_REGISTERED = None

~/anaconda3/envs/pycoco/lib/python3.6/site-packages/matplotlib/backends/__init__.py in pylab_setup(name)
     58     # imports. 0 means only perform absolute imports.
     59     backend_mod = __import__(backend_name, globals(), locals(),
---> 60                              [backend_name], 0)
     61 
     62     # Things we pull in from all backends

~/anaconda3/envs/pycoco/lib/python3.6/site-packages/matplotlib/backends/backend_macosx.py in <module>()
     17 
     18 import matplotlib
---> 19 from matplotlib.backends import _macosx
     20 
     21 from .backend_agg import RendererAgg, FigureCanvasAgg

RuntimeError: Python is not installed as a framework. The Mac OS X backend will not be able to function correctly if Python is not installed as a framework. See the Python documentation for more information on installing Python as a framework on Mac OS X. Please either reinstall Python as a framework, or try one of the other backends. If you are using (Ana)Conda please install python.app and replace the use of 'python' with 'pythonw'. See 'Working with Matplotlib on OSX' in the Matplotlib FAQ for more information.

pcc.utils.get_notebooks() crash

pcc.utils.get_notebooks() crashes for me. I think you need to change line 609 of utils.py:

    tar = tarball.open(tarball, "r:gz")

to
tar = tarfile.open(tarball, "r:gz")

or at least that worked for me.

Negative Temperature for SN1993J_16.0.dat pcc.kcorr.fit_bb(...)

1993J_16.0.txt
<class 'collections.OrderedDict'> OrderedDict([('BessellB', <pycoco.classes.FilterClass object at 0x1a1e0a15f8>), ('BessellV', <pycoco.classes.FilterClass object at 0x1a1de9e400>), ('BessellR', <pycoco.classes.FilterClass object at 0x1a1def8198>), ('BessellI', <pycoco.classes.FilterClass object at 0x1a1e0a8198>)])
<class 'numpy.ndarray'> BessellV
only one overlapping filter
<class 'collections.OrderedDict'> OrderedDict([('BessellB', <pycoco.classes.FilterClass object at 0x1a1e0a15f8>), ('BessellV', <pycoco.classes.FilterClass object at 0x1a1de9e400>), ('BessellR', <pycoco.classes.FilterClass object at 0x1a1def8198>), ('BessellI', <pycoco.classes.FilterClass object at 0x1a1e0a8198>)])
0 BessellB
flux in filter BessellB  is  4.70761648627e-18
1 BessellV
flux in filter BessellV  is  1.34701077008e-17
2 BessellR
flux in filter BessellR  is  1.50920189511e-17
3 BessellI
flux in filter BessellI  is  1.37368916145e-20
Traceback (most recent call last):
  File "/Users/berto/Code/verbose-enigma/scripts/extend_all_spectra.py", line 31, in <module>
    new_spec.load_table(pcc.kcorr.fit_bb(S, filter_dict=sn.phot.data_filters, return_table=True))
  File "/Users/berto/Code/verbose-enigma/pycoco/kcorr.py", line 799, in fit_bb
    kws=({"verbose": verbose}))
  File "/Users/berto/anaconda3/lib/python3.6/site-packages/lmfit/minimizer.py", line 1956, in minimize
    return fitter.minimize(method=method)
  File "/Users/berto/anaconda3/lib/python3.6/site-packages/lmfit/minimizer.py", line 1649, in minimize
    return function(**kwargs)
  File "/Users/berto/anaconda3/lib/python3.6/site-packages/lmfit/minimizer.py", line 1302, in leastsq
    lsout = scipy_leastsq(self.__residual, vars, **lskws)
  File "/Users/berto/anaconda3/lib/python3.6/site-packages/scipy/optimize/minpack.py", line 387, in leastsq
    gtol, maxfev, epsfcn, factor, diag)
  File "/Users/berto/anaconda3/lib/python3.6/site-packages/lmfit/minimizer.py", line 488, in __residual
    out = self.userfcn(params, *self.userargs, **self.userkws)
  File "/Users/berto/Code/verbose-enigma/pycoco/kcorr.py", line 722, in bb_min
    bb_flux = bb.blackbody_lambda(bb_wavelength, temperature=T*u.Kelvin)
  File "/Users/berto/anaconda3/lib/python3.6/site-packages/astropy/modeling/blackbody.py", line 345, in blackbody_lambda
    bb_nu = blackbody_nu(in_x, temperature) * u.sr  # Remove sr for conversion
  File "/Users/berto/anaconda3/lib/python3.6/site-packages/astropy/modeling/blackbody.py", line 294, in blackbody_nu
    raise ValueError('Temperature should be positive: {0}'.format(temp))
ValueError: Temperature should be positive: -19698.712029014747 K

Process finished with exit code 1

Make Python 3 compatible

http://python3porting.com/differences.html

SDSS and Bessell Bands in wrong place for SN2009jf

Resampling a filter to a spectrum that ends before the filter crops the filter

This needs to be addressed - is a warning sufficient? In a sense it is a fair representation of that particular spectrum going through the filter. On the other hand it systematically underestimates the flux that would come from a real spectrum, which would not be cut off in the same way...

Use BaseClass inheritance for both LCfit and PhotometryClass

Seems to be quite a few shared methods

Reduce number of required Environment Variables

It should be possible to work out default filter directories etc from the COCO_ROOT_DIR etc