musevlt / zap Goto Github PK
View Code? Open in Web Editor NEWThis project forked from ktsoto/zap
the Zurich Atmosphere Purge. Sky subtraction for MUSE.
Home Page: http://zap.readthedocs.io/
License: MIT License
zap's People
Contributors
Stargazers
Watchers
zap's Issues
Option to mask wavelength ranges (with known spectral features) in the reprojection
In the PCA sky subtraction developed by Wild & Hewett for SDSS spectra a spectral feature mask was utilised, so that spectra with emission lines at known places would not change the amplitudes to favour components that could mimic this feature.
I think such a feature could also improve the quality of the sky-subtraction (especially in the filled) with ZAP.
Here is the link to the paper Wild & Hewett: https://academic.oup.com/mnras/article/358/3/1083/1025290
Treatment of variance [STAT] cubes
From @Azeret on September 4, 2018 17:52
It seems the ZAP does not apply to variance [STAT] cubes, only to the data ones.
Is there a way to change this?
After running the ZAP it results in original (therefore very large) variance [STAT] cubes that prevents usage and further analysis.
Thank you for response!
Tereza (tjerabko(at)eso.org)
Copied from original issue: ktsoto#4
Python Version 3.8
Was the zap code already tested with python version 3.8. I am trying to use it and get a persistent error that looks like this :
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/spawn.py", line 134, in _check_not_importing_main
raise RuntimeError('''
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.
This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:
if __name__ == '__main__':
freeze_support()
...
The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.
Traceback (most recent call last):
File "zap_noskycube.py", line 40, in <module>
zap.process('DATACUBE_NOSKY.fits', outcubefits='DATACUBE_NOSKY_ZAP_NOMASK.fits', mask='SKYMASK.fits')
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/zap/zap.py", line 173, in process
extSVD = SVDoutput(cubefits, clean=clean, zlevel=zlevel,
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/zap/zap.py", line 229, in SVDoutput
zobj._prepare(clean=clean, zlevel=zlevel, cftype=cftype,
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/zap/zap.py", line 283, in wrapped
res = func(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/zap/zap.py", line 455, in _prepare
self._zlevel(calctype=zlevel)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/zap/zap.py", line 283, in wrapped
res = func(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/zap/zap.py", line 579, in _zlevel
self.zlsky = np.hstack(parallel_map(func, self.stack, NCPU, axis=0))
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/zap/zap.py", line 883, in parallel_map
manager = Manager()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py", line 57, in Manager
m.start()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/managers.py", line 583, in start
self._address = reader.recv()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/connection.py", line 250, in recv
buf = self._recv_bytes()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/connection.py", line 414, in _recv_bytes
buf = self._recv(4)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/connection.py", line 383, in _recv
raise EOFError
EOFError
Problem at the edge in the reconstructed data cube/image
Incomplete spectrum at the edges of the data cubes are not processed by ZAP, since they have too many NaNs (ZAP filter out spaxels with more than 25% of NaNs). Because of this the zapped cubes contains edges values strongly positive or negative, and this propagates in the combined cubes.
We could provide a script or an option to remove these edges, I already have something for this somewhere.
Warn if a slice contains too many NaN values
Zap cleans the NaN values before doing the PCA (Spaxels with more then 25% of NaN values are removed, the others are replaced with an interpolation from the neighbors.) but this does not take into account specific cases where a slice of the cube has many NaNs, typically at the beginning or end of the cube.
This may lead to unexpected results because of the interpolation on these pixels, so it would be useful to at least emit a warning when a slice has more than x % of NaNs.
ZAP - KCWI
I'm currently looking at using your code ZAP on some KCWI data. Messing about with some test data I am running into problems where it seems the FITS header isn't being correctly parsed by the program [see below]. Do you know of anyone using ZAP for KCWI data?
It seems to be a problem with fits.open in astropy.io:
I realise that this may not be related to ZAP but will be grateful for any help you are able to provide.
Documentation - interactive mode - wrong command?
In index.rst on line 222, where the interactive mode is described, it says:
zobj.writefits(outcubefits='DATACUBE_FINAL_ZAP.fits')
Shouldn't this be:
zobj.mergefits(outcubefits='DATACUBE_FINAL_ZAP.fits')
Thanks,
Johannes
Subaru/FOCAS support
There is an IFU mode for FOCAS at Subaru Telescope (https://www2.nao.ac.jp/~shinobuozaki/focasifu/) and I wanted to use ZAP for data taken with it. I've just copied and pasted the code blocks for KCWI so that ZAP runs for my FOCAS IFU data. The current FOCAS IFU pipeline () produces a FITS file with the hdu[0] for the cube data.
The modification I made (9d7c596) apparently works nicely with my science cubes (though I haven't done extensive test).
However, it failed to produce the explained variance curves with the following error message. Do you have any ideas how to fix it?
I also think that it would be nice to have options to specify which extension (and axis configuration) ZAP should look at rather than coding instrument-by-instrument basis.
Python 3.6.9 |Anaconda, Inc.| (default, Jul 30 2019, 13:42:17)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.8.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import zap
zobj = zap.
In [2]: zobj = zap.process('cube01.fits', outcubefits='output01.fits')
[INFO] Running ZAP 2.1 !
WARNING: FITSFixedWarning: RADECSYS= 'FK5 ' / The equatorial coordinate system
the RADECSYS keyword is deprecated, use RADESYSa. [astropy.wcs.wcs]
[WARNING] FITSFixedWarning: RADECSYS= 'FK5 ' / The equatorial coordinate system
the RADECSYS keyword is deprecated, use RADESYSa.
WARNING: FITSFixedWarning: 'datfix' made the change 'Set MJD-OBS to 58664.000000 from DATE-OBS.
Set DATE-END to '2019-06-30T06:35:48.104' from MJD-END'. [astropy.wcs.wcs]
[WARNING] FITSFixedWarning: 'datfix' made the change 'Set MJD-OBS to 58664.000000 from DATE-OBS.
Set DATE-END to '2019-06-30T06:35:48.104' from MJD-END'.
[INFO] Cleaning NaN values in the cube
[INFO] Rejected 0 spaxels with more than 25.0% NaN pixels
[INFO] Fixing 0 remaining NaN pixels
[INFO] _nanclean - Time: 0.36 sec.
[INFO] Extract to 2D, 1536 valid spaxels (100%)
[INFO] _extract - Time: 0.08 sec.
[INFO] Median zlevel subtraction
[INFO] _zlevel - Time: 0.12 sec.
[INFO] Applying Continuum Filter, cftype=median
[INFO] Using cfwidth=300
[INFO] _continuumfilter - Time: 2.55 sec.
[INFO] _prepare - Time: 3.14 sec.
[INFO] Calculating SVD on 2 segments ([[ 0 3561]
[3561 3781]])
[INFO] _msvd - Time: 0.60 sec.
[INFO] Compute number of components
[INFO] Choosing [33 6] eigenspectra for segments
[INFO] Reconstructing Sky Residuals
[INFO] reconstruct - Time: 0.05 sec.
[INFO] Applying correction and reshaping data product
[INFO] Cube file saved to output01.fits
[INFO] Zapped! (took 4.60 sec.)
In [3]: zobj = zap.process('cube01.fits', outcubefits='output01-2.fits', varcurvefits='output01_varcurve.fits')
[INFO] Running ZAP 2.1 !
WARNING: FITSFixedWarning: RADECSYS= 'FK5 ' / The equatorial coordinate system
the RADECSYS keyword is deprecated, use RADESYSa. [astropy.wcs.wcs]
[WARNING] FITSFixedWarning: RADECSYS= 'FK5 ' / The equatorial coordinate system
the RADECSYS keyword is deprecated, use RADESYSa.
WARNING: FITSFixedWarning: 'datfix' made the change 'Set MJD-OBS to 58664.000000 from DATE-OBS.
Set DATE-END to '2019-06-30T06:35:48.104' from MJD-END'. [astropy.wcs.wcs]
[WARNING] FITSFixedWarning: 'datfix' made the change 'Set MJD-OBS to 58664.000000 from DATE-OBS.
Set DATE-END to '2019-06-30T06:35:48.104' from MJD-END'.
[INFO] Cleaning NaN values in the cube
[INFO] Rejected 0 spaxels with more than 25.0% NaN pixels
[INFO] Fixing 0 remaining NaN pixels
[INFO] _nanclean - Time: 0.02 sec.
[INFO] Extract to 2D, 1536 valid spaxels (100%)
[INFO] _extract - Time: 0.08 sec.
[INFO] Median zlevel subtraction
[INFO] _zlevel - Time: 0.06 sec.
[INFO] Applying Continuum Filter, cftype=median
[INFO] Using cfwidth=300
[INFO] _continuumfilter - Time: 2.55 sec.
[INFO] _prepare - Time: 2.75 sec.
[INFO] Calculating SVD on 2 segments ([[ 0 3561]
[3561 3781]])
[INFO] _msvd - Time: 0.48 sec.
[INFO] Compute number of components
[INFO] Choosing [33 6] eigenspectra for segments
[INFO] Reconstructing Sky Residuals
[INFO] reconstruct - Time: 0.05 sec.
[INFO] Applying correction and reshaping data product
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-3-a0bd10bfa0c3> in <module>
----> 1 zobj = zap.process('cube01.fits', outcubefits='output01-2.fits', varcurvefits='output01_varcurve.fits')
~/anaconda3/envs/astroconda/lib/python3.6/site-packages/zap/zap.py in process(cubefits, outcubefits, clean, zlevel, cftype, cfwidthSVD, cfwidthSP, nevals, extSVD, skycubefits, mask, interactive, ncpu, pca_class, n_components, overwrite, varcurvefits)
186
187 if varcurvefits is not None:
--> 188 zobj.writevarcurve(varcurvefits=varcurvefits, overwrite=overwrite)
189
190 zobj.mergefits(outcubefits, overwrite=overwrite)
~/anaconda3/envs/astroconda/lib/python3.6/site-packages/zap/zap.py in writevarcurve(self, varcurvefits, overwrite)
804 """Write the explained variance curves to an individual fits file."""
805 from astropy.table import Table
--> 806 table = Table([m.explained_variance_ for m in self.models])
807 hdu = fits.table_to_hdu(table)
808 _newheader(self, hdu.header)
~/anaconda3/envs/astroconda/lib/python3.6/site-packages/astropy/table/table.py in __init__(self, data, masked, names, dtype, meta, copy, rows, copy_indices, **kwargs)
536
537 # Finally do the real initialization
--> 538 init_func(data, names, dtype, n_cols, copy)
539
540 # Set table meta. If copy=True then deepcopy meta otherwise use the
~/anaconda3/envs/astroconda/lib/python3.6/site-packages/astropy/table/table.py in _init_from_list(self, data, names, dtype, n_cols, copy)
836 cols.append(col)
837
--> 838 self._init_from_cols(cols)
839
840 def _init_from_ndarray(self, data, names, dtype, n_cols, copy):
~/anaconda3/envs/astroconda/lib/python3.6/site-packages/astropy/table/table.py in _init_from_cols(self, cols)
886 if len(lengths) > 1:
887 raise ValueError('Inconsistent data column lengths: {0}'
--> 888 .format(lengths))
889
890 # Set the table masking
ValueError: Inconsistent data column lengths: {1536, 220}
In [4]:
Use DQ extension if present
To solve issues like #13, zap should check if a DQ extension is present and make sure that the masked voxels are set to NaN.
NaN left in zapped data cube
dir08 = '/muse/MXDF/private/data/reduced/0.8/zap/gradeAB'
dname = dir08+'/DATACUBE_ZAP.fits'
cube = Cube(dname)
sp = cube07[1000:1040,200,200]
sp.data
masked_array(data=[--, --, --, --, --, --, --, --, --, --, --, --, nan,
0.5418558716773987, 0.24525581300258636,
-0.19818709790706635, 0.059832070022821426,
while this was not the case for the previous version
sp = cube07[1000:1040,200,200]
sp.data
masked_array(data=[--, --, --, --, --, --, --, --, --, --, --, --, --,
0.5386847853660583, 0.24470342695713043,
-0.2502848207950592, 0.10236702114343643,
Note this might be due to the new DRS version, a better way to mask the AO masked filtered range should be implemented
Unexpected crashes for zap
Hi, I'm trying to zap a fairly old datacube (from commissioning) which was successfully zapped back with the old version but I haven't touched it for years. now however it keeps failing and it is not clear what is going on (latest version of zap [2.1.dev191], latest CRAL version of MPDAF [3.3]). This happens with or without masking. Any idea what is going wrong?
In [3]: extSVD = zap.SVDoutput('DATACUBE_FINAL.fits')
[INFO] Processing DATACUBE_FINAL.fits to compute the SVD
[INFO] Cleaning NaN values in the cube
[INFO] Rejected 7034 spaxels with more than 25.0% NaN pixels
[INFO] Fixing 25689091 remaining NaN pixels
/home/jarle/Source/zap/zap/zap.py:1074: RuntimeWarning: Mean of empty slice
cleancube[z, y, x] = np.nanmean(neighbor, axis=1)
[INFO] _nanclean - Time: 67.47 sec.
[INFO] Extract to 2D, 0 valid spaxels (0%)
[INFO] _extract - Time: 1.39 sec.
[INFO] Median zlevel subtraction
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:2920: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/data/jarle/anaconda3/lib/python3.7/site-packages/numpy/core/_methods.py:78: RuntimeWarning: invalid value encountered in true_divide
ret, rcount, out=ret, casting='unsafe', subok=False)
[INFO] _zlevel - Time: 2.24 sec.
[INFO] Applying Continuum Filter, cftype=median
[INFO] Using cfwidth=300
[INFO] _continuumfilter - Time: 2.07 sec.
[INFO] _prepare - Time: 73.19 sec.
[INFO] Calculating SVD on 1 segments ([[ 0 4445]])
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-3-3f99c02818d7> in <module>
----> 1 extSVD = zap.SVDoutput('DATACUBE_FINAL.fits')
~/Source/zap/zap/zap.py in SVDoutput(cubefits, clean, zlevel, cftype, cfwidth, mask, ncpu, pca_class, n_components)
230 zobj._prepare(clean=clean, zlevel=zlevel, cftype=cftype,
231 cfwidth=cfwidth, mask=mask)
--> 232 zobj._msvd()
233 return zobj
234
~/Source/zap/zap/zap.py in wrapped(*args, **kwargs)
282 def wrapped(*args, **kwargs):
283 t0 = time()
--> 284 res = func(*args, **kwargs)
285 logger.info('%s - Time: %.2f sec.', func.__name__, time() - t0)
286 return res
~/Source/zap/zap/zap.py in _msvd(self)
656 ncomp = None
657
--> 658 self.models.append(self.pca_class(n_components=ncomp).fit(x))
659
660 def chooseevals(self, nevals=[]):
/data/jarle/anaconda3/lib/python3.7/site-packages/sklearn/decomposition/pca.py in fit(self, X, y)
338 Returns the instance itself.
339 """
--> 340 self._fit(X)
341 return self
342
/data/jarle/anaconda3/lib/python3.7/site-packages/sklearn/decomposition/pca.py in _fit(self, X)
379
380 X = check_array(X, dtype=[np.float64, np.float32], ensure_2d=True,
--> 381 copy=self.copy)
382
383 # Handle n_components==None
/data/jarle/anaconda3/lib/python3.7/site-packages/sklearn/utils/validation.py in check_array(array, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, ensure_min_samples, ensure_min_features, warn_on_dtype, estimator)
580 " minimum of %d is required%s."
581 % (n_samples, shape_repr, ensure_min_samples,
--> 582 context))
583
584 if ensure_min_features > 0 and array.ndim == 2:
ValueError: Found array with 0 sample(s) (shape=(0, 4445)) while a minimum of 1 is required.
Add notch filter limits for the NFM
NFM is not handled currently :
https://github.com/musevlt/zap/blob/master/zap/zap.py#L56-L60
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.