Why do some spectra files need to be in a subdirectory (LyaSNR)? about desimodel

[andreufont]

@weaverba137 These files are used (by Pat McDonald and myself) to compute Fisher forecasts for Lyman alpha in DESI, I'm not sure if anyone else uses them.

There are older files (like desimodel/data/spectra/sn-spec-lya-20.25.dat) that are obsolete, since they used an older version of the DESI throughput.

At the time I made the new files I wasn't sure whether anyone was using the old files, so I didn't remove them, and instead added a new folder with the new Lya SNR files. These have a somewhat different name, that includes the band used to identify the quasar magnitude and the exposure time (like desimodel/data/spectra/LyaSNR/sn-spec-lya-r21.25-t4000.dat).

If none else is using these files, I'd be happy to move them to the main directory.

And if none else is using the old sn-spec-lya files, I'd be happy to delete these (specially since as I said I believe these use an old version of the throughput).

These files where generated using a python code developed by @dkirkby, so he might know better if someone else is using these files.

[dkirkby]

@andreufont I think we should remove these files from desimodel and instead put the script to generate them with specsim into an igmhub package. That would allow us to always generate them on the fly with the latest throughputs, fiberloss fractions, etc (which have changed quite a bit) and also generate corresponding eBOSS SNR files (once @belaa has finished her project to adapt specsim for eBOSS). I can help with this if you point me to the original code you used.

[andreufont]

@dkirkby The script I used was written by you, and it is already in the right place (I think): desihub/desimodel/bin/desi_quicklya.py.
I haven't used it in over a year, and I'm not even sure it is working now (I just tried to do what I think I did last time, but it crashes).
While I agree that it would be great to be able to compute whatever SNR you need on the fly, the current Fisher forecast code can not do this right now and it needs to use pre-existing files from disk.

[weaverba137]

While this is being resolved, let return to the initial question. If, as you say:

There are older files (like desimodel/data/spectra/sn-spec-lya-20.25.dat) that are obsolete, since they used an older version of the DESI throughput.

Why not just replace those files with the files in LyaSNR? This is an svn repository after all, & the old files will still be there in the older versions.

[andreufont]

@weaverba137 When I did this I had no idea how desimodel was organized and whether anyone was using those files, and I didn't want to modify other's results. It started with me just playing around, but then we decided to use the new files for our Fisher forecast and I had to add the files to svn...
But yes, I think now we could safely remove the old files and move the files in LyaSNR to the main folder.

[dkirkby]

The newer Lya SNR files are most likely also out of date.

…

On Fri, May 5, 2017 at 11:52 AM Andreu Font-Ribera ***@***.***> wrote: @weaverba137 <https://github.com/weaverba137> When I did this I had no idea how desimodel was organized and whether anyone was using those files, and I didn't want to modify other's results. It started with me just playing around, but then we decided to use the new files for our Fisher forecast and I had to add the files to svn... But yes, I think now we could safely remove the old files and move the files in LyaSNR to the main folder. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#49 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AALSrzjgZr1wp8DZ5aLLod3Sl1tZK57nks5r227LgaJpZM4NRWu6> .

[andreufont]

Sure. And so is the quasar luminosity function used for the forecasts.
I found some time to work on a new python-based forecast code, but it is more complicated than I was hoping for and I'm not really close to finished :-(

[weaverba137]

Adding label 'data model' note that fixing this issue will also involve changes to the data model.

[andreufont]

Ben, I just talked to Pat (the only other user of these files), and he is fine if we move the files from the current subfolder to the main folder where the other snr files are. We can also remove the old sn-spec-lya-* files.

[weaverba137]

OK, thanx. We should coordinate this with the next tag of desimodel.

[weaverba137]

Fixed in #53. Will close this once the changes are implemented on the svn side.

[andreufont]

Extra Lya SNR directory has been deleted from SVN.

[weaverba137]

Very good. Closing.