GithubHelp home page GithubHelp logo

gcalderone / qsfit Goto Github PK

View Code? Open in Web Editor NEW
13.0 7.0 6.0 20.39 MB

Quasar Spectral Fitting

Home Page: http://qsfit.inaf.it/

License: GNU General Public License v3.0

IDL 6.65% Prolog 0.02% Perl 0.01% Makefile 88.25% sed 5.07%

qsfit's Introduction

QSFit

DOI

QSFit (Quasar Spectral Fitting) is a software package written in IDL to automatically perform the analysis of Active Galactic Nuclei (AGN) optical spectra. The software provides luminosity estimates for the AGN continuum, the Balmer continuum, both optical and UV iron blended complex, host galaxy and emission lines, as well as width, velocity offset and equivalent width of 20 emission lines (Halpha, Hbeta, Mgii, [Oiii], Civ, etc.). The ultimate purpose of QSFit is to allow astronomers to run standardized recipes to analyze the AGN data, in a simple, replicable and shareable way. The whole fitting process is customizable for specific needs, and can be extended to analyze spectra from other data sources.

The main website is: http://qsfit.inaf.it.

If you find QSFit useful please cite the refernce paper: Calderone et al. 2017 (also available on arXiv).

Usage

To use QSFit you need IDL (ver >= 8.1) and Gnuplot (ver. >=5.0) to be installed on your system.

Start an IDL session and change to the directory where you downloaded the QSFit files, e.g.:

CD, 'C:\the\path\where\qsfit\code\is\located'  ;  on windows

or

CD, '/the/path/where/qsfit/code/is/located'   ; on GNU/Linux

Code compilation

Before using QSFit you need to compile its code and and dependencies:

compile

The compile procedure and QSFit routines can not be used within the same procedure. Best practice is to call compile manually, at the very beginning of the IDL session.

There is no need to modify your IDL_PATH settings to use QSFit.

Download the spectra FITS files:

currently only SDSS DR10 FITS files are supported by QSFit. These files can be downloaded from: http://skyserver.sdss.org/dr10/en/tools/explore/obj.aspx

click, on Search, enter search cirteria and click on Go. Verify the resulting object is the one you're interested in and click on FITS (in the middle of the column on the left), and on Download.

Alternativaly you may copy and paste an URL similar to the following:

http://dr10.sdss3.org/sas/dr10/sdss/spectro/redux/26/spectra/0752/spec-0752-52251-0323.fits

directly on the web browser. The numbers to be tweaked are:

  • the plate ID (0752 in the example above. NOTE: this number appears twice in the URL!);
  • the MJD (52251 in the example above);
  • the fiber ID (0323 in the example above);

The QSFit package comes already with one such file in the data directory. This can be used to quickly test the QSFit procedures (see below).

Run QSFit analysis

Enter the following commands:

in = qsfit_input('data/spec-0752-52251-0323.fits', type='SDSS_DR10', z=0.3806, ebv=0.06846)
res = qsfit(in)

Parameters are:

  • the path to a SDSS-DR10 FITS file;
  • the redshift of the source;
  • the color excess;

The results are returned in the res variable.

If you want the results to be saved in a directory (which must have been previously created) use the OUTNAME keyword, e.g.

res = qsfit(in, outname='output/MySavedFile')

This will save all the logs in a file named output/MySavedFile_QSFIT.log and the results in output/MySavedFile_QSFIT.dat. The latter contains a binary representation of the results, and can be used to restore those data in IDL. Note that if you run again the above command the program will stop immediately with a message: File output/MySavedFile_QSFIT.dat already exists.

This is not an error, it simply says that the analysis has already been performed and there is no need to run it again. To actually re-run the analysis you should manually delete that file.

Plot the results:

qsfit_plot, res

To plot the data rebinned by a factor of 5:

res.gfit.obs.(0).plot.rebin = 5
qsfit_plot, res

To save the gnuplot files in a specific path:

qsfit_plot, res, filename='output/MySavedFile'

Note that the directory "output" must have been previously created and that the file name must not have any extension. Two files will be created: one with the .gp extension and one with the _resid.gp suffix. The former is the actual plot, to compare the data and the model. The latter is the plot of the residuals.

These are just gnuplot files, and IDL is not needed to handle them. You can simply load them into gnuplot for the gnuplot shell:

gnuplot> load 'output/MySavedFile.gp'

or modify them for to suite your goal. If you use a modern terminal for gnuplot, such as wxt, you may use the "Export to plot file" button to obtain a pdf copy.

Alternatively you may modify the plot files by changing the line:

set term wxt

with

set term pdf
set output 'filename.pdf'

and reload the file in gnuplot.

Monte Carlo resampling:

To run the Monte Carlo reampling method (see corresponding section in the reference paper) simply use the RESAMPLE= keyword in the qsfit call, e.g.:

res = qsfit(in, resample=10)

The res variable will be an array of 1000 elements (or whatver number you set with the RESAMPLE keyword) where the first is the actual analysis on the real data, while the remaining ones will be the analysis on the mock data generated from the best fit model of the actual analysis.

QSFit Customization:

Some details of QSFit behavior can be customized through the !QSFIT_OPT global variable. Check the qsfit_prepare_options procedure for a list of default values and their meaning.

For instance to disable the Balmer continuum component set:

!qsfit_opt.balmer = 0

Or to 1 to re/enable it

To add absorption lines at rest frame wavelengths 2705A and 3810A set:

!qsfit_opt.abslines_wavelengths = '2705, 3810'

To use a Lorentzian profile for the emission line (rather than a Gaussian one) set:

!qsfit_opt.lorentzian = 1

Further customization can be performed by modifying the source code in qsfit.pro. Check the documentation in the reference paper and in the code itself.

Fitting multiple datasets:

Starting from v1.3.0 QSFit allows to analyze multiple data sets simultaneously. E.g. to consider both the optical spectrum and photometry from SDSS:

in1 = qsfit_input('data/spec-0752-52251-0323.fits', type='SDSS_DR10', z=0.3806, ebv=0.06846)
in2 = qsfit_input('data/photometry.txt', type='ASCII', z=0.3806, ebv=0.06846)
res = qsfit(LIST(in1, in2))

Note that we used type='ASCII' to read the text file data/photometry.txt which contains ASCII data in three (space separated) columns: the wavelength (observer frame, in Angstrom), the observed flux and uncertainties (in units of 10-17 erg s^-1 cm^-1 A^-1).

Alternatively the data can be provided directly as numerical arrays (with the same units as above), e.g.:

in1 = qsfit_input('data/spec-0752-52251-0323.fits', type='SDSS_DR10', z=0.3806, ebv=0.06846)
wavelength = [3588.52,4862.24,6289.31,7712.08,9230.77]
flux = [14.6768,8.92081,6.38598,4.82714,4.85875]
uncert = [0.391382,0.148469,0.150169,0.0933146, 0.147523]
in2 = qsfit_input(wavelength, flux, uncert, z=0.3806, ebv=0.06846)
res = qsfit(LIST(in1, in2))

To plot the results we can assign individual labels and plotting options to each dataset, e.g.:

res.gfit.obs.(0).data.(0).plot.label = 'Spectrum'
res.gfit.obs.(0).data.(1).plot.label = 'Photometry'
res.gfit.obs.(0).data.(1).plot.gp = 'pt 7 ps 2 lt rgb "dark-green"'
qsfit_plot, res

qsfit's People

Contributors

gcalderone avatar

Stargazers

Felipe Avila avatar Navaneeth P K avatar Qian Yang avatar  avatar Yuming Fu avatar Orlando Vásquez avatar Zhang-Liang Xie avatar  avatar Mike Walmsley avatar Irham Taufik Andika avatar Zhu Fei-Fan avatar David Rupke avatar Chienting Chen avatar

Watchers

James Cloos avatar David Rupke avatar Luciano Nicastro avatar  avatar  avatar Felipe Avila avatar paper2code - bot avatar

qsfit's Issues

Compile Error

When I try to compile it shows @gcalderone
`IDL Version 8.5, Microsoft Windows (Win32 x86_64 m64).
(c) 2015, Exelis Visual Information Solutions, Inc., a subsidiary of Harris Corporation.
Installation number: .
Licensed for use by:

IDL> compile
% Compiled module: GCOMPILE.
% Compiled module: GCOMPILE_GLIB.
% Compiled module: GCOMPILE_QSFIT.
% Compiled module: COMPILE.
% GCOMPILE: File IDL/Contrib/ccm_unred.pro does not exists
% Execution halted at: COMPILE 45 C:\Users\User\Desktop\qsfit-1.0\qsfit-1.0\compile.pro
% $MAIN$ `

qsfit_plot issue

Dear G. Calderone,
I successfully installed the code on a kubuntu machine with idl 8.5 and gnuplot 5.2.
The fit on the provided test file runs perfectly and I can write the results.
The execution of the plotting line to look at the results gives a strange error, pasted below. I am wondering if it is a known issue.
Thank you for your help,
Best regards,
Johan

As you may see below the error involves tmp files, so the error is hard to track.
Executing the command with an output file does not solve the problem.

IDL> qsfit_plot, res, $filename='data/qsfit/spec-0752-52251-0323'
...
Fit results
#|FIT_STAT|TEST_STAT|TEST_DOF|TEST_PROB|MPFIT_STATUS|ELAPSED_TIME
0| 3914.3| 3914.3| 3248| 0| 0| 21.717
Total elapsed time: 21.717 seconds.
Log file: data/qsfit//spec-0752-52251-0323_QSFIT.log
Results saved in: data/qsfit//spec-0752-52251-0323_QSFIT.dat

IDL> qsfit_plot, res
Using galaxy template: SWIRE_ELL5
galaxy template flux density at 3650AA = 0.44222
galaxy template flux density at 4400AA = 0.78098
galaxy template flux density at 5100AA = 0.9311
galaxy template flux density at 5500AA = 1
galaxy template flux integrated in the range 3650AA +/- 340AA = 268.84
galaxy template flux integrated in the range 4400AA +/- 490AA = 780.31
galaxy template flux integrated in the range 5100AA +/- 50AA = 92.804
galaxy template flux integrated in the range 5500AA +/- 445AA = 849.11
Interpolation of high order Balmer lines template...
Interpolation of Vestergaard and Wilkes (2001) UV iron template...
Interpolation of Veron-Cetty (2004) optical iron template...
% Program caused arithmetic error: Floating underflow
IDL>
reset session
^
"/tmp/idl.gp.MbiffAhX", line 1: warning: expected optional axis name

                                                                                         reset session
                                                                                                            ^
                                                                                                             "/tmp/idl.gp.MbiffAhX", line 1: ';' expected


                             reset session
                                                ^
                                                 "/tmp/idl.gp.TbllqnPR", line 1: warning: expected optional axis name

                                                                                                                     reset session
            ^
             "/tmp/idl.gp.TbllqnPR", line 1: ';' expected

Reading JSON files

JSON formatted files should be accepted, and actually they should also be readable from socket/pipe stream.
Accepted set of key-value should match the accepted (and documented) FITS file keywords. The JSON file can deliver all the parameters, required and optional, to perform the fit, like z and ebv.
In case only 3 arrays are present, they are assumed to be the same as for ASCII files.

JSON_PARSE ref.: https://www.harrisgeospatial.com/docs/json_parse.html

Error while running

Hi,

I tried my first attempt to run qsfit on the spectrum file present inside data folder. I am using IDL 8.7 and have Gnuplot v5.0. I have a few doubts which I request you to clarify.

1- Do I need to compile every time I run the program?

2- Can I use qsfit to analyze SDSS-DR14 spectra?

3- I am able to compile without any error. Then I ran
res = qsfit('data/spec-0752-52251-0323.fits', z=0.3806, ebv=0.06846)

The program ran for sometime, generated a few tables, and finally threw the following error
% Program caused arithmetic error: Floating underflow

Being new to IDL, I then tried to plot using 'qsfit_plot, res'. It then gave the following output and nothing happened.

Using galaxy template: SWIRE_ELL5
galaxy template flux density at 3650AA = 0.44222
galaxy template flux density at 4400AA = 0.78098
galaxy template flux density at 5100AA = 0.9311
galaxy template flux density at 5500AA = 1
galaxy template flux integrated in the range 3650AA +/- 340AA = 268.84
galaxy template flux integrated in the range 4400AA +/- 490AA = 780.31
galaxy template flux integrated in the range 5100AA +/- 50AA = 92.804
galaxy template flux integrated in the range 5500AA +/- 445AA = 849.11
Interpolation of high order Balmer lines template...
Interpolation of Vestergaard and Wilkes (2001) UV iron template...
Interpolation of Veron-Cetty (2004) optical iron template...
% Program caused arithmetic error: Floating underflow

qsfit with GDL

I tried to compile Qsfit with my GDL installation. It is obviously not possible (?).
Sorry, keep in mind I cannot afford IDL.
Regards from German
Charlyms

Error about SWIRE

Thanks for your tools and it helps me a lot.
This is my second time to use this code. The code was running well on CentOS last time while have some trouble this time on macOS.
I run qsfit using IDL82 and Gnuplot v5.0 and got this error:

`IDL> in = qsfit_input('spec-1959-53440-0066.fits', type='SDSS_DR10', z=0.17806, ebv=0.0194)
IDL> res = qsfit(in)


QSFIT, ver. 2.0.0
started: Tue Oct 9 22:30:34 2018

ID:
Spectral resolution (min, max): 68.999, 69.163 km s^-1
Fraction of "good" channels: 99.451%
z=0.178059995174408
ebv=0.019400000572205

E(B-V) : 0.0194
Dereddening factors @ 1450, 3000, 5100 AA: 1.1639, 1.106, 1.0633

Cosmology:
#| H0|OMEGA_M|LAMBDA0
0| 70| 0.3| 0.7
Redshift: 0.17806
Lum. distance (Gpc): 0.86128

The line Lya has a coverage of 0.00000
The line NV_1241 has a coverage of 0.00000
The line OI_1306 has a coverage of 0.00000
The line CII_1335 has a coverage of 0.00000
The line SiIV_1400 has a coverage of 0.00000
The line CIV_1549 has a coverage of 0.00000
The line CIII_1909 has a coverage of 0.00000
The line CII has a coverage of 0.00000
The line MgII_2798 has a coverage of 0.00000
The line NeVI_3426 has a coverage of 0.910000
The line OII_3727 has a coverage of 0.910000
The line NeIII_3869 has a coverage of 0.910000
The line Hd has a coverage of 0.997500
The line Hg has a coverage of 0.997500
The line Hb has a coverage of 0.997500
The line OIII_4959 has a coverage of 0.910000
The line OIII_5007 has a coverage of 0.910000
The line HeI_5876 has a coverage of 0.997500
The line NII_6549 has a coverage of 0.980000
The line Ha has a coverage of 0.991667
The line NII_6583 has a coverage of 0.910000
The line SII_6716 has a coverage of 0.910000
The line SII_6731 has a coverage of 0.910000

Adding continuum component(s)...
Continuum
Galaxy
Using galaxy template: SWIRE_ELL5

=============================== GPRINT_ERROR ===============================
Object reference type required in this context: SWIRE (COM_QSFIT_COMP_GALAXYTEMPLATE).

ERROR_STATE:
NAME |BLOCK | CODE|SYS_CODE|SYS_CODE_TYPE|SYS_MSG|MSG_PREFIX
IDL_M_REQOBJREF |IDL_MBLK_CORE | -762| 0, 0| | |%

IDL call stack:
ROUTINE |FILENAME | LINE
QSFIT_COMP_GALAXYTEMPLATE_CDATA |/Users/wenke/IDLWorkspace82/qsfit-master/IDL/qsfit/components/qsfit_comp_galaxytemplate.pro | 126
GFIT_COMPILE |/Users/wenke/IDLWorkspace82/qsfit-master/IDL/Glib/gfit/gfit_compile.pro | 125
QSFIT_COMPILE |/Users/wenke/IDLWorkspace82/qsfit-master/IDL/qsfit/qsfit.pro | 662
QSFIT_ADD_CONTINUUM |/Users/wenke/IDLWorkspace82/qsfit-master/IDL/qsfit/qsfit.pro | 767
QSFIT_RUN |/Users/wenke/IDLWorkspace82/qsfit-master/IDL/qsfit/qsfit.pro | 2553
QSFIT |/Users/wenke/IDLWorkspace82/qsfit-master/IDL/qsfit/qsfit.pro | 3015
==============================================================================`

Even I tried release v1.2.4 and v1.2.0, this problem still occurred. I'm not quite sure if this is QSFit's problem or IDL's and I never meet error like this when I first use.
Hoping for your reply and thanks a lot.

Gaussian vs. Lorentzian line profile

Hello,

I understand that qsfit uses a Gaussian shape of the emission line to perform the fitting. Would it be possible to add a feature so that a user can consider a Lorentzian line profile to fit the broad emission lines? A few recent papers have reported that broad H_beta/H_alpha emission lines are better fitted with a Lorentzian than a Gaussian profile (e.g. Rakshit et al., 2017). Alternatively, the program can itself fit both type of profiles and can choose the best one. This may be done only for the broad lines.

I also want to know about using qsfit to analyze the optical spectrum above z=2. I tried once on a source and got the warning that 'no. of good channels are less than xxx%' and the program stopped.

Having a very limited understanding of IDL, I am unable to modify the program myself and look forward to your help in this regard.

Thanks in advance.

report some bugs

Many thanks for this wonderful tool!

I find some bugs in the new version 2.0, which are as follow:

  1. In the " ;; Calculate the sum of all QSFit components except "known" emission lines " part of Function qsfit_reduce,
    sum_wo_lines -= (out.expr.expr_broadlines - out.expr.expr_narrowlines)
    should be
    sum_wo_lines -= (out.expr.expr_broadlines + out.expr.expr_narrowlines)

  2. In Function qsfit_reduce_line, I suggest using the following code to deal with the associated line components:

`

;;If no association is made we just return the GFIT values
IF (gn(assoc) EQ 1) THEN BEGIN
   line.ncomp    = 1
   line.lum      = gfit.comp.(assoc).par.norm.val
   line.lum_err  = gfit.comp.(assoc).par.norm.err

   IF (gfit.comp.(assoc).par.norm.expr EQ '') THEN BEGIN
      line.lum      = gfit.comp.(assoc).par.norm.val
      line.lum_err  = gfit.comp.(assoc).par.norm.err
   ENDIF $
   ELSE BEGIN
      line.lum     = gfit.comp.(assoc).par.norm.actual
      line.lum_err = gfit.comp.(assoc).par.norm.err / gfit.comp.(assoc).par.norm.val * gfit.comp.(assoc).par.norm.actual
   ENDELSE

   IF (gfit.comp.(assoc).par.v_off.expr EQ '') THEN BEGIN
      line.voff     = gfit.comp.(assoc).par.v_off.val
      line.voff_err = gfit.comp.(assoc).par.v_off.err
   ENDIF $
   ELSE BEGIN
      line.voff     = gfit.comp.(assoc).par.v_off.actual
      line.voff_err = gfit.comp.(assoc).par.v_off.err
   ENDELSE

   IF (gfit.comp.(assoc).par.fwhm.expr EQ '') THEN BEGIN
      line.fwhm     = gfit.comp.(assoc).par.fwhm.val
      line.fwhm_err = gfit.comp.(assoc).par.fwhm.err
   ENDIF $
   ELSE BEGIN
      line.fwhm     = gfit.comp.(assoc).par.fwhm.actual
      line.fwhm_err = gfit.comp.(assoc).par.fwhm.err / gfit.comp.(assoc).par.fwhm.val * gfit.comp.(assoc).par.fwhm.actual
   ENDELSE

ENDIF $
ELSE BEGIN

`

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.