mcclearyj / cweb_psf

Out of date -- need to update!

cweb_psf

PSF characterization code repository for COSMOS-Web lensing working group. Still under active development! Comments and issues welcome.

environment.yml is @mcclearyj's Conda environment for this project; it may serve as an example for other users -- TO BE UPDATED

get_jwst_psf.py

Make SExtractor catalogs, star catalogs, and piff and psfex PSF models for COSMOS-Web observations in *i2d format. Code also adds an ERR cutout for the source into the star catalog.

Can call get_jwst_psf.py with:

# Astromatic & PIFF config files
export CONFIGDIR='/your/path/name/cweb_psf/astro_config'
export DATADIR='/your/path/name/data/April2023'
export CODEDIR='/your/path/name/cweb_psf'

get_jwst_psf.py $DATADIR/image1 [image2 ...] -config RUN_CONFIG [-outdir OUTDIR]
          -configdir [CONFIGDIR] [--overwrite] [--help] [--vb]

positional arguments:
  image1 [image2 ...]   Images to process (wildcards OK)

options:
  -h, --help            show this help message and exit
  -outdir OUTDIR        Where to save files [default: ./]
  -configdir CONFIGDIR  Location of SE/PSFEx/PIFF configs [default: astro_config]
  -config CONFIGFILE    Config file for running this script
  --vb                  Print detailed messages [does nothing for now]

Dependencies: numpy, re, os, astropy, matplotlib, pdb, glob, argparse, esutil

master_psf_diagnostic.py

Makes number of diagnostic figures and tables for input PSFEx and/or PIFF PSF models. Imports classes from psfmaker.py and starmaker.py in diagnostics

Options include:

• Test a PSFEx model [-psfex_model]
• Test a PSFEx model using esheldon psfex package: [--epsfex] (default)
• Test a PSFEx model using galsim rendering [--gpsfex] (needs -im_name too)
• Test a PIFF model [-piff_model]
• Test a PSFEx model with no spatial variation [-single_model]
• Test a WebbPSF model [-webb_model]

Can call master_psf_diagnostic.py with

master_psf_diagnostic.py basedir star_cat [-h] [-min_snr MIN_SNR]
    [-pix_scale PIX_SCALE] [-outdir OUTDIR] [-psfex_name PSFEX_NAME]
    [-im_name IM_NAME] [-piff_model PIFF_NAME] [--epsfex]
    [--gpsfex] [-single_model SIMPLE_PSFEX_MODEL]
    [-webb_model WEBBPSF_MODEL] [-vignet_size] [--verbose]

positional arguments:
  basedir               Directory containing star catalogs & images
  star_cat              Cleaned-up star catalog to use for PSF diagnostic

options:
  -h, --help            show this help message and exit
  -min_snr MIN_SNR      Optional S/N cut for star catalog [default=None]
  -pix_scale PIX_SCALE  Image/PSF pixel scale [default=0.03]
  -outdir OUTDIR        Output directory for diagnostics [default=./psf_diagnostics]
  -psfex_name PSFEX_NAME
                        PSFEx model filename
  -im_name IM_NAME      FITS image filename for GalSim PSFEx diagnostic
  -piff_model PIFF_NAME PIFF psf model filename
  -single_model PSFEX_MODEL
                        PSFEx model with no spatial variation
  -webb_model WEBBPSF_MODEL WebbPSF model
  -vignet_size          Sub-section of star & PSF stamps to consider
  --epsfex              Run esheldon psfex diagnostic (default)
  --gpsfex              Run galsim.des_psfex diagnostic
  --verbose, -v         Verbosity

Dependencies: numpy, re, os, astropy, matplotlib, pdb, argparse, galsim, galsim.des, piff, glob, treecorr, psfex

cweb_psf's Issues

Bad pixel sentinel values need to be configurable!

The sentinel value for bad pixels in make_starcat() is hardcoded:

    # Filter out saturated stars
    badstar_ext = 'ERR_VIGNET'; sentinel = 0
    #badstar_ext = 'DQ_VIGNET'; sentinel = [1, 2]

    selected_stars = _exclude_satpix(selected_stars,
                                     ext=badstar_ext,
                                     sentinel=sentinel
                                     )

This needs to be made a configurable value, since sentinel values differ for single exposures (cal files) and mosaics. Otherwise, the star selection is too restrictive for single exposures.

Os path join errors

The python debugger showcases the following error when running get_jwst_psf.py:

/home/eddieberman/research/mcclearygroup/cweb_psf/get_jwst_psf.py(122)run_sextractor()
-> cmd = ' '.join([
(Pdb) catname
*** NameError: name 'catname' is not defined
(Pdb) cat_name
'/research/mcclearygroup/mock_data/mosaics/COSMOS2020_sims//research/mcclearygroup/mock_data/mosaics/COSMOS2020_sims/mosaic_nircam_f115w_COSMOS-Web_cat.fits'
(Pdb) outdir
'~/research/mcclearygroup/mock_data/mosaics/COSMOS2020_sims'

Readme is very out of date

Codes are no longer the same, please fix @mcclearyj

Refactor PSF fitting & catalog-augmenting code

The PSF fitting and catalog making, filtering, and augmenting are all happening in an unholy loop in get_galaxy_cutouts.py. Many of the scripts in runners are also now deprecated.

The various functions in get_galaxy_cutouts should be separated into classes that can be imported by a runner script.

Webb PSF Plots

Lines 284-287 in src/psfmaker.py instantiate a single image Webb Model by accessing ['PSF_DATA']. This is housed in a try except block, however, both the initial attempt and exception failed to parse single['PSF_DATA']. In trying to unpack this I loaded a webbPSF model in an ipython session and got the same key error: KeyError: "Extension 'PSF_DATA' not found." The program still ran to completion.

Near term to-do list

To make this codebase more useful for the rest of the collaboration as an independent PSF fitting and diagnostic tool, the following tasks need to be completed. This list will be updated as we proceed and new scripts get added.

• Add capability to use external reference star catalog for PSF fitting in get_jwst_psf.py
• Add a decorator to PIFF models that adds the PSFEx functionality required by Sextractor++
• Run PSFEx model in get_jwst_psf.py
• Remove hard-coded image names!
• Turn this hard-coded star selection function into a configuration file that can be adapted for other data and passed to the PSF fitting code. Useful for cases where a reference catalog is not provided.
• Make PIFF auto-star selection work
• Possibly turn this loop into a class, especially since variables get passed back and forth between functions. Perhaps not strictly necessary, but would clean up code a bit.
• Push PSF data products to Candide for others to use
• Add Tobias Liaudat tools?

Unused arguments

In line 119 of src/psfmaker.py, psm_im is instantiated with make_single, supplying arguments for x and y with x_pos = x , y_pos = y. x_pos and y_pos are arguments for _make_single, however, x_pos and y_pos are not used within the make_single function itself so I'm not sure why they are arguments. In _make_pexim make_piff, x_pos and y_pos are used to specify the location of the psf model, so my main concern would be that make_single is supposed to make webb psfs that very across the field of view, but is instead displaying the same webb PSF for each (x,y) pair. If this is the case, a fix might be adding an nc.detector_position somewhere. If this is not the case, I think that the x_pos y_pos may be removed as arguments in line 271 and in line 119. In render_psf, the x and y arguments would be optional depending on the psf type.

Detector Name

WebbPSF expects detector names to be '5' and not 'LONG', for example 'NRCB5' instead of 'NRCBLONG'. Line 26 declares detector = imhead['DETECTOR'] which will cause an error for longer wavelengths.

This can be fixed with detector = imhead['DETECTOR'].replace('LONG', '5')