wtbarnes / hissw

Integrate SSW into your Python workflows

🐍 ☀️ hiss(w)

Easily integrate SSWIDL into your Python workflows. See the docs for more info.

Install

To install hissw,

$ pip install hissw

which will automatically install the package and its dependencies. hissw depends on the Jinja2 and scipy libraries. You can also install these manually with conda or from PyPI, i.e. pip install <package-name>. Additionally, you'll need a local install of IDL and the Solarsoft library.

Alternatively, you can install hissw from GitHub,

$ git clone https://github.com/wtbarnes/hissw.git
$ cd hissw && python -m pip install -e[all,test]

and then run the tests to make sure everything is working alright. Note that you must specify paths to SSW and IDL appropriately.

$ pytest --ssw-home=<path to top of SSW tree> --idl-home=<path to top level dir of IDL install>

Note: hissw relies on executing several shell commands. This has not been tested on Windows.

Citing hissw

If you use hissw in your work, please use the following citation,

@software{will_barnes_2022_6640421,
  author       = {Will Barnes and
                  Bin Chen},
  title        = {wtbarnes/hissw: v2.0},
  month        = jun,
  year         = 2022,
  publisher    = {Zenodo},
  version      = {v2.0},
  doi          = {10.5281/zenodo.6640421},
  url          = {https://doi.org/10.5281/zenodo.6640421}
}

Reporting Issues and Contributing

Open an issue on GitHub to report a problem. Pull requests welcome.

hissw's Issues

Add ability to call IDL without SSW

There may be times where you just want to call IDL code without an SSW installation. Currently, this isn't possible the startup script for the session calls sswidl. It would be nice to optionally set this such that it does not require SSW to be installed, i.e. just call idl instead of sswidl.

Expose SSW_HOME variable in script template.

Sometimes this is needed when specifying paths.

Add DEM example

Add example in the docs that shows how to use the Hannah and Kontar DEM code.

Simplify release process

Add a GH action to automate release so that I don't have to keep doing this by hand. See https://github.com/sunpy/sunpy-soar/blob/9e657a25a0efa80ba7401f199f64b85001c6c3b7/.github/workflows/python-release.yml. Need to setup appropriate PyPI secrets as well.

v1.1 Release

Need a new release following nomenclature change (see #6)

1.2 release

Since 1.1, I've added another exception to catch IDL license errors that can sometimes make failures confusing. This should be included in a released version, 1.2

Add filter for strings

When inserting strings into a script, you need quotes around them. You can do this in the template itself or there could be a filter for doing this automatically. Additionally, there should be a filter for quoting every string in a list of strings.

Rename ScriptMaker object

This is dumb name that does not make a lot of sense. Rename to Env or something more intuitive.

Logging

Currently, STDOUT/STDERR are just printed to the screen and controlled by a verbose kwarg. It would be better to control this with a logger and to also optionally log the scripts used to run the IDL code.

hissw + CHIANTI not working in Jupyter notebook?

The following code snippet works fine in a terminal, but not a Jupyter notebook. Is there anything special about Jupyter that would prevent hissw from working properly?

>>> import hissw
>>> env = hissw.Environment(ssw_packages=['chianti'])
>>> script="""
... print,max_temp('Fe X')
... """
>>> env.run(script)

Error in Jupyter:

---------------------------------------------------------------------------
SSWIDLError                               Traceback (most recent call last)
Cell In[23], line 6
      2 env = hissw.Environment(ssw_packages=['chianti'])
      3 script="""
      4 print,max_temp('Fe X')
      5 """
----> 6 env.run(script)

File ~/anaconda3/lib/python3.11/site-packages/hissw/environment.py:185, in Environment.run(self, script, args, save_vars, verbose, **kwargs)
    182     subprocess.call(['chmod', 'u+x', shell_filename])
    183     cmd_output = subprocess.run([shell_filename], shell=True, stderr=subprocess.PIPE,
    184                                 stdout=subprocess.PIPE)
--> 185     self._check_for_errors(cmd_output, verbose, **kwargs)
    186     results = readsav(save_filename)
    188 return results

File ~/anaconda3/lib/python3.11/site-packages/hissw/environment.py:200, in Environment._check_for_errors(self, output, verbose, **kwargs)
    198 if kwargs.get('raise_exceptions', True):
    199     if 'execution halted' in stderr.lower():
--> 200         raise SSWIDLError(stderr)
    201     if 'failed to acquire license' in stderr.lower():
    202         raise IDLLicenseError(stderr)

SSWIDLError: IDL 8.9.0 (darwin x86_64 m64).
(c) 2023, L3Harris Geospatial Solutions, Inc.

Licensed for use by: University of Hawaii (MAIN)
License: 828837
A new version is available: IDL 9.0
https://www.nv5geospatialsoftware.com/Product-Downloads/
 
% Program caused arithmetic error: Floating overflow
% Compiled module: USE_CHIANTI.
% Compiled module: DIR_EXIST.
% OPENR: Error opening file. Unit: 1
         File: /Users/reep/ssw/packages/chianti/dbase/VERSION
  No such file or directory
% Execution halted at: USE_CHIANTI       155
   /Users/reep/ssw/packages/chianti/idl/setup/use_chianti.pro
%                      $MAIN$          
% Compiled module: HISSW_PROCEDURE.
% Compiled module: MAX_TEMP.
% Compiled module: PICKFILE.
% Compiled module: READ_IONEQ.

The VERSION file it's failing to read does in fact exist:

(base) reep@atrcw9 dbase % more /Users/reep/ssw/packages/chianti/dbase/VERSION
8.0.7

A hello-world with hissw does work in Jupyter:

import hissw
env = hissw.Environment()
script = """
print,'hello world!'
"""
env.run(script)

so this leads me to suspect that CHIANTI is not being properly instantiated in a Jupyter notebook. Have you encountered this?

Install issues on Mac

Working on MacOS Sonoma 14.2.

I've tried installing, and can't get the pytest to run. Likely a user error, but I'm trying to follow the install directions . . .

(base) reep@Jeffreys-iMac hissw % pytest --version
pytest 7.4.0

Installed with pip:

(base) reep@Jeffreys-iMac hissw % pwd
/Users/reep/anaconda3/lib/python3.11/site-packages/hissw
(base) reep@Jeffreys-iMac hissw % ls
__init__.py	environment.py	read_config.py	tests
__pycache__	filters.py	templates	util.py

The $HOME/.hissw was not created automatically (directions sound like it should be?). I created it myself:

(base) reep@atrcw8 .hissw % ls
hisswrc
(base) reep@atrcw8 .hissw % pwd
/Users/reep/.hissw

Along with the hisswrc file:

[hissw]
ssw_home=/Users/reep/ssw/
idl_home=/Applications/harris/idl89/

And then run pytest in the directory /Users/reep/anaconda3/lib/python3.11/site-packages/hissw, which returns 23 errors, all of which say: E fixture 'idl_home' not found.

Manually trying to add idl_home:

(base) reep@Jeffreys-iMac hissw % pytest --ssw_home="/Users/reep/ssw/" --idl_home="/Applications/harris/idl89/"
ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: unrecognized arguments: --ssw_home=/Users/reep/ssw/ --idl_home=/Applications/harris/idl89/
  inifile: None
  rootdir: /Users/reep/anaconda3/lib/python3.11/site-packages/hissw

I'm lost. What have I done wrong here?

Header and footer in script

In some cases, you may want to run the same action before and/or after all of your scripts. There should be an option for adding a header and footer script before and after the main script. These would be set when instantiating the environment. One possible use case for this is setting needed environment variables in your scripts.

Need to think about where the best place to do this is. These could be injected into the script itself or into the parent script.

JOSS Paper

Publish JOSS paper?

having issues getting hissw to work

Hey Will- this is a great package!

I'm having an issue, and its probably something I'm doing wrong. I'm trying to use the SSWIDL function spex_read_fit_results.pro but I'm getting the following error. Any ideas?

>>> import hissw

>>> ssw = hissw.Environment(ssw_home="/usr/local/ssw/", idl_home='/Applications/harris/idl', 
	                    ssw_packages=['spex', 'xray'], ssw_paths=['hessi'])

>>> res = ssw.run('''spex_read_fit_results('/Users/laura.hayes/STIX_analysis/fermi_alex/ospex_results_10_dec_2021.fits')''')

~/opt/miniconda3/envs/fermi_alex/lib/python3.10/site-packages/hissw/environment.py in _check_for_errors(self, output, verbose)
    147         # have to check it for certain keywords to see if an error occurred
    148         if 'execution halted' in stderr.lower():
--> 149             raise SSWIDLError(stderr)
    150         if 'failed to acquire license' in stderr.lower():
    151             raise IDLLicenseError(stderr)

SSWIDLError: IDL 8.8.1 (darwin x86_64 m64).
(c) 2021, Harris Geospatial Solutions, Inc.

Licensed for use by: ESA/ESTEC - SITU
License: 9029
% Compiled module: SSW_PATH.
% Compiled module: SSW_INSTRUMENTS.
% Compiled module: WC_WHERE.
% Compiled module: PATHFIX.
% Compiled module: UNIQO.
% Compiled module: STRJUSTIFY.
% Compiled module: UNIQ.
% Compiled module: DEFAULT.
% Compiled module: PRSTR.
% SSW_PATH: Number of paths changed from 121 to 153

    spex_read_fit_results('/Users/laura.hayes/STIX_analysis/fermi_alex/ospex_results_10_dec_2021.fits')
     ^
% Syntax error.
  At: /var/folders/wm/gysdzbp10qv1kcm4zxb5q95xkjzv5q/T/tmp8hazv5z1/idl_procedure_20211210-124815.pro, Line 3
% 1 Compilation error(s) in module HISSW_PROCEDURE.
% Attempt to call undefined procedure: 'HISSW_PROCEDURE'.
% Execution halted at: $MAIN$          

Replace PyPI badge

Add option for suppressing exceptions

If hissw thinks IDL has thrown an error, it will raise a Python exception with the accompanying STDERR output. However, sometimes this makes it hard to debug things as you might want to see the accompanying STDOUT as well. There should be an option to stop hissw from raising an exception that is turned off by default.

typos in aia example

Very minor, but there are some typos in the aia_get_response example - the logt variables (and interp_logt) should be logte (interp_logte)... or the variables passed into interpol() should have the 'e' removed.

Great package btw, I was very pleased to learn it existed!

Docstring for `Environment` is incomplete

The docstring for Environment does not include the two new keyword arguments idl_only (bool) and filters (dict).

Add CHIANTI continuum example

Add an example that shows how to use the CHIANTI IDL code to compute bremsstrahlung

Double-precision inputs from python become IDL floats (single-precision)

Lists of double (or extended) precision values (dtypes np.float64 or np.float128) used as inputs arrive in IDL as floats (type code 4, single precision). See simple example below, python code first followed by content of idl script.

import numpy as np
import hissw

#Make double precision + extended precision lists of random values, and verify data types
input_list = list(np.random.rand(10))
input_list_extended = list(np.random.rand(10).astype(np.longdouble))
print(type(input_list[0]))
print(type(input_list_extended[0]))

inputs = {'input_list': input_list, 'input_list_extended': input_list_extended}
ssw = hissw.Environment()
ssw_resp = ssw.run('/path/to/script/floattest.pro', args=inputs)

;IDL Script floattest.pro

print, 'List of random numbers which were double precision in Python:'
INPUTLIST = {{ input_list }}
print, INPUTLIST 
print, 'Size of first entry: ', size(INPUTLIST[0])

print, 'List of random numbers which were extended precision in Python:'
INPUTLIST_EXTEND = {{ input_list_extended }}
print, INPUTLIST_EXTEND
print, 'Size of first entry: ',size(INPUTLIST_EXTEND[0])

Simplify script loading from file

In Environment.custom_script, I go to the trouble of rebuilding the Jinja environment just to load a template from a file. Instead, I should just read the file into a string and then pass to the from_string method on the default environment.

Finish examples in docs

There are still several incomplete examples in the docs. They should either be removed or finished.

Move off of Travis

It is dead. Move to GH actions

Syntax highlighting in docs no longer works

The code snippets in the docs no longer have syntax highlighting. Not sure why...

Windows Compatibility

hissw was written and tested on macOS and Linux. No tests have been performed on Windows, but it appears that it does not seem to work there.

As noted by @schonfsj in #7,

I actually can't get the functionality to work, but that's because I'm running a weird hybrid system with IDL running in windows and python running in a WSL linux environment. So it's a compatibility problem of my own creation.

It's not clear what changes would need to be made to the subprocess calls to get this working.

Additional filters

It would be nice to have some filters for automatically converting arrays (or quantities) to lists in scripts.