wtbarnes / hissw Goto Github PK
View Code? Open in Web Editor NEWIntegrate SSW into your Python workflows
Home Page: https://wtbarnes.github.io/hissw/
License: MIT License
Integrate SSW into your Python workflows
Home Page: https://wtbarnes.github.io/hissw/
License: MIT License
Add an example that shows how to use the CHIANTI IDL code to compute bremsstrahlung
If hissw
thinks IDL has thrown an error, it will raise a Python exception with the accompanying STDERR
output. However, sometimes this makes it hard to debug things as you might want to see the accompanying STDOUT
as well. There should be an option to stop hissw
from raising an exception that is turned off by default.
When inserting strings into a script, you need quotes around them. You can do this in the template itself or there could be a filter for doing this automatically. Additionally, there should be a filter for quoting every string in a list of strings.
It would be nice to have some filters for automatically converting arrays (or quantities) to lists in scripts.
Working on MacOS Sonoma 14.2.
I've tried installing, and can't get the pytest to run. Likely a user error, but I'm trying to follow the install directions . . .
(base) reep@Jeffreys-iMac hissw % pytest --version
pytest 7.4.0
Installed with pip:
(base) reep@Jeffreys-iMac hissw % pwd
/Users/reep/anaconda3/lib/python3.11/site-packages/hissw
(base) reep@Jeffreys-iMac hissw % ls
__init__.py environment.py read_config.py tests
__pycache__ filters.py templates util.py
The $HOME/.hissw
was not created automatically (directions sound like it should be?). I created it myself:
(base) reep@atrcw8 .hissw % ls
hisswrc
(base) reep@atrcw8 .hissw % pwd
/Users/reep/.hissw
Along with the hisswrc
file:
[hissw]
ssw_home=/Users/reep/ssw/
idl_home=/Applications/harris/idl89/
And then run pytest
in the directory /Users/reep/anaconda3/lib/python3.11/site-packages/hissw
, which returns 23 errors, all of which say: E fixture 'idl_home' not found
.
Manually trying to add idl_home
:
(base) reep@Jeffreys-iMac hissw % pytest --ssw_home="/Users/reep/ssw/" --idl_home="/Applications/harris/idl89/"
ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: unrecognized arguments: --ssw_home=/Users/reep/ssw/ --idl_home=/Applications/harris/idl89/
inifile: None
rootdir: /Users/reep/anaconda3/lib/python3.11/site-packages/hissw
I'm lost. What have I done wrong here?
The docstring for Environment
does not include the two new keyword arguments idl_only
(bool) and filters
(dict).
In Environment.custom_script
, I go to the trouble of rebuilding the Jinja environment just to load a template from a file. Instead, I should just read the file into a string and then pass to the from_string
method on the default environment.
Very minor, but there are some typos in the aia_get_response example - the logt variables (and interp_logt) should be logte (interp_logte)... or the variables passed into interpol() should have the 'e' removed.
Great package btw, I was very pleased to learn it existed!
Currently, STDOUT/STDERR are just printed to the screen and controlled by a verbose
kwarg. It would be better to control this with a logger and to also optionally log the scripts used to run the IDL code.
It is dead. Move to GH actions
There are still several incomplete examples in the docs. They should either be removed or finished.
Hey Will- this is a great package!
I'm having an issue, and its probably something I'm doing wrong. I'm trying to use the SSWIDL function spex_read_fit_results.pro
but I'm getting the following error. Any ideas?
>>> import hissw
>>> ssw = hissw.Environment(ssw_home="/usr/local/ssw/", idl_home='/Applications/harris/idl',
ssw_packages=['spex', 'xray'], ssw_paths=['hessi'])
>>> res = ssw.run('''spex_read_fit_results('/Users/laura.hayes/STIX_analysis/fermi_alex/ospex_results_10_dec_2021.fits')''')
~/opt/miniconda3/envs/fermi_alex/lib/python3.10/site-packages/hissw/environment.py in _check_for_errors(self, output, verbose)
147 # have to check it for certain keywords to see if an error occurred
148 if 'execution halted' in stderr.lower():
--> 149 raise SSWIDLError(stderr)
150 if 'failed to acquire license' in stderr.lower():
151 raise IDLLicenseError(stderr)
SSWIDLError: IDL 8.8.1 (darwin x86_64 m64).
(c) 2021, Harris Geospatial Solutions, Inc.
Licensed for use by: ESA/ESTEC - SITU
License: 9029
% Compiled module: SSW_PATH.
% Compiled module: SSW_INSTRUMENTS.
% Compiled module: WC_WHERE.
% Compiled module: PATHFIX.
% Compiled module: UNIQO.
% Compiled module: STRJUSTIFY.
% Compiled module: UNIQ.
% Compiled module: DEFAULT.
% Compiled module: PRSTR.
% SSW_PATH: Number of paths changed from 121 to 153
spex_read_fit_results('/Users/laura.hayes/STIX_analysis/fermi_alex/ospex_results_10_dec_2021.fits')
^
% Syntax error.
At: /var/folders/wm/gysdzbp10qv1kcm4zxb5q95xkjzv5q/T/tmp8hazv5z1/idl_procedure_20211210-124815.pro, Line 3
% 1 Compilation error(s) in module HISSW_PROCEDURE.
% Attempt to call undefined procedure: 'HISSW_PROCEDURE'.
% Execution halted at: $MAIN$
hissw was written and tested on macOS and Linux. No tests have been performed on Windows, but it appears that it does not seem to work there.
I actually can't get the functionality to work, but that's because I'm running a weird hybrid system with IDL running in windows and python running in a WSL linux environment. So it's a compatibility problem of my own creation.
It's not clear what changes would need to be made to the subprocess calls to get this working.
Add example in the docs that shows how to use the Hannah and Kontar DEM code.
Since 1.1, I've added another exception to catch IDL license errors that can sometimes make failures confusing. This should be included in a released version, 1.2
Lists of double (or extended) precision values (dtypes np.float64 or np.float128) used as inputs arrive in IDL as floats (type code 4, single precision). See simple example below, python code first followed by content of idl script.
import numpy as np
import hissw
#Make double precision + extended precision lists of random values, and verify data types
input_list = list(np.random.rand(10))
input_list_extended = list(np.random.rand(10).astype(np.longdouble))
print(type(input_list[0]))
print(type(input_list_extended[0]))
inputs = {'input_list': input_list, 'input_list_extended': input_list_extended}
ssw = hissw.Environment()
ssw_resp = ssw.run('/path/to/script/floattest.pro', args=inputs)
;IDL Script floattest.pro
print, 'List of random numbers which were double precision in Python:'
INPUTLIST = {{ input_list }}
print, INPUTLIST
print, 'Size of first entry: ', size(INPUTLIST[0])
print, 'List of random numbers which were extended precision in Python:'
INPUTLIST_EXTEND = {{ input_list_extended }}
print, INPUTLIST_EXTEND
print, 'Size of first entry: ',size(INPUTLIST_EXTEND[0])
In some cases, you may want to run the same action before and/or after all of your scripts. There should be an option for adding a header and footer script before and after the main script. These would be set when instantiating the environment. One possible use case for this is setting needed environment variables in your scripts.
Need to think about where the best place to do this is. These could be injected into the script itself or into the parent script.
Sometimes this is needed when specifying paths.
Add a GH action to automate release so that I don't have to keep doing this by hand. See https://github.com/sunpy/sunpy-soar/blob/9e657a25a0efa80ba7401f199f64b85001c6c3b7/.github/workflows/python-release.yml. Need to setup appropriate PyPI secrets as well.
Need a new release following nomenclature change (see #6)
This is dumb name that does not make a lot of sense. Rename to Env
or something more intuitive.
import hissw
ssw = hissw.Environment()
ssw.run('dindgen(10)')
/urs/local/ssw/gen/setup/setup.ssw: No such file or directory.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/sm/.virtualenvs/stixpy/lib/python3.11/site-packages/hissw/environment.py", line 186, in run
results = readsav(save_filename)
^^^^^^^^^^^^^^^^^^^^^^
File "/Users/sm/.virtualenvs/stixpy/lib/python3.11/site-packages/scipy/io/_idl.py", line 752, in readsav
f = open(file_name, 'rb')
^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/var/folders/h1/wy_cdhsj1c9_9pr0x9shjrcr0000gp/T/tmpdvs0lsx2/idl_vars_20240612-223603.sav'
hisswrc
[hissw]
ssw_home=/urs/local/ssw
idl_home=/Applications/NV5/idl/bin
hissw 2.3
Python 3.11.7
IDL 9.0.0 (darwin arm64 m64)
The following code snippet works fine in a terminal, but not a Jupyter notebook. Is there anything special about Jupyter that would prevent hissw from working properly?
>>> import hissw
>>> env = hissw.Environment(ssw_packages=['chianti'])
>>> script="""
... print,max_temp('Fe X')
... """
>>> env.run(script)
Error in Jupyter:
---------------------------------------------------------------------------
SSWIDLError Traceback (most recent call last)
Cell In[23], line 6
2 env = hissw.Environment(ssw_packages=['chianti'])
3 script="""
4 print,max_temp('Fe X')
5 """
----> 6 env.run(script)
File ~/anaconda3/lib/python3.11/site-packages/hissw/environment.py:185, in Environment.run(self, script, args, save_vars, verbose, **kwargs)
182 subprocess.call(['chmod', 'u+x', shell_filename])
183 cmd_output = subprocess.run([shell_filename], shell=True, stderr=subprocess.PIPE,
184 stdout=subprocess.PIPE)
--> 185 self._check_for_errors(cmd_output, verbose, **kwargs)
186 results = readsav(save_filename)
188 return results
File ~/anaconda3/lib/python3.11/site-packages/hissw/environment.py:200, in Environment._check_for_errors(self, output, verbose, **kwargs)
198 if kwargs.get('raise_exceptions', True):
199 if 'execution halted' in stderr.lower():
--> 200 raise SSWIDLError(stderr)
201 if 'failed to acquire license' in stderr.lower():
202 raise IDLLicenseError(stderr)
SSWIDLError: IDL 8.9.0 (darwin x86_64 m64).
(c) 2023, L3Harris Geospatial Solutions, Inc.
Licensed for use by: University of Hawaii (MAIN)
License: 828837
A new version is available: IDL 9.0
https://www.nv5geospatialsoftware.com/Product-Downloads/
% Program caused arithmetic error: Floating overflow
% Compiled module: USE_CHIANTI.
% Compiled module: DIR_EXIST.
% OPENR: Error opening file. Unit: 1
File: /Users/reep/ssw/packages/chianti/dbase/VERSION
No such file or directory
% Execution halted at: USE_CHIANTI 155
/Users/reep/ssw/packages/chianti/idl/setup/use_chianti.pro
% $MAIN$
% Compiled module: HISSW_PROCEDURE.
% Compiled module: MAX_TEMP.
% Compiled module: PICKFILE.
% Compiled module: READ_IONEQ.
The VERSION
file it's failing to read does in fact exist:
(base) reep@atrcw9 dbase % more /Users/reep/ssw/packages/chianti/dbase/VERSION
8.0.7
A hello-world with hissw does work in Jupyter:
import hissw
env = hissw.Environment()
script = """
print,'hello world!'
"""
env.run(script)
so this leads me to suspect that CHIANTI is not being properly instantiated in a Jupyter notebook. Have you encountered this?
Publish JOSS paper?
There may be times where you just want to call IDL code without an SSW installation. Currently, this isn't possible the startup script for the session calls sswidl
. It would be nice to optionally set this such that it does not require SSW to be installed, i.e. just call idl
instead of sswidl
.
The code snippets in the docs no longer have syntax highlighting. Not sure why...
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.