iraf-community / pyraf Goto Github PK
View Code? Open in Web Editor NEWCommand language for IRAF based on Python.
Home Page: https://iraf-community.github.io/pyraf.html
License: Other
Command language for IRAF based on Python.
Home Page: https://iraf-community.github.io/pyraf.html
License: Other
The <skipped>
must have been an accidental copy/paste on my part.
This would have never passed under Python 3.6 anyway, so we need to sanitize python ver =
a little further under Python 3.
tests/test_describe.py:29: AssertionError
________________________________ test_dumpspecs ________________________________
@pytest.mark.skipif(not HAS_IRAF, reason='Need IRAF to run')
def test_dumpspecs():
# get dumpspecs output
out_f = six.StringIO()
wutil.dumpspecs(outstream=out_f, skip_volatiles=True)
out_str = out_f.getvalue()
out_f.close()
# modify out_str to remove a path that will always be changing
out_str = '\n'.join([l for l in out_str.split('\n')
if 'python exec =' not in l])
# modify out_str to handle old versions which printed Tkinter as camel-case
out_str = out_str.replace('Tkinter', 'tkinter')
# verify it (is version dependent)
key = ('2' if IS_PY2 else '3', sys.platform.replace('2', ''))
expected = REF[key]
> assert expected.strip() == out_str.strip(), \
'Unexpected output from wutil.dumpspecs: {}'.format(out_str)
E AssertionError: Unexpected output from wutil.dumpspecs: python ver = 3.6
E platform = linux
E PY3K = True
E c.OF_GRAPHICS = False
E /dev/console owner = <skipped>
E tkinter use unattempted.
E
E assert 'python ver =... unattempted.' == 'python ver = ... unattempted.'
E - python ver = 3<skipped>
E + python ver = 3.6
E platform = linux
E PY3K = True
E c.OF_GRAPHICS = False
E /dev/console owner = <skipped>
E tkinter use unattempted.
tests/test_graphics.py:194: AssertionError```
~/Downloads/iraf-2.16.1-2021.06.14
mkiraf
and then pyraf
~/.iraf-source
export PATH=/home/firestar/.iraf/bin/:$PATH
mkiraf
and then pyraf
, it said no directories found: ~/Downloads/iraf-2.16.1-2021.06.14/unix/hlib/zzsetenv.def
PS: epar
fonts are still small
These are the some notes from @jhunkeler from his work on #45. Investigation is needed whether they are real bugs or just the tests need to be re-designed.
File | Test name | Note | Status |
---|---|---|---|
test_core_nongraphics.py | test_irafparlist_getName | BUG: Returns file path instead of name | Fixed in #76 |
test_core_nongraphics.py | test_irafparlist_getParList | BUG: raises 'TypeError: number coercion failed' | Fixed in #76 |
test_core_nongraphics.py | test_irafparlist_setParam_float | BUG: TypeError: cannot concatenate 'str' and 'float' objects | Fixed in #76 |
Fixed in #51 | |||
test_core_nongraphics.py | test_irafparlist_incompatible_assignment_raises | Can overwrite string type with uncast integer | |
test_graphics.py | setup_module | Does not work under Darwin, because c.OF_GRAPHICS is True | |
test_graphics.py | test_gki_prow_to_different_devices | Erroneously sends jobs to network printers | |
test_invocation.py | test_invoke_command_direct | BUG: Why is there a single newline on stderr? |
When importing pyraf from within pytest, I get the following error on Python 3:
==================================== ERRORS ====================================
_______________________ ERROR collecting debian/tests.py _______________________
tests.py:8: in <module>
from pyraf import iraf
/usr/lib/python3/dist-packages/pyraf/__init__.py:126: in <module>
iraf.Init(doprint=0, hush=1)
/usr/lib/python3/dist-packages/pyraf/iraffunctions.py:257: in Init
userpkg.run(_doprint=0, _hush=hush, _save=1)
/usr/lib/python3/dist-packages/pyraf/iraftask.py:359: in run
self._run(redirKW, specialKW)
/usr/lib/python3/dist-packages/pyraf/iraftask.py:1667: in _run
???
/usr/lib/python3/dist-packages/pyraf/iraftask.py:1470: in _runCode
???
<CL script clpackage.user>:28: in login
Pipe1 = iraf.cl(Stdin=Pipe2, Stdout=1)
/usr/lib/python3/dist-packages/pyraf/iraftask.py:767: in __call__
return self.run(*args, **kw)
/usr/lib/python3/dist-packages/pyraf/iraftask.py:359: in run
self._run(redirKW, specialKW)
/usr/lib/python3/dist-packages/pyraf/iraftask.py:1202: in _run
self._pyFunction(*pl)
/usr/lib/python3/dist-packages/pyraf/iraffunctions.py:2485: in _clProcedure
clExecute(_sys.stdin.read(), locals=theLocals, Stdin=_sys.__stdin__)
/usr/lib/python3/dist-packages/pyraf/iraffunctions.py:3088: in clExecute
exec(pycode.vars.proc_name+"(taskObj=iraf.cl)", locals)
<string>:1: in <module>
???
<CL script CL1>:12: in string_proc
???
/usr/lib/python3/dist-packages/pyraf/iraffunctions.py:2088: in clOscmd
status = _subproc.subshellRedir(s, shell=shell)
/usr/lib/python3/dist-packages/pyraf/subproc.py:736: in subshellRedir
return systemRedir((shell, "-c", cmd))
/usr/lib/python3/dist-packages/pyraf/subproc.py:718: in systemRedir
process.run()
/usr/lib/python3/dist-packages/pyraf/subproc.py:790: in run
sys.stderr.write(tostr(s))
/usr/lib/python3/dist-packages/stsci/tools/for2to3.py:82: in tostr
return s.decode(encoding)
E UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 56: ordinal not in range(128)
!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!
=========================== 1 error in 2.74 seconds ============================
On Python 2, it also does not work (#43) works well.
Hello everyone --
I've just installed the community iraf and pyraf on a Fedora-33 system to be sure that it will work correctly on newer linuces where 32-bit binaries are no longer supported.
One minor detail -- on older systems I have been setting the environment variable PYRAFGRAPHICS to 'matplotlib', which leads to somewhat nicer graphics rendering. This now fails with the message:
/usr/local/lib64/python3.9/site-packages/pyraf-2.1.15-py3.9-linux-x86_64.egg/pyraf/MplCanvasAdapter.py:21:
MatplotlibDeprecationWarning:
The 'resize_callback' parameter of __init__() was deprecated in Matplotlib 3.4 and will be removed two minor
releases later. Use get_tk_widget().bind('<Configure>', ..., True) instead. If any parameter follows
'resize_callback', they should be passed as keyword, not positionally.
tkagg.FigureCanvasTkAgg.__init__(self, figure, master,
Traceback (innermost last):
File "<CL script CL1>", line 1, in <module>
AttributeError: 'MplCanvasAdapter' object has no attribute 'show'
I'm running matplotlib version 3.4.2, with python 3.9.6.
Any ideas? If this is an easy fix, it'd be nice to have.
Thanks -- John
c/c @cdsontag
As reported by @olebole below:
When compiling for Python 3 (3.6), ... the C modules (sscanf
and xutil
) don't work "out of the box".
Pragmatically, I could resolve that by renaming them in setup.cfg
:
-[extension=pyraf.sscanfmodule]
+[extension=pyraf.sscanf]
sources = src/sscanfmodule.c
optional = True
fail_message = If this is Windows, it is ok.
-[extension=pyraf.xutilmodule]
+[extension=pyraf.xutil]
...
but I am not sure whether this is the best solution. Any idea?
Can't find the executable:
--> sobsolete
Traceback (innermost last):
File "<CL script CL1>", line 1, in <module>
IrafError: Cannot find executable for task sobsolete
This seems to happen as the Python interpreter is shutting down, after being started without an existing cache. Apparently things are getting cleaned up in the wrong order or something. It's not a disaster for interactive use, but causes our regression tests to fail without producing any results. I started looking into this but hadn't got to the bottom of the problem and had better document it in the meantime.
> rm -fr pyraf
> pyraf
Created directory /home/jturner/test/pyraf for cache
setting terminal type to xterm...
NOAO/IRAF PC-IRAF Revision 2.16 EXPORT Thu May 24 15:41:17 MST 2012
This is the EXPORT version of IRAF V2.16 supporting PC systems.
Welcome to IRAF. To list the available commands, type ? or ??. To get
detailed information about a command, type `help <command>'. To run a
command or load a package, type its name. Type `bye' to exit a
package, or `logout' to get out of the CL. Type `news' to find out
what is new in the version of the system you are using.
Visit http://iraf.net if you have questions or to report problems.
The following commands or packages are currently defined:
(Updated on 2013-12-13)
clpackage/:
adccdrom/ esowfi/ mem0/ rvsao/ user/
cfh12k/ finder/ mscdb/ softools/ utilities/
cirred/ fitsutil/ mscred/ sqiid/ vo/
clpackage/ gemini/ mtools/ stecf/ xdimsum/
ctio/ gmisc/ nfextern/ stsdas/ xray/
cutoutpkg/ guiapps/ noao/ system/
dataio/ images/ obsolete/ tables/
dbms/ language/ plot/ ucsclris/
deitab/ lists/ proto/ upsqiid/
PyRAF 2.2.dev Copyright (c) 2002 AURA
Python 3.6.2 Copyright (c) 2001-2017 Python Software Foundation.
Python/CL command line wrapper
.help describes executive commands
--> .exit
Exception ignored in: <bound method _CodeCache.__del__ of <pyraf.clcache._CodeCache object at 0x7f8a7f730860>>
Traceback (most recent call last):
File "/home/jturner/anaconda2/envs/test3/lib/python3.6/site-packages/pyraf/clcache.py", line 193, in __del__
File "/home/jturner/anaconda2/envs/test3/lib/python3.6/site-packages/pyraf/clcache.py", line 184, in close
File "/home/jturner/anaconda2/envs/test3/lib/python3.6/shelve.py", line 144, in close
File "/home/jturner/anaconda2/envs/test3/lib/python3.6/shelve.py", line 172, in sync
File "/home/jturner/anaconda2/envs/test3/lib/python3.6/dbm/dumb.py", line 121, in _commit
TypeError: 'NoneType' object is not callable
Exception ignored in: <bound method _Database.close of <dbm.dumb._Database object at 0x7f8a7f7309e8>>
Traceback (most recent call last):
File "/home/jturner/anaconda2/envs/test3/lib/python3.6/dbm/dumb.py", line 279, in close
File "/home/jturner/anaconda2/envs/test3/lib/python3.6/dbm/dumb.py", line 121, in _commit
TypeError: 'NoneType' object is not callable
I have installed iraf/pyraf with the astroconda python 2.7 conda environment.
Pyraf works well within this environment but I have been searching ways to run pyraf tasks with my own scripts in supported python versions.
I have been succesfull launching pyraf tasks in a python 3.7 using the subprocess library and activating the pyraf environement. I am using ubuntu 20.04 and the code looks something like:
bash_command = "bash -c 'source /home/user/anaconda3/bin/activate && conda activate iraf27 && python RunIraf_task.py task_conf_file.txt /home/user/data/folder/ && conda deactivate'"
p1 = subprocess.run(bash_command, shell=True, capture_output=True)
This works very well for most tasks and it makes possible to keep the pyraf installation isolated. However, I noticed an issue in the identify task: While trying to define a line (m key) the plot window becomes kind of frozen. This does not happen in the fit window (f key) which works as expected.
I guess this is because the task wants to display some information in the terminal or is waiting for some user input. I wonder if anyone has a workaround this issue or if they would suggest some strategy to run pyraf accross python environments.
Thank you for your work
If I use epar ccdproc
in imred >> ccdred >> ccdproc
, I can put [261:280,1:1032]
in biassec
, which is in a old iraf code by my teacher:
ccdproc @list.all o//@list.all ccdtype='' overscan+ biassec=[261:280,1:1032]
But if I use the terminal, it will say:
SyntaxError: Too many positional parameters for task ccdproc
And if I put (261:280,1:1032)
, (261:280;1:1032)
or (261:280 1:1032)
, it will also pop out SyntaxError
. It seems that pyraf syntax is slightly different from iraf. What is the right parameter syntax?
'conda install pyraf' installs stsci.tools-4.0.1 as well.
'from pyraf import iraf' , however, yields an error of
"ModuleNotFoundError: No module named 'stsci.tools.for2to3'"
Downgrading of stsci.tools to the version 3.6.0 by 'conda install stsci.tools==3.6.0' was needed to execute 'from pyraf import iraf' successfully.
conda version : 4.10.3
python version : 3.7.4.final.0
platform : osx-64 (OSX/10.15.7)
Thanks
Follow up of #45 -- Setup Jenkins nightly cron. We can skip cron if there has been a run on master in the past 24 hours (e.g., from push or merge commit).
First, I want to thank you for your incredible work on revitalizing IRAF. I was able to install IRAF (from Github) on a macOS Monterey machine (run_tests only showed 2 failed and 1 skipped, so remarkably good).
I installed XQuartz to run xterm and verify if most of the tasks I use (noao onedspec/twodspec, etc) work fine (they do but there's s a minor issue with some printed messages in APALL). I also installed X11iraf flawlessly (although I'm not using xgterm right now). Then, I installed the latest PyRAF version from this GitHub repository in order to run a Python code that I modified (not mine; it uses STSDAS, but I was not able to install that external package from their Github [by their words, it is nonfunctional]); imcopy, imarith and the tracing/extraction of the spectra in APALL (a test) work fine in PyRAF.
The only problem that I currently have is that the first frame of the interactive plot in APALL (the plot to define the apertures) doesn't behave correctly: I attach a short video (it is just an example) showing the problem: the first frame only allows certain keys to function: in this case, I used 'l', which worked; after that, I tried 'u', but the "cursor" got stuck at the center of the plot: it may "escape" from that position briefly, but returns almost immediately. The same happens for 'a', 'n', 'm' keys. The interesting part is that 'b' always works, so the plot to fit the background pops out and the fitting of the background ('f') runs OK. I can return to the first frame ('q'), but the problem still persists. 'b', 'f' and 'q' worked again in the second try.
So, I've been reading the Python codes of PyRAF, and I could figure out, for example, that to stop the blinking of the cursor, we can comment out line "hideTkCursor(self)" of the function "activateSWCursor" in "Ptkplot.py" file. About the major problem, I haven't solved it yet although I think I located the files related with that. This is my question: as I'm a macOS newbie, I don't know if someone ever faced this kind of behaviour in an older macOS or linux distributions (I remember I had a similar issue with CentOS and PyRAF many years ago but I don't remember the solution). Any help would be great!
Thanks.
PyRAF is packaged for Python 3 by running 2to3
at build time (confirmed by @cdsontag). This translation produces a clPrint
function that does not emulate CL accurately, causing at least one of our Gemini IRAF CL scripts to fail. Specifically, the Python 3 clPrint
adds one extra space at the end of the line and one space in between string arguments, which CL does not. This is not a cosmetic issue, because it's quite common to manipulate text data in CL by piping a string from print
into commands like scan
or translit
or redirecting it to a file.
Strictly speaking, the PyRAF source is not at fault here, but 1) that's a bit academic when it results in a broken build and 2) solving the problem here would avoid maintaining patches downstream. I already have a candidate fix for this, but best add this issue with a bit of explanation...
Link to results for master
is available at https://ssbjenkins.stsci.edu/job/STScI/job/pyraf/job/master/ (this link is also reachable by clicking the "build status" badge on README).
On left menu, click on "Open Blue Ocean".
Select row pointing to "master" branch.
Click on the Python 2 build as shown
To make sure, you will now see a line like conda install -y -q python=2.7 iraf-all six
on that page below the diagram.
Expand the line with with_env pytest tests --basetemp=tests_output --junitxml results.xml
to see Python 2 test results. Example:
[STScI_pyraf_master-52EC45PPOTV4YGTS2XRZCI6UCVSWOPR23BCYCLQME64HJ7JJ445Q] Running shell script
+ with_env pytest tests --basetemp=tests_output --junitxml results.xml
============================= test session starts ==============================
platform linux2 -- Python 2.7.14, pytest-3.5.0, py-1.5.3, pluggy-0.6.0
rootdir: /home/jenkins/workspace/STScI_pyraf_master-xxx, inifile: setup.cfg
collected 76 items
tests/test_basic.py .... [ 5%]
tests/test_clcache.py . [ 6%]
tests/test_cli.py .... [ 11%]
tests/test_core_nongraphics.py .......x........x...xx.. [ 43%]
tests/test_describe.py . [ 44%]
tests/test_graphics.py ..........s [ 59%]
tests/test_invocation.py ........................ [ 90%]
tests/test_using_tasks.py ....... [100%]
generated xml file: /home/jenkins/workspace/STScI_pyraf_master-xxx/results.xml
=============== 71 passed, 1 skipped, 4 xfailed in 21.12 seconds ===============
c/c @cdsontag
PY in the sky. Har har har...
Hi, could you please provide another link for the release notes and tutorials etc. ? because the original website (http://www.stsci.edu/resources/software_hardware/pyraf) is now under construction. That would be very helpful!
As requested in #51 (comment) but after #53 is merged.
After the recent work to get all our Gemini IRAF tests running, I've had a "quick" look at the differences in results compared with PyRAF 2.1.15 on Python 2.7, suspecting that most of the 189/932 failures may be caused by just a couple of small issues, which does indeed seem to be the case.
The first thing is that iraffunctions.nint()
follows the new behaviour of round()
in Python 3, rounding half-integers to the nearest even number instead of up. So eg. CL and PyRAF on Python 2 both give nint(2.5) == 3
, whereas PyRAF on Python 3 gives 2
.
The other, less well-documented thing is that Python 3 float.__str__()
produces 17-digit precision (like repr
), rather than 12 digits in Python 2. Part of the issue there is just our tests being overly sensitive/brittle. However, it can lead to some odd consequences, such as comments getting truncated where there's less space or imexpr
switching to double precision output when given extra digits. It can also produce somewhat larger differences where the outcome of pixel rejection differs here and there, as well as uglier user output in some cases.
So which is the "correct" behaviour? I think the baseline intention here is to emulate CL. For example, division has already been made to follow the CL & Python 2 behaviour on Python 3: 1/2 == 0
. I think nint()
is analogous to that and a lot of CL code has been written assuming that it always rounds upwards. At a glance, I can see some cases where breaking that assumption causes arcane bugs, eg. a large imaging WCS error due to the WCS getting copied from the wrong input FITS extension, because the reference extension number is now getting calculated incorrectly. "Fixing" nint()
is a 2-line patch that I think would be worth including, for compatibility with CL (including the ability to write scripts that work with both cl
and pyraf
) and to avoid unnecessary work.
The argument for trying to emulate the precision of CL's real()
is less obvious though. First, it's a bit of an uphill struggle to override a built-in type, with Python's native precision likely to crop up wherever any function fails to override it. Second, it seems to be coincidental that Python 2's float()
behaves so much like IRAF's real()
, printing 12 significant digits, switching to scientific notation at the same threshold etc. I do think IRAF has occasionally influenced Python and it may be that their considerations were similar here, but I don't see any evidence of PyRAF deliberately taking measures to reproduce the format shown by CL on Python 2; in fact, I think I found some corner case where it differs slightly, when looking into it. Third, an increase in precision should be numerically harmless. Fourth, there are other factors leading to (larger) floating-point differences anyway, such as what C library version IRAF was linked against. Fifth, CL scripts could explicitly format numeric output if they really care. So maybe it's better to accept this one and edit our tests and code to cope with it; I still have to finish looking into that.
Having said all of the above(!), since I had 189 failures to characterize, I actually did have a quick go at hacking PyRAF to reproduce Python 2's float.__str__()
behaviour, successfully. Combining those changes with the nint()
fix and a few more fixes in our tests (and disregarding 14 tests that still fail for reasons unrelated to Python 3), I was able to prove that all of the remaining failures are due to those language changes in Python 3, which I think is a useful point of reference and means that we're close to being able to say that Gemini IRAF works on Python 3.
Originated from a request by @olebole in #36 (comment)
p.s. Some of the tests in pandokia might require IRAF. I don't think those are going to be ported over.
p.p.s. This is more painful than it sounds. Given the resources we have, this might actually never happen...
Can't load this one due to another package not existing.
--> song
Traceback (innermost last):
File "<CL script CL1>", line 1, in <module>
File "<CL script clpackage.song>", line 27, in song
iraf.rvx()
AttributeError: Undefined IRAF task `rvx'
User has HiDPI display, with 4K resolution, and has problems with PyRAF epar
GUIs, with labels being too small. According to @cdsontag , there does not appear to be a way for user to configure font size.
As reported by @sosey , using the following quote (the acute character), which was from a copy-and-paste of instructions for an IRAF package:
โ
which would be used to set a param like this:
dimension_in = โ+100,0,0,0โ
gets translated to this:
dimension_in = '\xe2\x80\x99+100,0,0,0\xe2\x80\x99'
That is:
--> str('โ')
'\xe2\x80\x99'
where PyRAF assumes the input is a string and adds the hex translation for the acute into the string during a strip()
which is supplied and barfs later when there's string inspection.
This is unfortunate for users following documentation, but probably doesn't happen often.
We have gotten approval to do this (finally). So we need to work out the logistics of this. On our end I think Joe Hunkeler can handle it on our side, but I believe he would also need access to make changes in this repository. Let me know what needs to be done.
Trying to load this subpackage throws an IRAF error on my OSX and Linux boxes:
IrafError: Cannot find executable for task xobsolete
The online doc for the xray package seems to indicate that this package is a good candidate for removal, as it will "soon be removed from the system."
Setting parameters like the following works for , for example, apphot.
[IN]
iraf.apphot.datapars.itime = 10.
print(iraf.apphot.datapars.itime)
[OUT]
10.
It also works for immatch.geotran, however, it doesn't for immatch.wcsmap.
[IN]
iraf.immatch.wcsmap.fitgeom = 'rxyscale'
print(iraf.immatch.wcsmap.fitgeom)
[OUT]
general
Do I miss something?
Using
pyraf 2.1.15
iraf-community 2.16.1+2021.06.14
Thanks.
Some of our tests are failing on Python 3 with errors like this:
CL syntax error at `=' (file "/rtfproc/rtftests/gemini_iraf/gmosspec2/testgsreduction/test6/testgsreduction_test6.cl", line 111)
The problem can be reduced to the following script:
from pyraf import clscan
cltext = '''
char cenwavvalue[4]
cenwavvalue[1] = "580"
'''
scanner = clscan.CLScanner()
tokens = scanner.tokenize(cltext)
print(tokens)
With PyRAF 2.1.15 on Python 2.7.16 (where the error does not occur), this produces the following output, showing how the CL script is supposed to get tokenized:
[char, cenwavvalue, [, 4, ], NEWLINE, cenwavvalue, [, 1, ], =, '580', NEWLINE]
But on Python 3.7.9 (with either the same PyRAF 2.1.15 or a recent checkout of 2.2.1dev3), it produces the following, where the array subscript on the second line is treated as a single string, '[1]', rather than being split into opening & closing brackets and an integer:
[char, cenwavvalue, [, 4, ], NEWLINE, cenwavvalue, '[1]', =, '580', NEWLINE]
PyRAF matches different types of tokens using a regex for each type, defined in clscan.py
. These get concatenated together into a long regex for a given "scanner" -- in this case _CommandScanner
-- which is used to determine which of the patterns matches the next character(s). After quite a bit of digging, it seems that the problem is a change in the order of the individual regexes for _CommandScanner
. Specifically, the t_string
regex now appears before t_lbracket
instead of afterwards, causing [1]
to get matched as a string instead of matching the opening bracket.
The order in which the component regexes get concatenated (and hence their precedence) is determined by _namelist
in generic.py
. This iterates over the methods in the scanner class __dict__.keys()
, which behaves differently in Python 2 & 3. In Python 2, it's just an ordinary dict
, so I think its order is undefined but (fortunately) appears to be repeatable... In Python >=3.6, it's a mappingproxy
object that preserves the order in which the methods are defined in the code. So t_string
has moved before t_lbracket
just because that's how they are written in the (parent) class _CommandScanner_1
.
Lines 269 to 291 in 7c63d6a
The error can therefore be solved simply by reversing the order of those 2 methods in in _CommandScanner_1
, putting t_lbracket
first. I confirm that this makes the error disappear and my test now runs to completion on Python 3 (with some small numerical differences that will be unrelated).
I would appreciate any input from @cdsontag here, because although I'm confident that this fixes the problem I was troubleshooting, I'm not sure what the consequences of reordering these regexes might be more generally. I think there can't be many other cases where the parsing is messed up in this way, since most of our tests are passing and I'm not spotting other errors that look similar but not identical. I suppose we could study the regex ordering that Python 2 actually produces and try to reproduce that, which should be safe, but I'm not sure whether it's possible to reproduce exactly, given that inheritance will force grouping of the methods within the . Thanks.__dict__
in Python 3
For example, see https://ssb.stsci.edu/pandokia/pandokia.cgi?query=summary&qid=3443
It is known that PyRAF will not support Python 3. So the question is:
The dither package wants pydrizzle, which it can't seem to find...
--> dither
+------------------------------------------------------------+
| DITHER Version 2.3 (13 Nov 2009) |
| |
| Deprecated tasks MultiDrizzle, PyDrizzle, xytosky, |
| and tweakshifts have been removed from this package. |
| Ureka 1.5.1 contains these deprecated tasks; found at |
| http://ssb.stsci.edu/ureka/1.5.1/ |
| The DrizzlePac Python package replaces those tasks. |
| Use 'import drizzlepac' |
| to load the new tasks under Python or pyraf. |
| See http://drizzlepac.stsci.edu for details. |
| No changes have been made to any IRAF-based tasks. |
+------------------------------------------------------------+
Traceback (innermost last):
File "<CL script CL1>", line 1, in <module>
File "<CL script analysis.dither>", line 64, in dither
iraf.pyexecute('dither$tran_iraf.py', tasknames = 'tran', PkgName=PkgName,PkgBinary=PkgBinary)
File "/Users/ely/anaconda/envs/iraf/variants/common//iraf/stsci_iraf//stsdas/pkg/analysis/dither/tran_iraf.py", line 9, in <module>
import tran
File "/Users/ely/anaconda/envs/iraf/variants/common/iraf/stsci_iraf/stsdas/python/tran.py", line 83, in <module>
import pydrizzle
ImportError: No module named pydrizzle
dither can be loaded if you first import drizzlepac. But If this doesn't work or shouldn't be used anymore, It should probably be removed from future distributions.
Another one that can't startup on OSX or Linux
--> mscdb
Traceback (innermost last):
File "<CL script CL1>", line 1, in <module>
IrafError: Cannot find executable for task mscdb
Update: this was the first of several related problems discussed here; see below.
It turns out that at least one of our CL scripts has a special character (non-breaking space) in a comment for some reason, which PyRAF happily treats as part of the comment under Python 2, while Python 3 gives the following error when trying to read the script (before PyRAF even gets as far as parsing it):
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa0 in position 7931: invalid start byte
This can be reproduced trivially with a 2-line Python script:
fh = open('/home/jturner/repos/gemini/gmos/mostools/gmskcreate.cl')
text = fh.read()
I wonder whether utf-8
is actually the best assumption for CL scripts? This one is probably latin-1
? Anyway, I'll remove the offending character from our script, but I just thought this might be worth mentioning in case other people report seeing the above error (especially when just running lpar
) and I thought I'd put it in an issue so they can find it in a Web search. Feel free to close this without doing anything if you don't think there's a better default encoding specification.
The following snippet does not work with Python 2:
from io import StringIO
from pyraf import iraf
out = StringIO()
iraf.imhead('dev$pix', Stdout=out)
print out.getValue()
I get the following exception:
/usr/lib/python2.7/dist-packages/pyraf/iraftask.pyc in __call__(self, *args, **kw)
765
766 def __call__(self,*args,**kw):
--> 767 return self.run(*args, **kw)
768
769 def __repr__(self):
/usr/lib/python2.7/dist-packages/pyraf/iraftask.pyc in run(self, *args, **kw)
357 if executionMonitor:
358 executionMonitor(self)
--> 359 self._run(redirKW, specialKW)
360 self._updateParList(save)
361 if Verbose>1:
/usr/lib/python2.7/dist-packages/pyraf/iraftask.pyc in _run(self, redirKW, specialKW)
806 """
807 try:
--> 808 irafexecute.IrafExecute(self, pyraf.iraf.getVarDict(), **redirKW)
809 except irafexecute.IrafProcessError, value:
810 raise IrafError("Error running IRAF task "+self._name+
/usr/lib/python2.7/dist-packages/pyraf/irafexecute.pyc in IrafExecute(task, envdict, stdin, stdout, stderr, stdgraph)
337 gki.kernel.pushStdio(None,None,None)
338 try:
--> 339 irafprocess.run(task, pstdin=stdin, pstdout=stdout, pstderr=stderr)
340 finally:
341 if stdgraph:
/usr/lib/python2.7/dist-packages/pyraf/irafexecute.pyc in run(self, task, pstdin, pstdout, pstderr)
506 try:
507 # begin slave mode
--> 508 self.slave()
509 finally:
510 self.running = 0
/usr/lib/python2.7/dist-packages/pyraf/irafexecute.pyc in slave(self)
643 xfer()
644 elif msg5 == 'xmit(':
--> 645 xmit()
646 elif msg[:4] == 'bye\n':
647 return
/usr/lib/python2.7/dist-packages/pyraf/irafexecute.pyc in xmit(self)
818 return
819
--> 820 self.stdout.write(txdata)
821 self.stdout.flush()
822 elif chan == 5:
TypeError: unicode argument expected, got 'str'
With Python 3, this works nicely (after changing the print statement into a function).
Hello,
I was trying to create a quick notebook on processing some data that I could share with a colleague online via mybinder.org. However, I encountered the issue that pyraf wouldn't recognize the iraf installation. It comes down to setting the environment variables iraf and irafarch. I have opened this issue yesterday on the mybinder github site, where you can check the brief discussion we had:
jupyterhub/binderhub#575
They recommended that I get in touch with you to resolve this issue. The main suggestion was to use the os python module to set the environment variables at the first cell. I did that and unfortunately they still weren't set. I would appreciate if you have any thoughts/suggestions?
Many thanks,
Iskren
I have encountered this in tests that copy FITS files (using 8ea6169 under Python 3.7), but the following reproduces it without requiring any additional data:
>>> from pyraf import iraf
>>> iraf.copy('dev$pix.imh', './')
PANIC in `/rtfproc/anaconda3_2019.10/envs/pyraf3test/iraf/bin.linux/x_system.e': Out of memory
Killing IRAF task `copy'
Traceback (most recent call last):
File "/rtfproc/anaconda3_2019.10/envs/pyraf3test/lib/python3.7/site-packages/pyraf/iraftask.py", line 880, in _run
irafexecute.IrafExecute(self, iraf.getVarDict(), **redirKW)
File "/rtfproc/anaconda3_2019.10/envs/pyraf3test/lib/python3.7/site-packages/pyraf/irafexecute.py", line 371, in IrafExecute
raise exc
File "/rtfproc/anaconda3_2019.10/envs/pyraf3test/lib/python3.7/site-packages/pyraf/irafexecute.py", line 348, in IrafExecute
irafprocess.run(task, pstdin=stdin, pstdout=stdout, pstderr=stderr)
File "/rtfproc/anaconda3_2019.10/envs/pyraf3test/lib/python3.7/site-packages/pyraf/irafexecute.py", line 512, in run
self.slave()
File "/rtfproc/anaconda3_2019.10/envs/pyraf3test/lib/python3.7/site-packages/pyraf/irafexecute.py", line 636, in slave
self.msg = self.readString()
File "/rtfproc/anaconda3_2019.10/envs/pyraf3test/lib/python3.7/site-packages/pyraf/irafexecute.py", line 578, in readString
return Iraf2AscString(self.read())
File "/rtfproc/anaconda3_2019.10/envs/pyraf3test/lib/python3.7/site-packages/pyraf/irafexecute.py", line 607, in read
str(header[0:2]))
pyraf.irafexecute.IrafProcessError: Not a legal IRAF pipe record: b''
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/rtfproc/rtftests/gemini_iraf/gmosspec2/iraf/test/test3.py", line 8, in <module>
iraf.copy('dev$pix.imh', './')
File "/rtfproc/anaconda3_2019.10/envs/pyraf3test/lib/python3.7/site-packages/pyraf/iraftask.py", line 836, in __call__
return self.run(*args, **kw)
File "/rtfproc/anaconda3_2019.10/envs/pyraf3test/lib/python3.7/site-packages/pyraf/iraftask.py", line 375, in run
self._run(redirKW, specialKW)
File "/rtfproc/anaconda3_2019.10/envs/pyraf3test/lib/python3.7/site-packages/pyraf/iraftask.py", line 883, in _run
str(value))
stsci.tools.irafglobals.IrafError: Error running IRAF task copy
Not a legal IRAF pipe record: b''
>>>
Hello @cdsontag, @stsci-ssb-ci (or @plim?),
since I am getting done with the Python 3 transition, I would also like to create a new version and push it to PyPI soon. However, the pyraf project there is still maintained by you (cdsontag, stsci-ssb). Could I ask to add me to the maintainer's list there so that I can upload new versions without bugging you?
Thank you very much!
Ole
More missing executables
--> xlocal
Traceback (innermost last):
File "<CL script CL1>", line 1, in <module>
IrafError: Cannot find executable for task xlocal
This came up in a conversation, so just documenting it in case there is real interest.
As STScI winds down its support in the future, it would be more useful for this package to be in its own organization so that people or other institution(s) out there who still rely on this package may ramp up their support of it.
There is already a precedence with https://github.com/airspeed-velocity/asv .
trying to load the kepler package hits an error as it's trying to load pyfits
--> kepler
Traceback (innermost last):
File "<CL script CL1>", line 1, in <module>
iraf.kepler(_save=1)
File "<CL script clpackage.kepler>", line 33, in kepler
iraf.pyexecute('kepler$kepbls.py', verbose = no, PkgName=PkgName,PkgBinary=PkgBinary)
File "/Users/ely/anaconda/envs/iraf/variants/common//iraf/kepler/kepbls.py", line 1, in <module>
import numpy, scipy, sys, time, pyfits, pylab, math, re
ImportError: No module named pyfits
I'm seeing quite a lot of Gemini IRAF tests failing with a message such as KeyError: "Error in parameter 'centerpars.cthreshold' for task phot\n'Key centerpars.cthreshold not found'"
, using a recent checkout of iraf-community PyRAF for Python 3. All but one are cases where the failing IRAF task gets passed parameters from an "additional" pset (can't recall whether that's the proper IRAF terminology) in the form centerpars.cthreshold=0.
.
For example, I see the error quoted above when doing the following with pyraf 2.2.1.dev3 and stsci.tools 4.0.1 on Python 3.7.9:
digiphot
apphot
phot dev$pix coords="" output="default" interactive- centerpars.calgorithm="none" centerpars.cthreshold=0.
This only happens some of the time, depending on the exact parameters used, due to the nature of the bug. With earlier PyRAF releases, the task instead continues to a prompt like the following in this particular case (which seems like dubious behaviour in itself, but that's a different problem!):
Centering algorithm (none) (CR or value):
After a bit of troubleshooting, this seems to be a regression introduced by no longer running 2to3
on the source code (making it hard to spot, since there's no change to the offending line in the revision history!). Because kw.keys()
no longer gets converted to a list
by 2to3
, the following block of code is now modifying a dict
in place while iterating over it (rather than over an existing copy of the keys), leading to unpredictable behaviour:
Lines 912 to 918 in 1a63a65
I think the solution is just to put L912 back to this (or use copy
or something):
for key in list(kw.keys()):
Thanks!
James.
I want to run pyraf non-interactively; f.e. as a test during an automated build. But even trying to import it does not work. The follwing minimal example:
from stsci.tools import capable
capable.OF_GRAPHICS = False
from pyraf import iraf
When running directly from the command line, it works. However, when I redirect the input, it does not:
$ python foo.py </dev/null
stty: 'standard input': Inappropriate ioctl for device
i: Traceback (most recent call last):
File "foo.py", line 3, in <module>
from pyraf import iraf
File "/usr/lib/python2.7/dist-packages/pyraf/__init__.py", line 126, in <module>
iraf.Init(doprint=0, hush=1)
File "/usr/lib/python2.7/dist-packages/pyraf/iraffunctions.py", line 257, in Init
userpkg.run(_doprint=0, _hush=hush, _save=1)
File "/usr/lib/python2.7/dist-packages/pyraf/iraftask.py", line 359, in run
self._run(redirKW, specialKW)
File "/usr/lib/python2.7/dist-packages/pyraf/iraftask.py", line 1667, in _run
self._runCode()
File "/usr/lib/python2.7/dist-packages/pyraf/iraftask.py", line 1470, in _runCode
self._clFunction(*parList, **kw)
File "<CL script clpackage.user>", line 32, in login
File "/usr/lib/python2.7/dist-packages/pyraf/iraftask.py", line 1220, in __getattr__
return self._taskObj.getParam(paramname,native=1,mode="h",exact=1)
File "/usr/lib/python2.7/dist-packages/pyraf/iraftask.py", line 564, in getParam
exact=exact, prompt=prompt)
File "/usr/lib/python2.7/dist-packages/pyraf/iraftask.py", line 616, in _getParValue
field, native, mode="h", prompt=prompt)
File "/usr/lib/python2.7/dist-packages/pyraf/iraftask.py", line 905, in _getParFromDict
native=native,mode=pmode,prompt=prompt)
File "/usr/lib/python2.7/dist-packages/stsci/tools/basicpar.py", line 367, in get
if prompt: self._optionalPrompt(mode)
File "/usr/lib/python2.7/dist-packages/stsci/tools/basicpar.py", line 643, in _optionalPrompt
self.getWithPrompt()
File "/usr/lib/python2.7/dist-packages/stsci/tools/basicpar.py", line 349, in getWithPrompt
raise EOFError("EOF on parameter prompt")
EOFError: EOF on parameter prompt
This happens on Python 2.7 as well as on Python 3.6.
I just tagged a new release candidate, 2.2.0rc1, and uploaded it to PyPI:
@pllim @cdsontag @jehturner @jhunkeler if you have a few moments to have a look there and check that the info shown there is correct. If you like, you can also try out the package ๐
Anyone else is warmly invited as well...
UnsatisfiableError: The following specifications were found
to be incompatible with the existing python installation in your environment:
Specifications:
- pyraf -> python=2.7
Your python: python=3.9
If python is on the left-most side of the chain, that's the version you've asked for.
When python appears to the right, that indicates that the thing on the left is somehow
not available for the python version you are constrained to. Note that conda will not
change your python version to a different minor version unless you explicitly specify
that.
The following specifications were found to be incompatible with your system:
- feature:/linux-64::__glibc==2.33=0
- python=3.9 -> libgcc-ng[version='>=7.5.0'] -> __glibc[version='>=2.17']
Your installed version is: 2.33
When I install pyRAF by conda install -c pkgw/label/superseded pyraf
on here, this above is what it says.
These two tests failed with similar error on pembry
public. @jhunkeler , do you know why?
https://ssb.stsci.edu/pandokia/pandokia.cgi?query=detail&key_id=38273319
https://ssb.stsci.edu/pandokia/pandokia.cgi?query=detail&key_id=38273203
Traceback (most recent call last):
File "/Users/shared/iraf_conda/bldtmp/porcelain.pkk9bc5SPq/porcelain/envs/public/lib/python2.7/site-packages/pandokia/helpers/runner_minipyt.py", line 583, in process_file
module = imp.load_source( module_name, filename )
File "site.py", line 703, in <module>
main()
File "site.py", line 694, in main
execsitecustomize()
File "site.py", line 548, in execsitecustomize
import sitecustomize
File "/Users/iraf/rtx/stsci_python/pyraf/.tmp/ssbvirt/ssbdev-osx/lib/python2.7/sitecustomize.py", line 13, in <module>
OSError: [Errno 2] No such file or directory: '/Users/shared/iraf_conda/bldtmp/porcelain.pkk9bc5SPq/porcelain/envs/public/python/lib/python2.7/site-packages'
https://pypi.org/project/pyraf/ still has old info from STScI. Info and metadata there needs to be updated to clarify that releases now come from iraf-community
, not STScI.
I am having an issue using my cursor on the graphics window on a Mac. It seems to work variably, so sometimes it's fine, but sometimes it gets stuck at won't move once it's on the graphics window. I am using iraf in conjunction with the python3 version of this pipeline:
https://github.com/svalenti/pessto
and was directed here after posting the issue to the pipeline repository.
A video demonstrating the issue I am having can be found here:
https://cf-my.sharepoint.com/:f:/g/personal/evanscr8_cardiff_ac_uk/Etzklvf_5VZAvQbcDMKOgZEBY2UVQgzhUqhvObBS3PRulg?e=XywCdF
System details:
Tyler Desjardins mentions that we should consider moving emails from help[at]stsci.edu
to point to the web portal where possible and appropriate. For HST (or any non-JWST), it is https://hsthelp.stsci.edu . For JWST, it is https://jwsthelp.stsci.edu . Please update info in setup.py
, setup.cfg
, documentation, etc as appropriate.
Please close this issue if it is irrelevant to your repository. This is an automated issue. If this is opened in error, please let pllim know!
This one hits a strange error when you try to load it.
--> kpno
Traceback (innermost last):
File "<CL script CL1>", line 1, in <module>
File "<CL script obsutil.kpno>", line 18, in kpno
if (iraf.access('spectimedb$')):
IrafError: Undefined variable `spectimedb' in string `spectimedb$'
Adding an issue for this as requested in #53 (comment). I'm currently not sure what these mean.
[...]
build/lib.linux-x86_64-3.6/pyraf/old_files/gki_sys_tests.py build/lib.linux-x86_64-3.6/pyraf/old_files/interactive_tests.py build/lib.linux-x86_64-3.6/pyraf/old_files/iraffunctions_tests.py build/lib.linux-x86_64-3.6/pyraf/old_files/irafpar_tests.py build/lib.linux-x86_64-3.6/pyraf/old_files/subproc_tests.py
Skipping optional fixer: buffer
Skipping optional fixer: idioms
Skipping optional fixer: set_literal
Skipping optional fixer: ws_comma
Can't parse build/lib.linux-x86_64-3.6/pyraf/ipython_api.py: ParseError: bad input: type=22, value='=', context=('', (384, 54))
Can't parse build/lib.linux-x86_64-3.6/pyraf/irafecl.py: ParseError: bad input: type=22, value='=', context=('', (191, 61))
Can't parse build/lib.linux-x86_64-3.6/pyraf/iraffunctions.py: ParseError: bad input: type=22, value='=', context=('', (2711, 22))
Can't parse build/lib.linux-x86_64-3.6/pyraf/iraftask.py: ParseError: bad input: type=22, value='=', context=('', (346, 67))
Can't parse build/lib.linux-x86_64-3.6/pyraf/cl2py.py: ParseError: bad input: type=22, value='=', context=('', (160, 56))
Can't parse build/lib.linux-x86_64-3.6/pyraf/clparse.py: ParseError: bad input: type=22, value='=', context=('', (438, 23))
Can't parse build/lib.linux-x86_64-3.6/pyraf/clscan.py: ParseError: bad input: type=22, value='=', context=('', (985, 32))
Can't parse build/lib.linux-x86_64-3.6/pyraf/tpar.py: ParseError: bad input: type=22, value='=', context=('', (1386, 96))
Fixing build/lib.linux-x86_64-3.6/pyraf/gkigcur.py build/lib.linux-x86_64-3.6/pyraf/gkiiraf.py build/lib.linux-x86_64-3.6/pyraf/gkitkbase.py build/lib.linux-x86_64-3.6/pyraf/gkitkplot.py build/lib.linux-x86_64-3.6/pyraf/graphcap.py build/lib.linux-x86_64-3.6/pyraf/gwm.py build/lib.linux-x86_64-3.6/pyraf/ipython_api.py
[...]
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.