dstndstn / astrometry.net Goto Github PK

Please maintain the changelog at http://astrometry.net/doc/changelog.html or add changelog information directly to the github release texts. Thanks!

So, there's entries missing for 0.48, 0.49, and 0.50.

Edit: Maybe it's easier for maintenance if you add a changelog.txt file directly to the repository so that whenever you fix a bug or add a feature you just add a line to this file. This way you won't forget the changes until the release.

I couldn't find http://astrometry.net/doc/net/api.html !

Also, it wouldn't hurt to finish that page!

Hi,
In sip.c the function void tan_pixelxy2xyzarr()
contains a comment
// Mysterious factor of -1 correcting for vector directions below.
In fact, the (i,j,r) defined in the routine is not right-ended.
So one is forced to introduce this extra -1 in x
But if one use a vector i' = -i, then (i',j,r) is right-ended and one uses normal (x,y)
value.

In brief; the present code is correct, but can be nicer using right-ended frame ;)
JE

fix this already!

astrometry.net doesn't compile with gcc 5.x, I think the reason is the different semantics for inline functions. Consult https://gcc.gnu.org/gcc-5/porting_to.html. A first solution or workaround: Add -std=gnu89 to used compiler flags.

Best Regards,
Christian Dersch

I've upgraded from release 0.46 to 0.50 and I have since not been able to receive the "RMS error of correspondences" or the "Weighted RMS error of correspondences" in the output. For 0.46 the -v -v option would return both of those but that doesn't seem to be the case in 0.50. I've looked in the code and found that it still exists in tweak.c but it looks like that code can no longer be called any more.

Hi Dustin
Look at the code I have copied from sip.c file. Why for instance the loop
for (q=0; q<=sip->ap_order; q++)
is not
for (q=0; (p+q)<=sip->ap_order; q++){
as
even for an inverse SIP transform p+q <= AP_ORDER (for the U -> u part) as described
in the 2008 Shupe & Hook paper?
Does this means that you have extended their paper ?

JE

void sip_calc_inv_distortion(const sip_t* sip, double U, double V, double* u, double *v) {
int p, q;
double fUV=0.;
double gUV=0.;

// avoid using pow() function
double powu[SIP_MAXORDER];
double powv[SIP_MAXORDER];
powu[0] = 1.0;
powu[1] = U;
powv[0] = 1.0;
powv[1] = V;
for (p=2; p <= MAX(sip->ap_order, sip->bp_order); p++) {
    powu[p] = powu[p-1] * U;
    powv[p] = powv[p-1] * V;
}

for (p=0; p<=sip->ap_order; p++)
    for (q=0; q<=sip->ap_order; q++)
        //fUV += sip->ap[p][q] * pow(U,p) * pow(V,q);
        fUV += sip->ap[p][q] * powu[p] * powv[q];
for (p=0; p<=sip->bp_order; p++) 
    for (q=0; q<=sip->bp_order; q++) 
        //gUV += sip->bp[p][q] * pow(U,p) * pow(V,q);
        gUV += sip->bp[p][q] * powu[p] * powv[q];

*u = U + fUV;
*v = V + gUV;

}

I guess this a missprint in sip.h file

// SIP projection rather than TAN.
anbool sin;

SIP -> SIN projection ?

JE

From https://groups.google.com/forum/#!topic/astrometry/vGMcxhjfq9s

For example, I run the first instance of solve-field.

$ solve-field -p -O -S none -U none -N none -R none --tag-all --depth 150 --objs 150 --crpix-center tmpframe.fits

This gives me the center of my FOV in the wcs file by the command --crpix-center. When I resolve to obtain more star matches, the --crpix-center command seems to be ignored because in the wcs file the values change to the edge of my FOV. Do you know why this is the case?

$ solve-field --verify tmpframe.wcs --crpix-center -p -S none -U none -N none -R none --tag-all --continue --axy rd2.axy tmpframe.axy

As per https://groups.google.com/forum/#!topic/astrometry/gyWYg3bQmZ0
apod3.jpg takes forever with index-4211.fits

Hi everyone,

I'm working with very wide FoV images (40degx40deg) and using a.net-0.50 for getting astrometric solution. For such huge FoV I need TAN-SIP with at least 5th order polynomial distortion.

However, in the a.net TAN-SIP FITS output images, I found that SExtracted coordinates (e.g. ALPHAWIN_J2000,DELTAWIN_J2000) are not correct (they are several arcmins offset from the cataloged coordinates). This issue is well know and is because SExtractor does support TAN-SIP projected FITS images.

So, as I knew that David Shupe (IPAC) had developed for Palomar Transient Factory (PTF) a SIP_A,B_i_j <-> TAN_PV_i_j converter down to 4th distorsion order, I contacted him if he could extend such converter to higher orders. He kindly accepted and he coded a python modularized script (see his github which converts SIP_A,B_i_j into TAN_PV_i_j down to 7th order.
After a few months of testing from my side, the code is stable and with no issues.

Now, I requested David if his python script could be ported to C, so that a.net can directly output TAN_PV_i_j projected FITS images with distorsion polynomials down to 7th order.
That --output-tpv a.net option would be very convenient, since it saves time of storing the TAN-SIP image on disk, and then converting it into TAN_PV with David python script.

Thus my request to a.net developers would be to port David's SIP_A,B_i_j ->TAN_PV_i_j converter script to a.net (in C, I guess).

Note README and LICENSE.txt in the github. David is fine with the C port as long as the terms of the license are met.

I think the temp files ( specified with --temp-dir ) are not all deleted after 'solve-field' end. I have tried the option '--no-delete-temp' and I have found the next files:

/tmp/tmp.sanitized.NvMQq7
/tmp/tmp.uncompressed.ptgyiy
/tmp/tmp.ppm.XmWfaZ
/tmp/tmp.xyls.d9lnRI
/tmp/tmp.removelines.TqXFXR
/tmp/tmp.sorted.F7SzB1
/tmp/tmp.uniform.h98kgb

However, when I remove the '--no-delete-temp', all the temp files are deleted, except /tmp/tmp.sanitized.XXXXX (the biggest one). It could cause that after solving hundred of files, the temp-dir could be full.

(It was observed with astrometry.net-0.50)

an-fitstopnm assumes the image is in the primary FITS extension. Should check for images in the first extension, since that's so common.

FITS images with, eg, invalid header cards are often encountered, and currently we fail hard on them.

I'm still on 0.43 currently but I'll test it as well on 0.50 soon. A quick look however showed no relevant changes in the code.

...
Field 1 did not solve (index index-4116.fits, field objects 31-40).
Field 1 did not solve (index index-4115.fits, field objects 31-40).
Field 1 did not solve (index index-4114.fits, field objects 31-40).
Field 1 did not solve (index index-4119.fits, field objects 41-50).
Field 1 did not solve (index index-4118.fits, field objects 41-50).
Field 1 did not solve (index index-4117.fits, field objects 41-50).
backend: tweak2.c:456: tweak2: Assertion `theta[i] < Nin' failed.
solve-field.c:523:run_backend backend failed.  Command that failed was:
  /opt/astrometry/bin/backend /tmp/tmpFLZvHz/astrometry/masked.axy
 ioutils.c:661:run_command_get_outputs Command was killed by signal 6

This happened quite a couple of times. Do you need the axy file for testing? If yes, where should I upload it (nova isn't suitable because it uses a different index set)?

When running two solve-field commands from the same machine in different shells, I've found that one or the other will fail to read the index files. Does each call to solve-field keep an open file stream for the index files? And therefore should solve-field only be called serially?

Thanks!

Hi, this is my question:
I am new to using the astrometry.net software. It has been installed in my /user/local/ directory, with the build directory astrometry.net-0.50 off my home directory. I downloaded several indexes (all labeled index-42XX-YY.fits) that say “are built from the 2MASS catalog,” and then moved them into my /user/local/astrometry/data directory. Here is what I get when I try to solve a field or perform astrometry from my home directory,

archos:~ mashbudj$ /usr/local/astrometry/bin/solve-field --scale-low 10 --overwrite ~/photometry/LFC/PSRJ1622/g_band/restart/dnpccd.068.1.fits
Reading input file 1 of 1: "/Users/mashbudj/photometry/LFC/PSRJ1622/g_band/restart/dnpccd.068.1.fits"...
Header has 107 cards
/usr/local/astrometry/lib/python/astrometry/util/fits2fits.py:49: PyfitsDeprecationWarning: The ascard function is deprecated as of version 3.1 and may be removed in a future version.

    Use the `.cards` attribute instead.

for c in hdr.ascard:
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pyfits/card.py:70: PyfitsDeprecationWarning: The CardList class has been deprecated; all its former functionality has been subsumed by the Header class, so CardList objects should not be directly created. See the PyFITS 3.1.0 CHANGELOG for more details.
PyfitsDeprecationWarning)
Extracting sources...
simplexy: found 648 sources.
Solving...
index.c:154:index_is_file_index: Index file /usr/local/astrometry/data/index-4107.quad.fits is not readable.
index.c:154:index_is_file_index: Index file /usr/local/astrometry/data/index-4107.quad.fits is not readable.
fitsbin.c:580:fitsbin_open: Failed to open file "/usr/local/astrometry/data/index-4107.skdt.fits"
starkd.c:257:my_open: Failed to open FITS file "/usr/local/astrometry/data/index-4107.skdt.fits"
index.c:414:index_reload: Failed to read star kdtree from file /usr/local/astrometry/data/index-4107.skdt.fits
engine.c:211:engine_add_index: Failed to load index from path /usr/local/astrometry/data/index-4107
Failed to add index "/usr/local/astrometry/data/index-4107".
index.c:154:index_is_file_index: Index file /usr/local/astrometry/data/index-4205-07.fits.quad.fits is not readable.
fitsbin.c:580:fitsbin_open: Failed to open file "/usr/local/astrometry/data/index-4205-07.fits.skdt.fits"
starkd.c:257:my_open: Failed to open FITS file "/usr/local/astrometry/data/index-4205-07.fits.skdt.fits"
index.c:414:index_reload: Failed to read star kdtree from file /usr/local/astrometry/data/index-4205-07.fits.skdt.fits
engine.c:211:engine_add_index: Failed to load index from path /usr/local/astrometry/data/index-4205-07.fits
Failed to add index "/usr/local/astrometry/data/index-4205-07.fits".
index.c:154:index_is_file_index: Index file /usr/local/astrometry/data/index-4204-29.fits.quad.fits is not readable.
fitsbin.c:580:fitsbin_open: Failed to open file "/usr/local/astrometry/data/index-4204-29.fits.skdt.fits"
starkd.c:257:my_open: Failed to open FITS file "/usr/local/astrometry/data/index-4204-29.fits.skdt.fits"
index.c:414:index_reload: Failed to read star kdtree from file /usr/local/astrometry/data/index-4204-29.fits.skdt.fits
engine.c:211:engine_add_index: Failed to load index from path /usr/local/astrometry/data/index-4204-29.fits
Failed to add index "/usr/local/astrometry/data/index-4204-29.fits".
index.c:154:index_is_file_index: Index file /usr/local/astrometry/data/index-4203-29.fits.quad.fits is not readable.
fitsbin.c:580:fitsbin_open: Failed to open file "/usr/local/astrometry/data/index-4203-29.fits.skdt.fits"
starkd.c:257:my_open: Failed to open FITS file "/usr/local/astrometry/data/index-4203-29.fits.skdt.fits"
index.c:414:index_reload: Failed to read star kdtree from file /usr/local/astrometry/data/index-4203-29.fits.skdt.fits
engine.c:211:engine_add_index: Failed to load index from path /usr/local/astrometry/data/index-4203-29.fits
Failed to add index "/usr/local/astrometry/data/index-4203-29.fits".
index.c:154:index_is_file_index: Index file /usr/local/astrometry/data/index-4202-29.fits.quad.fits is not readable.
fitsbin.c:580:fitsbin_open: Failed to open file "/usr/local/astrometry/data/index-4202-29.fits.skdt.fits"
starkd.c:257:my_open: Failed to open FITS file "/usr/local/astrometry/data/index-4202-29.fits.skdt.fits"
index.c:414:index_reload: Failed to read star kdtree from file /usr/local/astrometry/data/index-4202-29.fits.skdt.fits
engine.c:211:engine_add_index: Failed to load index from path /usr/local/astrometry/data/index-4202-29.fits
Failed to add index "/usr/local/astrometry/data/index-4202-29.fits".
index.c:154:index_is_file_index: Index file /usr/local/astrometry/data/index-4201-29.fits.quad.fits is not readable.
fitsbin.c:580:fitsbin_open: Failed to open file "/usr/local/astrometry/data/index-4201-29.fits.skdt.fits"
starkd.c:257:my_open: Failed to open FITS file "/usr/local/astrometry/data/index-4201-29.fits.skdt.fits"
index.c:414:index_reload: Failed to read star kdtree from file /usr/local/astrometry/data/index-4201-29.fits.skdt.fits
engine.c:211:engine_add_index: Failed to load index from path /usr/local/astrometry/data/index-4201-29.fits
Failed to add index "/usr/local/astrometry/data/index-4201-29.fits".

You must list at least one index in the config file (/usr/local/astrometry/bin/../etc/astrometry.cfg)

See http://astrometry.net/use.html about how to get some index files.

solve-field.c:529:run_engine engine failed. Command that failed was:
/usr/local/astrometry/bin/astrometry-engine /Users/mashbudj/photometry/LFC/PSRJ1622/g_band/restart/dnpccd.068.1.axy
ioutils.c:569:run_command_get_outputs Command failed: return value 255

Strangely, it continues to look for indexes that are no longer in /usr/local/astrometry/data/, (I have these below)
index-4203-29.fits
index-4201-29.fits index-4204-29.fits
index-4202-29.fits index-4205-07.fits
How do I get the program to recognize these files?

I only find astrometry.net-0.50/etc/astrometry.cfg in the build directory (adding names there did not work either). And there is no command called ‘install-indexes’ that I can find, does this mean I built it wrong?

I don't have any *skdt.fits or *quad.fits files; should these be downloaded, or am I supposed to create them? If so, how do I do that?

Thank you.
Daniel Mashburn
Physics Department
University WI-Milwaukee

Hi Dustin,

I integrated in my code solve-field via popen(...) for solving dslr images and sending the ra,dec values to the Celestron mount for
automatically align the mount/telescope. For that I parsed, the output line (here the apod1.jpg example):

solve-field --overwrite --downsample 2 -L 6 -H 7 -u arcsecperpix apod1.jpg

...
Field: apod1.jpg
** => Field center: (RA,Dec) = (100.2, 9.832) deg. **<=
Field center: (RA H:M:S, Dec D:M:S) = (06:40:51.759, +09:49:53.843).
...

The RA,Dec degree output is limit to four digits:
logmsg("Field center: (RA,Dec) = (%.4g, %.4g) deg.\n", ra, dec);

How about changing the output to:

logmsg("Field center: (RA,Dec) = (%3.6f, %3.6f) deg.\n", ra, dec);
This would result in:
** =>Field center: (RA,Dec) = (100.215663, 9.831623) deg. **<=

The Celestron library I am using wants the values in degrees.
It is not a big deal to parse for RA H:M:S, ... and convert the values to degrees. How about
increasing in general the precision output:

-- logmsg("Field center: (RA,Dec) = (%.4g, %.4g) deg.\n", ra, dec);
++ logmsg("Field center: (RA,Dec) = (%3.6f, %3.6f) deg.\n", ra, dec);

Cheers
Thomas

Reported on the Google Group as well.
Basically solve-field *.fit crashes at image #252 (starts counting at #1), then I can re-run it with the --skip-solved flag at which point it picks up fine (solves 252) and crashes at #503.
The version is Revision 24247, date 2013-12-17 14:33:48 +0000 (Tue, 17 Dec 2013).

I notice some use of char* in many places.

Of course when linking with C++ code there are some issues dealing with
string::c_str() function which provides a 'const char*'

For instance it is necessary to use char* in:
engine_add_search_path
engine_find_index
index_params_t::tempdir
build_index_files

JE

fixed against git master for a couple problems running make install-indexes

From 7128d99f4af75d31f4d7b6fa83c37adf939200fd Mon Sep 17 00:00:00 2001
From: Michael Plante 
Date: Thu, 2 Jul 2015 14:41:19 -0400
Subject: [PATCH 1/2] Makefile: fix missing redirection for bzip2'ed indices

---
 Makefile |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/Makefile b/Makefile
index 879c6a0..84c1f06 100644
--- a/Makefile
+++ b/Makefile
@@ -148,7 +148,7 @@ install-indexes:
        echo tar xvjf $$x -C '$(DATA_INSTALL_DIR)'; \
        tar xvjf $$x -C '$(DATA_INSTALL_DIR)'; \
    done
-   @for x in `ls index-*.bz2 | grep -v tar.bz2 2>/dev/null`; do \
+   @for x in `ls index-*.bz2 2>/dev/null | grep -v tar.bz2 2>/dev/null`; do \
        echo Installing $$x in '$(DATA_INSTALL_DIR)'...; \
        echo "cp $$x '$(DATA_INSTALL_DIR)' && bunzip2 --force '$(DATA_INSTALL_DIR)/'$$x;"; \
        cp $$x '$(DATA_INSTALL_DIR)' && bunzip2 --force '$(DATA_INSTALL_DIR)/'$$x; \
-- 


From dab24076e67a1947bca8ff1ff14a58612bd2ff8e Mon Sep 17 00:00:00 2001
From: Michael Plante 
Date: Thu, 2 Jul 2015 14:41:50 -0400
Subject: [PATCH 2/2] Makefile: fix missing semicolons for uncompressed indices

---
 Makefile |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index 84c1f06..9e9c1d8 100644
--- a/Makefile
+++ b/Makefile
@@ -160,8 +160,8 @@ install-indexes:
    done
    @for x in `ls index-*.fits 2>/dev/null`; do \
        echo Installing $$x in '$(DATA_INSTALL_DIR)'...; \
-       echo "cp $$x '$(DATA_INSTALL_DIR)'" \
-       cp $$x '$(DATA_INSTALL_DIR)' \
+       echo "cp $$x '$(DATA_INSTALL_DIR)'"; \
+       cp $$x '$(DATA_INSTALL_DIR)'; \
    done
 
 reconfig:
-- 

Fatal error on make:

bzlib.h: no such file or directory

Environment: DistroAstro2.0 (Ubuntu 12.04 with Kernel 3.8.0)
All prerequisites installed followed by apt-get update.

a la https://groups.google.com/forum/#!topic/astrometry/qWaOzW-QN7E

Typical summary of valgrind on MAC OS X Mavericks

1. During Index making
   ==92456== 28,815 (32 direct, 28,783 indirect) bytes in 1 blocks are definitely lost in loss record 1,068 of 1,077
   ==92456== at 0x6DB9: calloc (vg_replace_malloc.c:627)
   ==92456== by 0x100077C27: anqfits_open_hdu
   ==92456== by 0x3: ???
   ==92456== by 0x1049C529F: ???
   ==92456== by 0x14: ???

2. During solving
   ==92497== 960 bytes in 1 blocks are definitely lost in loss record 977 of 1,005
   ==92497== at 0x66BB: malloc (vg_replace_malloc.c:300)
   ==92497== by 0x10003C79E: permutation_init
   ==92497== by 0x1000EC8B9: tweak2
   ==92497== by 0x1049C1D1B: ???
   ==92497== by 0x1049C1D07: ???
   ==92497== by 0x1049C1D0F: ???
   ==92497==

Hope it helps to secure the code.
JE

ugh.
https://groups.google.com/d/msg/astrometry/DbKK0p6grpc/S5Z9Cbj3BQAJ

Figure out how to get everything; "git pull" doesn't do the trick.

Hi,
Looking at sip_calc_distortion in sip.c file here is a part of the code

    for (q=0; q<=sip->a_order; q++)

        //if ((p+q > 0) && (p+q <= sip->a_order))

        if (p+q <= sip->a_order)

            fuv += sip->a[p][q] * powu[p] * powv[q];

for (p=0; p<=sip->b_order; p++) 

    for (q=0; q<=sip->b_order; q++) 

        //if ((p+q > 0) && (p+q <= sip->b_order))

        if (p+q <= sip->b_order)

            guv += sip->b[p][q] * powu[p] * powv[q];

In fact it uses constant and linear terms which are both not used for the Foward distortion computattions by SIP conventions
http://fits.gsfc.nasa.gov/registry/sip/SIP_distortion_v1_0.pdf

One can simply modify for instance the 'A' part as followed

for (p=0; p<=sip->a_order; p++)

for (q=0; (p+q)<=sip->a_order; q++){

  if(p+q < 1) continue; //constant term

  if(drop_linear && (p+q <2)) continue; //linear terms

  fuv +=  sip->a[p][q] * powu[p] * powv[q];

}

JE

Based on the discuss in the thread mentioned in #46, which I found because I had an image that wasn't solving locally but was solving on the web, I tried running astrometry.net locally on the same field, once with the 200 series and once with the 4200 series.

It solves with the 200 series but not the 4200 series (even if I downsample). Thought an example might help narrow down the problem.

Command line (using v0.50 installed via homebrew):

solve-field --scale-low 0.5 --scale-high 0.6 --scale-units arcsecperpix --continue --use-sextractor sa-110-010B.fit

Link to fits file: http://physics.mnstate.edu/craig/sa-110-010B.fit

Solving a time series of many images could be sped up significantly if multiple x,y lists could be submitted per job, since the job setup time dominates the process. For AstroImageJ purposes, optimum efficiency could be achieved if at least 8-10 x,y lists could be submitted per job set up.

Potential complications could occur if one or more x,y lists fail to solve. A method would need to be worked out to handle such cases for both the download of the WCS headers and the annotations.

The C code demands two digits in the HH of HH:MM:SS RA parsing. This caused a nastly little bug here
https://groups.google.com/forum/#!topic/astrometry/yeyI3Ry9Xlw
Should be an easy fix to the regex.

http://nova.astrometry.net/status/412916

Hit/miss: +ff+f+++c-f+f+f+-f+f+f++(best)ffff+c+++cf-ffff+++++c+-ffff++f-+c--ff+ff+-c----ff+ff+--+f--ff+ff+c++-c-ff-f
../net/testscript-astro: line 30: 31750 Segmentation fault      $BACKEND -v --to-stderr -c $CFG job.axy >> backend.log
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors

Solving a FITS file with thousand of NaNs values, I get the next error:

fitstable.c:956:read_array_into: Failed to read column from FITS file
fitstable.c:956:read_array_into: Failed to read column from FITS file
Failed to read xylist field.

However, with a FITS file with lower number of NaNs or if NaNs are converted to 0s, it works with no problem.

Then, are NaNs supported by Astrometry.net ?

Hi Dustin
I wander if there is a way to take into account pathological piels at the stage of index making and the solve actions?
JE

Hey @dstndstn, I'm taking you up on the offer for help. I'm trying to run solve-field on a file and it looks like my installation didn't work properly:

Traceback (most recent call last):
    from astrometry.util.shell import shell_escape
ImportError: No module named shell

Does this indicate what might not have worked in the install?

It's worth noting that I don't have root access on this machine, and writing to /usr/local/... gets denied.

If I try to make install again, I get

make[1]: *** [image2xy-files.o] Error 1
make[1]: Leaving directory `.../astrometry.net-0.50/blind'
make: *** [subdirs] Error 2

Hello dustin,

I have edited the .cfg file and specified only the required index files which can lessen the cpu time, however I run into this error , on the other hand with autoindex unhashed it takes very long to solve, please help

sameer@sameer-HP-406-G1-MT://media/sameer/Data/tobereduced/3c454/r4b2-lemontest/28oct06$ lemon astrometry --cores=8 -v --radius=3 --rak 343.4906163 --deck 16.1482111 -o "--downsample = 3" R*.fits R_28oct06/

Images will be saved to directory 'R_28oct06/'
Using a local build of Astrometry.net.
Doing astrometry on the 26 paths given as input.
Reading input file 1 of 1: "R_zs_3c454R0008.fits"...
Reading input file 1 of 1: "R_zs_3c454R0006.fits"...
Reading input file 1 of 1: "R_zs_3c454R0004.fits"...
Reading input file 1 of 1: "R_zs_3c454R0007.fits"...
Reading input file 1 of 1: "R_zs_3c454R0001.fits"...
Reading input file 1 of 1: "R_zs_3c454R0005.fits"...
Reading input file 1 of 1: "R_zs_3c454R0003.fits"...
Reading input file 1 of 1: "R_zs_3c454R0002.fits"...
Extracting sources...
Downsampling by 3...
Extracting sources...
Downsampling by 3...
Extracting sources...
Extracting sources...
Downsampling by 3...
Extracting sources...
Downsampling by 3...
Downsampling by 3...
Extracting sources...
Downsampling by 3...
Extracting sources...
simplexy: found 70 sources.
Downsampling by 3...
Extracting sources...
Downsampling by 3...
simplexy: found 75 sources.
simplexy: found 71 sources.
simplexy: found 77 sources.
simplexy: found 73 sources.
simplexy: found 76 sources.
simplexy: found 79 sources.
simplexy: found 78 sources.
Solving...
index.c:141:index_is_file_index: Index file /usr/local/astrometry/data//index-4201-.fits.quad.fits is not readable.
index.c:141:index_is_file_index: Index file /usr/local/astrometry/data//index-4201-.fits.quad.fits is not readable.
fitsbin.c:567:fitsbin_open: Failed to open file "/usr/local/astrometry/data//index-4201-.fits.skdt.fits"
starkd.c:244:my_open: Failed to open FITS file "/usr/local/astrometry/data//index-4201-.fits.skdt.fits"
index.c:401:index_reload: Failed to read star kdtree from file /usr/local/astrometry/data//index-4201-.fits.skdt.fits
engine.c:196:engine_add_index: Failed to load index from path /usr/local/astrometry/data//index-4201-.fits
Failed to add index "/usr/local/astrometry/data//index-4201-.fits".
index.c:141:index_is_file_index: Index file /usr/local/astrometry/data//index-4202-.fits.quad.fits is not readable.
index.c:141:index_is_file_index: Index file /usr/local/astrometry/data//index-4202-.fits.quad.fits is not readable.
fitsbin.c:567:fitsbin_open: Failed to open file "/usr/local/astrometry/data//index-4202-.fits.skdt.fits"
starkd.c:244:my_open: Failed to open FITS file "/usr/local/astrometry/data//index-4202-.fits.skdt.fits"
index.c:401:index_reload: Failed to read star kdtree from file /usr/local/astrometry/data//index-4202-.fits.skdt.fits
engine.c:196:engine_add_index: Failed to load index from path /usr/local/astrometry/data//index-4202-.fits
Failed to add index "/usr/local/astrometry/data//index-4202-.fits".
index.c:141:index_is_file_index: Index file /usr/local/astrometry/data//index-4203-.fits.quad.fits is not readable.
index.c:141:index_is_file_index: Index file /usr/local/astrometry/data//index-4203-.fits.quad.fits is not readable.
fitsbin.c:567:fitsbin_open: Failed to open file "/usr/local/astrometry/data//index-4203-.fits.skdt.fits"
starkd.c:244:my_open: Failed to open FITS file "/usr/local/astrometry/data//index-4203-.fits.skdt.fits"
index.c:401:index_reload: Failed to read star kdtree from file /usr/local/astrometry/data//index-4203-.fits.skdt.fits
engine.c:196:engine_add_index: Failed to load index from path /usr/local/astrometry/data//index-4203-.fits
Failed to add index "/usr/local/astrometry/data//index-4203-*.fits".

You must list at least one index in the config file (/usr/local/astrometry/bin/../etc/astrometry.cfg)

I want to use --dont-augment together with --keep-xylist path.axy to just get the result of star extraction. The solve-field help says -A / --dont-augment: quit after writing the unaugmented xylist but this seems to be not working. It tries to invoke astrometry-engine with the unaugmented .axy file and then obviously fails.

requested by Roberto Abraham

Hi,

for Debian, we will create packages for a number of different CPUs with both little and big endian. This requires currently to provide index data file sets for both endianness types, which is quite a waste of space. Swapping the bytes during package installation would make package verification complicated.
Also, the same index files cannot be used in a heterogeneous environment (f.e. arm and x86).

It would be great, if the index files were automatically swapped during read (if possible) instead of just printing an error.

I got a really weird error which happens in 0.50 when using --crpix-center and only on one of my machines, it's very strange. The error refers to a failing assertion at: https://github.com/dstndstn/astrometry.net/blob/0.50/util/fit-wcs.c#L300

I use indices 4111-4119. The input file is here: http://nova.astrometry.net/status/414582

The machine where it fails:

$ solve-field -v sources.axy --dir out --no-plots --pixel-error 10 --scale-low 54.8670002998 --scale-high 67.0596670331 --scale-units arcsecperpix --overwrite --no-fits2fits --no-remove-lines --no-tweak --crpix-center

[...]
Got a new best match: logodds 0.860712.
Got a new best match: logodds 2.46718.
object 41 of 50: 13 quads tried, 99 matched.
object 42 of 50: 17 quads tried, 129 matched.
Got a new best match: logodds 3.52938.
Got a new best match: logodds 19.138.
Trying to tune up this solution (logodds = 19.138; 2.04887e+08)...
solver_tweak2: setting orders 3, 3
tweak2: starting orders 3, 3
Tweak2: starting from WCS:
TAN-SIP Structure:
  crval=(47.124, -46.1361)
  crpix=(2836.52, 291.105)
  CD = (     0.015235      0.0029137 )
       (   -0.0029137       0.015235 )
  image size = (4272 x 2832)
  SIP order: A=3, B=3, AP=3, BP=3
  A =            0           0           0           0
                 0           0           0
                 0           0
                 0
  B =            0           0           0           0
                 0           0           0
                 0           0
                 0
  AP =            0           0           0           0
                 0           0           0
                 0           0
                 0
  BP =            0           0           0           0
                 0           0           0
                 0           0
                 0
  sqrt(det(CD))=55.8406 [arcsec]
Annealing: order 1, step 0, gamma = 1
288 reference sources within the image.
RoR: 1584.79
Test stars in RoR: 57 of 66
Good bins: 48 / 117; effA 1.20983e+07 of 4.96341e+06
Ref stars in RoR: 147 of 288
Logodds: 27.8559
25 matches, 9 distractors, 2 conflicts (at best log-odds); 66 field sources, 288 index sources
43 matches, 20 distractors, 3 conflicts (all sources)
Hit/miss: -+-+++++++++-c-c-++-++--++-++-++++++(best)-+c--++--+++---+--+++fffffffff
Moved quad center to (2889.9, 308.4)
tweak2: setting orders 1, 3
Moving tangent point to given CRPIX (0,0)
astrometry-engine: fit-wcs.c:300: fit_sip_wcs: Assertion `ok' failed.
solve-field.c:529:run_engine engine failed.  Command that failed was:
  /opt/astrometry/bin/astrometry-engine --verbose out3/sources.axy
 ioutils.c:561:run_command_get_outputs Command was killed by signal 6

And the one where it works:

$ solve-field -v sources.axy --dir out --no-plots --pixel-error 10 --no-fits2fits --no-remove-lines --scale-low 54.8670002998 --scale-high 67.0596670331 --scale-units arcsecperpix --no-fits2fits --no-remove-lines --overwrite --no-tweak --crpix-center

[...]
Got a new best match: logodds 0.860712.
Got a new best match: logodds 2.46718.
object 41 of 50: 13 quads tried, 99 matched.
object 42 of 50: 17 quads tried, 129 matched.
Got a new best match: logodds 3.52938.
Got a new best match: logodds 19.138.
Trying to tune up this solution (logodds = 19.138; 2.04887e+08)...
solver_tweak2: setting orders 3, 3
tweak2: starting orders 3, 3
Tweak2: starting from WCS:
TAN-SIP Structure:
  crval=(47.124, -46.1361)
  crpix=(2836.52, 291.105)
  CD = (     0.015235      0.0029137 )
       (   -0.0029137       0.015235 )
  image size = (4272 x 2832)
  SIP order: A=3, B=3, AP=3, BP=3
  A =            0           0           0           0
                 0           0           0
                 0           0
                 0
  B =            0           0           0           0
                 0           0           0
                 0           0
                 0
  AP =            0           0           0           0
                 0           0           0
                 0           0
                 0
  BP =            0           0           0           0
                 0           0           0
                 0           0
                 0
  sqrt(det(CD))=55.8406 [arcsec]
Annealing: order 1, step 0, gamma = 1
288 reference sources within the image.
RoR: 1584.79
Test stars in RoR: 57 of 66
Good bins: 48 / 117; effA 1.20983e+07 of 4.96341e+06
Ref stars in RoR: 147 of 288
Logodds: 27.8559
25 matches, 9 distractors, 2 conflicts (at best log-odds); 66 field sources, 288 index sources
43 matches, 20 distractors, 3 conflicts (all sources)
Hit/miss: -+-+++++++++-c-c-++-++--++-++-++++++(best)-+c--++--+++---+--+++fffffffff
Moved quad center to (2889.9, 308.4)
tweak2: setting orders 1, 3
Moving tangent point to given CRPIX (0,0)
# FAILURE HAPPENED HERE
Total weight: 25.6668
sip_compute-inverse_polynomials: A 1, AP 3
NX,NY 40,40
Applying shift of sx,sy = -0,-0 deg (-0,0 pix) to CRVAL and CD.
Annealing: order 1, step 1, gamma = 0.9
0 reference sources within the image.
After tuning, logodds = 19.138 (2.04887e+08)
Checking tuned result: logodds = 19.138 (2.04887e+08)
  log-odds ratio 19.138 (2.04887e+08), 23 match, 2 conflict, 12 distractors, 288 index.
  RA,Dec = (39.2586,-27.4069), pixel scale 55.8406 arcsec/pix.
object 43 of 50: 98 quads tried, 747 matched.
object 44 of 50: 144 quads tried, 1073 matched.
object 45 of 50: 223 quads tried, 1629 matched.
object 46 of 50: 258 quads tried, 1889 matched.
[...]

The first machine without --crpix-center:

$ solve-field -v sources.axy --dir out --no-plots --pixel-error 10 --no-fits2fits --no-remove-lines --scale-low 54.8670002998 --scale-high 67.0596670331 --scale-units arcsecperpix --no-fits2fits --no-remove-lines --overwrite --no-tweak

Got a new best match: logodds 0.860712.
Got a new best match: logodds 2.46718.
object 41 of 50: 13 quads tried, 99 matched.
object 42 of 50: 17 quads tried, 129 matched.
Got a new best match: logodds 3.52938.
Got a new best match: logodds 19.138.
Trying to tune up this solution (logodds = 19.138; 2.04887e+08)...
solver_tweak2: setting orders 3, 3
tweak2: starting orders 3, 3
Tweak2: starting from WCS:
TAN-SIP Structure:
  crval=(47.124, -46.1361)
  crpix=(2836.52, 291.105)
  CD = (     0.015235      0.0029137 )
       (   -0.0029137       0.015235 )
  image size = (4272 x 2832)
  SIP order: A=3, B=3, AP=3, BP=3
  A =            0           0           0           0
                 0           0           0
                 0           0
                 0
  B =            0           0           0           0
                 0           0           0
                 0           0
                 0
  AP =            0           0           0           0
                 0           0           0
                 0           0
                 0
  BP =            0           0           0           0
                 0           0           0
                 0           0
                 0
  sqrt(det(CD))=55.8406 [arcsec]
Annealing: order 1, step 0, gamma = 1
288 reference sources within the image.
RoR: 1584.79
Test stars in RoR: 57 of 66
Good bins: 48 / 117; effA 1.20983e+07 of 4.96341e+06
Ref stars in RoR: 147 of 288
Logodds: 27.8559
25 matches, 9 distractors, 2 conflicts (at best log-odds); 66 field sources, 288 index sources
43 matches, 20 distractors, 3 conflicts (all sources)
Hit/miss: -+-+++++++++-c-c-++-++--++-++-++++++(best)-+c--++--+++---+--+++fffffffff
Moved quad center to (2889.9, 308.4)
tweak2: setting orders 1, 3
Total weight: 25.6668
sip_compute-inverse_polynomials: A 1, AP 3
NX,NY 40,40
Applying shift of sx,sy = -0.0530491,0.0125858 deg (-3.54184,0.134672 pix) to CRVAL and CD.
Annealing: order 1, step 1, gamma = 0.9
274 reference sources within the image.
RoR: 1626.41
Test stars in RoR: 57 of 66
Good bins: 48 / 117; effA 1.20983e+07 of 4.96341e+06
Ref stars in RoR: 142 of 274
Logodds: 40.5695
37 matches, 10 distractors, 1 conflicts (at best log-odds); 66 field sources, 274 index sources
46 matches, 18 distractors, 2 conflicts (all sources)
Hit/miss: -++++++++++++--+-++-++--++-++-++++++-+c--++--+++(best)-c-+--+++fffffffff
Moved quad center to (2921.1, 307.3)
tweak2: setting orders 1, 3
Total weight: 28.6816
sip_compute-inverse_polynomials: A 1, AP 3
NX,NY 40,40
Applying shift of sx,sy = -0.0292478,-0.00214954 deg (-1.80574,-0.523363 pix) to CRVAL and CD.
Annealing: order 1, step 2, gamma = 0.81
273 reference sources within the image.
RoR: 1629.5
[...]

I really need to get this sorted as it happens on a production machine whereas on my local one it works. If it helps, the -v -v -v log of the failing machine: https://www.dropbox.com/s/a4l0yd7gj9ehruc/astrometry_failing.txt?dl=0

Refactor solve-field.c into command-line parsing and running parts; add python interface to the running part.

Hello, I tryied the UCAC4 guide in 5 computers, with different OS and in different networks. I all my tries the files z431 and z664 returns errors when I run ucac4tofits. The error returned is:

ucac4tofits.c:145:main: bzip2 error: code 4

My actual solution is manual download and compress the files, replacing this 2 files with errors. Then, the program runs normally.

As saturated stars degrade the accuracy of the solution, maybe they could be excluded during source extraction?

A simple way could be to allow the user to define a threshold in ADU via a command line parameter that safely marks saturated pixels in the respective image(s). In this way, every detected source that contains a pixel with a value higher than this threshold could be discarded.

Would be useful :-).

As mentioned in astrometry.net google group (https://groups.google.com/forum/#!topic/astrometry/UyoiamWiX1M)
It would be great if astrometry.net API could provide annotations for deepsky objects in images with a bigger fov than 10° and constellations.

Hello,

In the process of (ra,dec) to pixel coordinates transform (in a tan transform) there is a step where the 3d unit vector defined by (ra,dec)-direction is projected on tangent plane. Fine in principle, but in the tan_xyzarr2iwc function after projection (ie. star_coords function), there is a step that I do not anderstand at all, why the intermediate coordinate (iwc) should be transfomed rad2deg ??????

JE

anbool tan_xyzarr2iwc(const tan_t* tan, const double* xyz,
double* iwcx, double* iwcy) {

double xyzcrval[3];
// FIXME be robust near the poles
// Calculate intermediate world coordinates (x,y) on the tangent plane
radecdeg2xyzarr(tan->crval[0], tan->crval[1], xyzcrval);

if (!star_coords(xyz, xyzcrval, !tan->sin, iwcx, iwcy))
    return FALSE;

*iwcx = rad2deg(*iwcx);
*iwcy = rad2deg(*iwcy);
return TRUE;

}


Hi Dustin et al,

It looks like fitsio.h used to be in one of the sub-directories blind, cfitsio, or util (or possibly I just copied it there in the past!) In any case, it isn't being found naturally.

Cheers,
Doug

root@cepheid2:/usr/local/src/astrometry.net-0.50# make
make -C qfits-an
make[1]: Entering directory /1/local/src/astrometry.net-0.50/qfits-an' make[1]: Nothing to be done forall'.
make[1]: Leaving directory /1/local/src/astrometry.net-0.50/qfits-an' make -C gsl-an make[1]: Entering directory/1/local/src/astrometry.net-0.50/gsl-an'
make[1]: Nothing to be done for all'. make[1]: Leaving directory/1/local/src/astrometry.net-0.50/gsl-an'
make -C util
make[1]: Entering directory /1/local/src/astrometry.net-0.50/util' make[1]: Nothing to be done forall'.
make[1]: Leaving directory /1/local/src/astrometry.net-0.50/util' make -C catalogs make[1]: Entering directory/1/local/src/astrometry.net-0.50/catalogs'
make[1]: Nothing to be done for all'. make[1]: Leaving directory/1/local/src/astrometry.net-0.50/catalogs'
make -C libkd
make[1]: Entering directory /1/local/src/astrometry.net-0.50/libkd' make[1]: Nothing to be done forall'.
make[1]: Leaving directory /1/local/src/astrometry.net-0.50/libkd' make -C blind make[1]: Entering directory/1/local/src/astrometry.net-0.50/blind'
gcc -g -Wall -ffinite-math-only -fno-signaling-nans -pthread -march=native -O3 -fomit-frame-pointer -DNDEBUG -fpic -Winline -I../include -I../include/astrometry -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -I../catalogs -I../include -I../include/astrometry -I../gsl-an -I../include -I../include/astrometry -I../gsl-an -I../util -I. -I/usr/include/cairo -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/pixman-1 -I/usr/include/freetype2 -I/usr/include/libpng12 -c -o image2xy-files.o image2xy-files.c
image2xy-files.c:32:20: fatal error: fitsio.h: No such file or directory
compilation terminated.
make[1]: *** [image2xy-files.o] Error 1
make[1]: Leaving directory `/1/local/src/astrometry.net-0.50/blind'
make: *** [subdirs] Error 2

The svn version info doesn't work any more...