spacetelescope / astrocut Goto Github PK
View Code? Open in Web Editor NEWTools for making image cutouts from sets of TESS full frame images
Home Page: https://astrocut.readthedocs.io
Tools for making image cutouts from sets of TESS full frame images
Home Page: https://astrocut.readthedocs.io
This might need to be updated to include a conditional involving the upper boundaries (in X and Y) of the FFIs. From what I understand, this is catching for cases where the lower limit of the cutout limits falls below 0 in either axis, but we don't have a conditional for when the upper limit goes beyond 2048 or whatever the FFI image size is:
Line 192 in 311488d
Looking at TESSCut products, the data array includes columns 'POS_CORR1' and 'POS_CORR2' but all the values are zero. For a typical TPF, these columns give the per-image pointing offsets relative to the WCS information, and they are important for matching source locations between TESS and other data sets. Is it possible to include the correct differential velocity aberration data in the TESSCut products? I have code that uses this information to analyze TPFs that I hope to use on TESSCut image arrays as well.
Per the astropy documentation:
When combining intervals and stretches, the stretch object must come before the interval object.
Currently they are applied in the opposite order.
Could this repo get a tag called "mast" so that a URL that looks like https://github.com/search?q=topic%3Amast+org%3Aspacetelescope would allow this to show up as public software relevant for MAST users?
On the s3-support
branch for making direct s3 cutouts, seeing an error with this set of input data:
from astrocut import CutoutFactory
astrocut_cutter = CutoutFactory()
cube_file = 's3://stpubdata/tess/public/mast/tess-s0030-4-3-cube.fits'
coordinates = '102.7 -70.5'
cutout_size = (3, 3)
output_path = '/tmp'
astrocut_cutter.cube_cut(cube_file, coordinates, cutout_size, output_path=output_path)
Traceback:
Traceback (most recent call last):
File "/Users/bfalk/repos/astrocut/astrocut/tests/test_small_s3_cutout.py", line 12, in <module>
astrocut_cutter.cube_cut(cube_file, coordinates, cutout_size, output_path=output_path)
File "/Users/bfalk/repos/astrocut/astrocut/cube_cut.py", line 736, in cube_cut
self._parse_table_info(cube.table.data, verbose)
File "/Users/bfalk/repos/astrocut/astrocut/cube_cut.py", line 930, in table
return self.loop.run_until_complete(self.cube_async.table)
File "/Users/bfalk/miniconda3/envs/astrocut/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
return future.result()
File "/Users/bfalk/repos/astrocut/astrocut/cube_cut.py", line 1018, in table
self._table = await self.read_table()
File "/Users/bfalk/repos/astrocut/astrocut/cube_cut.py", line 1032, in read_table
hdu2_str = await self._read_block(hdu2_offset, length=None)
File "/Users/bfalk/repos/astrocut/astrocut/cube_cut.py", line 1064, in _read_block
resp = await self.s3_client.get_object(
File "/Users/bfalk/miniconda3/envs/astrocut/lib/python3.9/site-packages/aiobotocore/client.py", line 187, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (InvalidRange) when calling the GetObject operation: The requested range is not satisfiable
Specifying a local file for the cube_file
, I don't see this error, and the cutout succeeds.
I'm wondering if this particular sector/file needs some special casing when reading in the table information?
@barentsen wondering if you might be able to take a look at this at some point?
Cutouts are able to be made on FITS Cut out of the box, but future mission like Roman will be housing data in ASDF files. Find a way to support this. ASDF Cut?
Currently there's no error handling for cases where a user feed TicaCubeFactory
SPOC FFIs and vice versa. Add some explicit error handling for this.
The astrocut.fits_cut
method fails to write the cutout file to the directory specified by the output_dir
argument. Instead the cutout file always appears to be written to the directory from which the image was read. A sample test file is attached which demonstrates this (sadly renamed as a '.txt' file to fool stupid GitHub attachment checks).
Below is the output from a successful test run. Note that the line:
OUTPUT_DIR=/tmp/outdir
is output from within _save_multiple
method of cutouts.py
, indicating that the method is receiving the correct value for the output_dir
parameter. However, the line:
co_file=/tmp/imgdir/m13_250.417651_36.458904_100x100_astrocut.fits
is printing the paths (in this case, just one) in the list returned by cutouts.fits_cut
and the path is pointing to the directory from which the image was read.
============================= test session starts ==============================
platform darwin -- Python 3.7.4, pytest-5.2.2, py-1.8.0, pluggy-0.13.0
Running tests with Astropy version 3.2.2.
Running tests in astrocut/tests/test_fits_output.py.
Date: 2019-10-27T22:12:47
Platform: Darwin-17.7.0-x86_64-i386-64bit
Executable: /usr/local/src/anaconda3/envs/astrocut/bin/python
Full Python Version:
3.7.4 (default, Aug 13 2019, 15:17:50)
[Clang 4.0.1 (tags/RELEASE_401/final)]
encodings: sys: utf-8, locale: UTF-8, filesystem: utf-8
byteorder: little
float info: dig: 15, mant_dig: 15
Numpy: 1.17.3
Scipy: 1.3.1
Matplotlib: not available
h5py: not available
Pandas: not available
astropy_helpers: 3.2.1
Using Astropy options: remote_data: none.
rootdir: /private/var/tmp/astrocut-test-u09wli0y/lib/python3.7/site-packages, inifile: setup.cfg
plugins: arraydiff-0.3, astropy-header-0.1.1, doctestplus-0.4.0, openfiles-0.4.0, remotedata-0.3.2
collected 1 item
OUTPUT_DIR=/tmp/outdir
astrocut/tests/test_fits_output.py
co_file=/tmp/imgdir/m13_250.417651_36.458904_100x100_astrocut.fits
<_io.TextIOWrapper name='<stderr>' mode='w' encoding='UTF-8'>
Following the work on PR #56
@falkben has a couple suggestions on how the GitHub Actions workflow can be improved for astrocut:
To whom it may concern,
If you are using https://github.com/astropy/ci-helpers in your appveyor.yml , please know that the Astropy project has dropped active development/support for Appveyor CI. If it still works, good for you, because we did not remove the relevant files (yet). But if it ever stops working, we have no plans to fix anything for Appveyor CI. Please consider using native Windows support other CI, e.g., Travis CI (see https://docs.travis-ci.com/user/reference/windows/). We apologize for any inconvenience caused.
If this issue is opened in error or irrelevant to you, feel free to close. Thank you.
In line
Line 189 in 311488d
Tyler Desjardins mentions that we should consider moving emails from help[at]stsci.edu
to point to the web portal where possible and appropriate. For HST (or any non-JWST), it is https://hsthelp.stsci.edu . For JWST, it is https://jwsthelp.stsci.edu . Please update info in setup.py
, setup.cfg
, documentation, etc as appropriate.
Please close this issue if it is irrelevant to your repository. This is an automated issue. If this is opened in error, please let pllim know!
APE 17 is accepted. Look out for a "conversion guide" that Tom Robitaille will send out to astropy dev mailing list in the near future.
Address final todo items in test_cube_cut.py
Suggestion from @falkben RE: Pull Request #59
"Can we use class inheritance here? It looks like most of the changes would just be in the class attributes. If we make those changes, I think we can make these changes much simpler and should be easier to maintain."
Also some commentary I initially made in TicaCubeFactory class:
NOTE: Later down the line
we might want to generalize CubeFactory so that we no longer need
a separate class for the TICA cubes.
Modifying the __init__
in CubeFactory so that it can switch parameters based on whether
we want cubes for TICA or TESS will probably be the route I take.
Or see how this is generalized when Zcut wraps CubeFactory?
"Cutout contains to data!" in cutout.py. Should be "Cutout contains no data." "To data" occurs in four places.
As recommended by @chamblee-st , it would make the cutout tests more modular
Right now fits_cut
and img_cut
assume the list of images they are given all share the same WCS and pixel scale, but no actual checking is done. A minor improvement would be to add checking and raise an error is the images don't match. A larger improvement would be to implement optional resampling.
In the astrocut document
https://astrocut.readthedocs.io/en/latest/astrocut/
the example given in the "Making Image Cube" section is incomplete and thus can not execute.
The line
input_files = glob("data/*ffic.fits")
refers to the data subdirectory which does not exist which causes input_files to be an empty list.
Suggest solution: Test to see if data/*ffic.fits files exist before defining input_files. If no such files exist, then create a small set of *ffic.fits images in a directory called data.
For easier troubleshooting and checks in the future.
Greetings! First, thank you so very much for developing astrocut / tesscut! It's a very useful tool -- I've just started using it, and sailing has overall been very smooth.
Recently though, I believe I encountered a potential bug!
Behavior: using the TESScut web service, when I enter (RA, dec) = (85.8, -1.5) as in the attached screenshot, I am told "Cutout Error: No FFIs found for input target. Cutout not performed." However, the location specified is definitely in sector 6, camera 1, ccd 1 (see image on the right, with the lovely M42). This error seems to be returned for a number of the celestial coordinates on this CCD.
I checked the holdings website, which indicated that sector 6 should be fine to query.
My plan is to write up a CubeFactory
+ CutoutFactory
routine to investigate a bit more closely, since I figure this may be because the backend database might not yet be updated through Sector 6? Nonetheless, this seemed worth reporting, since I am likely not the only one!
Thank you again your work developing and maintaining this project!
--Luke
The Python 3.7 with dev astropy tests are failing, this needs to be fixed before the next astropy release.
Issue to capture small non-urgent things that I notice while adding new functionality:
_get_cutout_limits
docstring parameters is wrong (center_coord is no longer a param)
target_pixel_file = os.path.join(output_path, tpf_name)
is indented one too far
Get rid of the overwrite=True or make it an argument in cube_cut
?
Make cube_cut
handle x vs y consistent with new fits_cut
functionality (which is correct)
Update file creation date to reflect cutout date not cube date
extname
argument in add_keywords
Thank you for making and maintaining TESSCut, it is a great service!
I've come across an issue making light curves for moving objects which I think is a function of the name used. The issue might be when forward slashes are in the object name, somewhere in the parsing of that information?
As an example, if I try to make a light curve for the comet C/2014 UN271 with the following (or through the web interface)
https://mast.stsci.edu/tesscut/api/v0.1/moving_target/astrocut?obj_id=C%2F2014%20UN271&y=20&x=20&units=px§or=30
I get a Cutout Error: Cutout could not be performed
response. However, if I use an alternate name for the same object, Bernardinelli-Bernstein, the equivalent command (https://mast.stsci.edu/tesscut/api/v0.1/moving_target/astrocut?obj_id=Bernardinelli-Bernstein&y=20&x=20&units=px§or=30
) works perfectly.
My assumption from this is that it must be something in the object name, and the forward slash is the most plausible suspect to me. Have you come across this before?
Rename CubeFactory
to SpocCubeFactory
? Or eventually consolidate the cube factory classes into 1 for TICA and SPOC.
When fit_wcs_from_points
went into astropy its call pattern changed, the local version in astrocut needs to be updated to match so that astrocut can use the astropy version if it is available and it's own if it isn't.
Additionally testing needs to be done to make sure the accuracy of the fit wcs remains the same.
Add header keyword value quality checks to unit tests for make_cube.py
.
There was an incident where the EXT1 header keyword NAXIS2
did not reflect the correct number of FFIs that were added to the cube. We are still unsure of what happened, but as a preventative measure, it would be good to have some checks in place to verify the quality of the primary and image header keyword values.
Hello!
We've come across an issue in the timestamps in TessCut frames (TessCut is a wonderful service, by the way!)
When I look the astrocut file timestamps, there is a digitization that I can see in them when we plot the difference in time between consecutive frames. We think this is a floating point rounding error in the timestamps, somewhere they are being stored as 32-bit floats and since the time values have four digits before the decimal place, this limits the precision to ~10 seconds on any individual data point.
You can see what I mean in figure form at afeinstein20/eleanor#119. The code to reproduce this is below:
tc = fits.open('tess-s0001-4-3_97.842681_-68.044906_10x15_astrocut.fits')
plt.plot(np.diff(86400*tc[1].data['time']), '.')
plt.ylim(1795, 1805)
plt.xlabel('Cadence', fontsize=20)
plt.ylabel('Delta Time (sec)', fontsize=20)
The TASOC light curve times look proper, so we think it's a processing issue along the way rather than the way the telescope is recording data. If you think it's something upstream from astrocut where the bug is then I'm happy to work my way up the chain until we figure out where this is being induced.
Thanks so much!
Ben
Hello,
I'm trying to create a cutout from a cfht megaprime image. I've confirmed the target is on one of the chips, but the software keeps telling me it isn't. I've tried using the extension = 'all' option, but I assume I'm just not handling the multi-extension file correctly.
The target appears on different chips in some of the images, so I was hoping I wouldn't need to determine which one (which extension) I need on each image.
Let me know if you have any suggestions, or if you've seen this issue before.
Thank you and all the best,
Scott
I'm currently trying to run the code from a Win10 machine, Anaconda, in a Jupyter notebook. The image (~340MB) can be obtained from here
from astrocut import fits_cut
from astropy.io import fits
from astropy.coordinates import SkyCoord
input_files = ['714745p.fits.fz.part']
center_coord = SkyCoord.from_name('[HS80] B337')
cutout_size = 500
cutout_file = fits_cut(input_files, center_coord, cutout_size, extension = 'all', single_outfile=True)
print(cutout_file)
This may be a known issue, but I thought it'd be worth highlighting the following caveat:
TESSCut appears to resolve names to their J2000.0 position. For example, Proxima Cen:
This is a caveat because nearby stars such as Proxima Cen have a very large proper motion, so anyone clicking "Download FFI Cutout" after having entered the name of this target may be surprised to find that the object is not in the center of the TPF. This is because its high proper motion since J2000 has moved it slightly out of the cutout! Example for Proxima Cen:
This may not be important to fix, because it only affects a small number of high proper motion targets, but some solutions could be to:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.