Comments (8)
340 new observations since 2024-05-30
That explains! CI got messed up sometimes between the 31st of May and the 7th of June :)
from astroquery.
After 3+ hours and 1.8 Gb, I've cancelled the test run, it was still at 50% and seemed to be stuck in a JWST download (using M51 is probably not the best test case).
Then I noticed that the remote-data tests in CI are run including the ones already marked with the "bigdata" pytest marker. So, as a workaround I'll skip all the bigdata tests. Nevertheless, even with that workaround, there may be room for improving the tests and cutting back on data sizes.
from astroquery.
Hmm... One or more missions have most likely released new data in the areas we are downloading, resulting in these huge download sizes. We should be able to find significantly smaller download sizes than that by switching the area queried.
I should have some time to look at this in July or August, so I recommend disabling the problematic tests for now. Of course, you are more than welcome to try to address the issue as well :)
from astroquery.
I went ahead and disabled running the tests marked as "bigdata", so the immediate urgency has been resolved, and the job is now running without running into space or timeout issues.
That being said, the test is also running really slowly, which maybe a separate problem (I mean it shouldn't take 3+ hr to download ~2Gb of data)
from astroquery.
Indeed, it should be much faster than that. Did you happen to notice which missions were slow? I did some quick spot tests, including on JWST data, and did not notice any speed issues or timeouts.
@jespinosaar, are you aware of any current issues with the download speed or timeouts of the archives?
from astroquery.
It was stuck on JWST-MID-IR
in the test_esasky_get_images
test
from astroquery.
Oh, and here is a CI run that ended up in timeout: https://github.com/astropy/astroquery/actions/runs/9524655265/job/26257814175#step:5:2467
from astroquery.
Ok, I see the problem. There are 394 observations in that area now. 340 new observations since 2024-05-30, and another batch of 52 observations was conducted 1 year ago. When this test was created, there were only 2 observations in the area.
from astroquery.
Related Issues (20)
- Splatalogue.query_lines() seems to be broken? HOT 2
- Slow Simbad.query_objects & IRSA.query_region searches HOT 6
- Use plot directive to generate figures
- modulate verbosity of astroquery.mast.Observations.download_products() HOT 1
- Add support for upload tables for queries in ALMA
- BUG: CI -- no space left on device HOT 2
- ESA Herschel download_data connection error HOT 2
- MAINT: retire/rename exceptions.TimeoutError in favour of requests
- BUG: alma test downloads 850+Mb of data HOT 8
- How to download Level 1 ancillary (*cbcd.fits and *cbunc.fits) from IRSA?
- Vizier.query_region() returns incorrect index for some catalog HOT 3
- CI: rerun failing tests at the end
- Bug in rows limitation in module vizier HOT 2
- Unexpected behavior when passing `obj_names` to `SDSS.query_region()`
- Handling No astronomical object found response HOT 5
- IRSA: adding SSA method HOT 1
- Astroquery.gaia API Error: RA cannot be converted to hour or hourangle. HOT 2
- MAST cloud dataset doesn't show level 3 data products
- SimbadClass got an unexpected keyword argument HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from astroquery.