Comments (4)
possible duplicate of: #715 (there I also saw the access URLs in the thredds html but they were missing with siphon. Both situations seem to involved a nested catalog too)
from siphon.
Agreed with @tdrwenski that this smells similar. I'd love to help here but will need a nudge in a direction.
from siphon.
Adding a little more detail here, hopefully useful.
For reference, a working example:
from siphon.catalog import TDSCatalog
thredds_url_fmrc = "https://tds.hycom.org/thredds/catalog/GLBy0.08/expt_93.0/FMRC/runs/catalog.xml"
cat_fmrc = TDSCatalog(thredds_url_fmrc)
ds_fmrc = cat_fmrc.datasets[0].remote_access()
where cat_fmrc.datasets
gives:
['GLBy0.08_930_FMRC_RUN_2023-09-19T12:00:00Z', 'GLBy0.08_930_FMRC_RUN_2023-09-18T12:00:00Z', 'GLBy0.08_930_FMRC_RUN_2023-09-17T12:00:00Z', 'GLBy0.08_930_FMRC_RUN_2023-09-16T12:00:00Z', 'GLBy0.08_930_FMRC_RUN_2023-09-15T12:00:00Z', 'GLBy0.08_930_FMRC_RUN_2023-09-14T12:00:00Z']
and cat_fmrc.datasets[0].access_urls
gives:
{'OPENDAP': 'https://tds.hycom.org/thredds/dodsC/GLBy0.08/expt_93.0/FMRC/runs/GLBy0.08_930_FMRC_RUN_2023-09-19T12:00:00Z',
'NetcdfSubset': 'https://ncss.hycom.org/thredds/ncss/grid/GLBy0.08/expt_93.0/FMRC/runs/GLBy0.08_930_FMRC_RUN_2023-09-19T12:00:00Z',
'WMS': 'https://wms.hycom.org/thredds/wms/GLBy0.08/expt_93.0/FMRC/runs/GLBy0.08_930_FMRC_RUN_2023-09-19T12:00:00Z',
'WCS': 'https://wcs.hycom.org/thredds/wcs/GLBy0.08/expt_93.0/FMRC/runs/GLBy0.08_930_FMRC_RUN_2023-09-19T12:00:00Z'}
and
In [8]: cat_fmrc.services
Out[8]: [<siphon.catalog.CompoundService at 0x7efc9b79fcd0>]
In [9]: cat_fmrc.services[0]
Out[9]: <siphon.catalog.CompoundService at 0x7efc9b79fcd0>
In [10]: cat_fmrc.services[0].services[0]
Out[10]: <siphon.catalog.SimpleService at 0x7efc9b79fc40>
In [11]: cat_fmrc.services[0].services[0].service_type
Out[11]: 'OPENDAP'
Now the same stuff but for the non-working example shown in the first post (not repeated here):
cat_hind.datasets
gives:
['GLBy0.08_expt_93.0 (ssh, ts3z, and uv3z aggregated)', 'GLBy0.08_expt_93.0_ssh (sea_surface_elevation)', 'GLBy0.08_expt_93.0_ts3z (sea_water_temperature and sea_water_salinity)', 'GLBy0.08_expt_93.0_uv3z (eastward_sea_water_velocity and northward_sea_water_velocity)', 'GLBy0.08_expt_93.0_ice (sst, sss, ssu, ssv, sic, sih, siu, siv, surtx, surty)', 'GLBy0.08_expt_93.0_sur (qtot, emp, steric_ssh, ssh, u_barotropic_velocity, v_barotropic_velocity, surface_boundary_layer_thickness, mixed_layer_thickness)']
and cat_hind.datasets[i].access_urls
returns {}
for any i
.
Then cat_hind.services
returns:
[<siphon.catalog.CompoundService at 0x7f2e9a0bf910>,
<siphon.catalog.CompoundService at 0x7f2e9a0bf100>,
<siphon.catalog.SimpleService at 0x7f2e9a0bf790>]
where cat_hind.services[2].service_type
returns 'FTP'
and cat_hind.services[1].services[0].service_type
returns 'OPENDAP'
.
from siphon.
In catalog.make_access_urls()
there are two clauses which, in the non-working example discussed here, are never satisfied or are empty:
- The first is on line 570:
if service_name in all_service_dict:
. This is never satisfied asservice_name
isNoneType
.cat_hind.metadata
does not contain an entry forserviceName
(per line 562).
Note that:
In [79]: all_service_dict
Out[79]:
{'all': <siphon.catalog.CompoundService at 0x7f2e9a0bf910>,
'all-ftp': <siphon.catalog.CompoundService at 0x7f2e9a0bf100>,
'ftp': <siphon.catalog.SimpleService at 0x7f2e9a0bf790>,
'ncdods': <siphon.catalog.SimpleService at 0x7f2e9a0bf550>,
'ncss': <siphon.catalog.SimpleService at 0x7f2e9a0bf7f0>,
'wms': <siphon.catalog.SimpleService at 0x7f2e9a0bf340>,
'wcs': <siphon.catalog.SimpleService at 0x7f2e9a0bf940>}
In [80]: all_service_dict['ncdods'].service_type
Out[80]: 'OPENDAP'
- The second is on line 585:
for service_type in self.access_element_info:
. This attribute is empty for allcat_hind.datasets[i]
. I'll note that in the working example, this attribute is also empty for allcat_fmrc.datasets[i]
.
Together, this results in the access_urls
never being populated. I'll work on this some more in the coming days.
from siphon.
Related Issues (20)
- issues getting CFS point data
- Update dataset in Wyoming upper air example
- ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) HOT 2
- Endian-ness issues when selecting on a coordinate HOT 3
- Datasets in nested catalogs have empty access_urls
- TDSCatalog does not include base path in access_url HOT 2
- NDBC Mid Great Lakes Station Do Not Show Up HOT 1
- WyomingUpperAir Missing Station Information
- Requesting a new release HOT 1
- Exception handling needed in _get_data method of WyomingUpperAir for missing or erroneous metadata HOT 2
- Protobuf module started making it impossible to access the UCAR THREDDS server using siphon TDSCatalog HOT 4
- TDSCatalog misses the last months entry HOT 2
- Add a walk function for navigating THREDDS catalogue HOT 2
- how to add the Specify additional variables you want to include HOT 1
- Descriptors cannot be created directly
- Wyoming Upper Air Data Request fails for 4/19/2024 00Z, returning only data for OUN HOT 1
- TDSCatalog not returning all datasets HOT 2
- CDMRemote xarray wrapper broken with xarray 2024.03.0
- Writing out a siphon'd xarray dataset as a netCDF HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from siphon.