AWS S3 folders at http://awsopendata.s3-website-us-west-2.amazonaws.com/noaa-gfs/ not explained about open-data-registry

[jflasher]

Hi @nehgu, thanks for reaching out. I believe you're looking at an older version of this dataset, could you please let me know where you found a link to this and I'll have it updated? https://registry.opendata.aws/noaa-gfs-bdp-pds/ should have more information and a better source of the data.

[nehgu]

This means there are 2 versions of the same dataset live out there:[https://registry.opendata.aws/noaa-gfs-pds/]
I have been using above link for a month now to download and process weather data. Here is the S3 browser which worked really well:[http://awsopendata.s3-website-us-west-2.amazonaws.com/noaa-gfs/]

Is "noaa-gfs-pds" going to get discontinued and when?

[jflasher]

Hi @nehgu, the https://registry.opendata.aws/noaa-gfs-bdp-pds/ dataset would be the better one to use as it's managed directly by NOAA. Can you tell me where you found a link to the other dataset so we can update that? Thanks!

[nehgu]

Pls see the attached screenshot, they are both on AWS, one is through Unidata though.

[jflasher]

Hi @nehgu, thanks for the info. You can use either of these datasets, depending on whichever works best for you. Note that there is currently a switchover in operational GFS model that is occurring at NOAA. To the initial question, the folders are related to the parameters from the GFS model output and then the vertical height. As this is a NOAA dataset, their sites will provide the best information on what those are. https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs should be a good starting point. Hope that helps!

[nehgu]

'Soil Temperature' and 'Downward Solar Radiation' are no longer available in 's3://noaa-gfs-bdp-pds' or 's3://noaa-gfspara-pds' while it was available in 's3://noaa-gfs-pds'. Also is the filename and prefix subject to change in the currently available s3 buckets?
Current names are like "s3://noaa-gfspara-pds/gfs.2019061706/" where "06" is the cycle getting appended to date, but for "s3://noaa-gfspara-pds/gfs.20190612/00" cycle is another prefix.

[jflasher]

Hi @nehgu, AWS doesn't own or manage the ingest of the data, so I can't answer these questions unfortunately. For any questions related to the noaa-gfs-pds or noaa-gfspara-pds buckets, I would reach out to the contact email listed on the page.