MATLAB Interface for NOAA Data-major issue about noaa

[cfletchg]

I have contacted NOAA regarding this.

[steveforczyk]

Chris: Appreciate any help you can give me. To summarize the problem I can download data using this interface but the data for Jan and nearly all of Feb each month is missing. When I check the datasets on line in PDF format I can SEE the data for the missing months From: "Chris Garvin" ***@***.***> To: "mathworks/noaa" ***@***.***> Cc: "Stephen M Forczyk" ***@***.***>, "Author" ***@***.***> Sent: Tuesday, November 22, 2022 1:14:27 PM Subject: Re: [mathworks/noaa] MATLAB Interface for NOAA Data-major issue (Issue #1) I have contacted NOAA regarding this. — Reply to this email directly, [ #1 (comment) | view it on GitHub ] , or [ https://github.com/notifications/unsubscribe-auth/AIANZIIZ2LR27HIXDBUFMYDWJULRHANCNFSM6AAAAAASH54ADQ | unsubscribe ] . You are receiving this because you authored the thread. Message ID: < ***@***.*** >

[cfletchg]

Hi Stephen, I have verified that I can get other data for January and February to verify that those input dates are not causing an issue with the interface. I will post again when I hear from NOAA. I'm not sure how quickly they respond to queries on the API.

[steveforczyk]

Chris: I am glad of the quick response. Could you send me an example of this so I can try it out. I have attached a sample script. This is trying to get data from the Huntsville Airport Site. The key code is which is from lines 106-116 of the script. the station id is stationidstr= "GHCND:USW00003856" ; I am running under R2022b for y = 2012:2019 annualTMAXData = [annualTMAXData;getdata(n, "GHCND" , ... datetime(strcat(num2str(y), "-01-01" )), ... datetime(strcat(num2str(y), "-12-31" )), ... stationid=stationidstr, ... datatypeid = "TMAX" , ... locationid = "FIPS:01" , ... limit = 1000, ... units = unitstr)]; end Greatly appreciate any help you can give me. In understand this is now the holiday so I don't expect and immediate answer!!! From: "Chris Garvin" ***@***.***> To: "mathworks/noaa" ***@***.***> Cc: "Stephen M Forczyk" ***@***.***>, "Author" ***@***.***> Sent: Wednesday, November 23, 2022 8:36:04 AM Subject: Re: [mathworks/noaa] MATLAB Interface for NOAA Data-major issue (Issue #1) Hi Stephen, I have verified that I can get other data for January and February to verify that those input dates are not causing an issue with the interface. I will post again when I hear from NOAA. I'm not sure how quickly they respond to queries on the API. — Reply to this email directly, [ #1 (comment) | view it on GitHub ] , or [ https://github.com/notifications/unsubscribe-auth/AIANZIIBF6FEAMKNQAKBW3DWJYTVJANCNFSM6AAAAAASH54ADQ | unsubscribe ] . You are receiving this because you authored the thread. Message ID: < ***@***.*** >

[cfletchg]

Hi Stephen,

I verified that I can get January data for other datasets and datatype to make sure the interface wasn't having an issue with those date inputs. But I'm guessing that data won't be of much value to you.

I tried the following code to see what datatypes are available for that station and date range:

d = getdata(n,"GHCND", ...
datetime("2022-01-01"), ...
datetime("2022-02-28"), ...
stationid = "GHCND:USW00003856", ...
limit = 1000, ...
units = "metric")

From here I looped through all of the datatypes. In each case there is only one data point, 02/28/2022, for each. I'm not sure why this is but I think it's because the API doesn't have any data for the other dates. I'm not sure why this is different from bulk data sets that you download.

for i = 1:length(d.datatype)
d.datatype{i}
getdata(n,"GHCND", ...
datetime(strcat("2022","-01-01")), ...
datetime(strcat("2022","-02-28")), ...
stationid = "GHCND:USW00003856", ...
datatypeid = "TMAX" , ...
limit = 1000, ...
units = "metric")
end

I don't have clarification from NOAA on this.

Chris

[steveforczyk]

Chris: You have captured part of the problem. When I look at the datasheets online there is data for every day and Jan and Feb for that site but the getdata command is only returning data from Feb 28 through the end of the year. I will try some other airport sites to see if I hit a ringer. Note that this was tru over the 2012 through 2019 period. We are ultimately try to get a large collection of data to look at warming trends so I sure hate to miss 2 months of data per year. Let me know if you find a work around. FYI I also tried a Python solution to the download-same damn problem so I don't really think it is a problem with the Matlab implementation. I am extensively using data from the GOES platform without any serious problems but I want some ground truth and also want data from regions outside the GOES coverage area. From: "Chris Garvin" ***@***.***> To: "mathworks/noaa" ***@***.***> Cc: "Stephen M Forczyk" ***@***.***>, "Author" ***@***.***> Sent: Wednesday, November 23, 2022 8:31:59 PM Subject: Re: [mathworks/noaa] MATLAB Interface for NOAA Data-major issue (Issue #1) Hi Stephen, I verified that I can get January data for other datasets and datatype to make sure the interface wasn't having an issue with those date inputs. But I'm guessing that data won't be of much value to you. I tried the following code to see what datatypes are available for that station and date range: d = getdata(n,"GHCND", ... datetime("2022-01-01"), ... datetime("2022-02-28"), ... stationid = "GHCND:USW00003856", ... limit = 1000, ... units = "metric") From here I looped through all of the datatypes. In each case there is only one data point, 02/28/2022, for each. I'm not sure why this is but I think it's because the API doesn't have any data for the other dates. I'm not sure why this is different from bulk data sets that you download. for i = 1:length(d.datatype) d.datatype{i} getdata(n,"GHCND", ... datetime(strcat("2022","-01-01")), ... datetime(strcat("2022","-02-28")), ... stationid = "GHCND:USW00003856", ... datatypeid = "TMAX" , ... limit = 1000, ... units = "metric") end I don't have clarification from NOAA on this. Chris — Reply to this email directly, [ #1 (comment) | view it on GitHub ] , or [ https://github.com/notifications/unsubscribe-auth/AIANZILWH6G5WLRKGEURUOTWJ3HR7ANCNFSM6AAAAAASH54ADQ | unsubscribe ] . You are receiving this because you authored the thread. Message ID: ***@***.***>

[cfletchg]

I will let you know if I learn of a workaround or get any information from NOAA

[steveforczyk]

Chris: I am going to try to download data from a few more airport sites. If I get the result at multiple sites that would support the notion that something is wrong in the calling sequence From: "Chris Garvin" ***@***.***> To: "mathworks/noaa" ***@***.***> Cc: "Stephen M Forczyk" ***@***.***>, "Author" ***@***.***> Sent: Monday, November 28, 2022 8:36:06 AM Subject: Re: [mathworks/noaa] MATLAB Interface for NOAA Data-major issue (Issue #1) I will let you know if I learn of a workaround or get any information from NOAA — Reply to this email directly, [ #1 (comment) | view it on GitHub ] , or [ https://github.com/notifications/unsubscribe-auth/AIANZINHB2Z4BCINDPPKS43WKS7NNANCNFSM6AAAAAASH54ADQ | unsubscribe ] . You are receiving this because you authored the thread. Message ID: ***@***.***>

[cfletchg]

I received confirmation from NOAA today that they are aware of the missing data issue and working to address it.

[steveforczyk]

Chris: Thanks for the update. It felt like I was beating my head against the wall. We are using the data for global climate studies (it is just one source) so it is annoying to get only 10 months out of 12 ! From: "Chris Garvin" ***@***.***> To: "mathworks/noaa" ***@***.***> Cc: "Stephen M Forczyk" ***@***.***>, "Author" ***@***.***> Sent: Wednesday, November 30, 2022 1:54:57 PM Subject: Re: [mathworks/noaa] MATLAB Interface for NOAA Data-major issue (Issue #1) I received confirmation from NOAA today that they are aware of the missing data issue and working to address it. — Reply to this email directly, [ #1 (comment) | view it on GitHub ] , or [ https://github.com/notifications/unsubscribe-auth/AIANZIPLJNUJA64YMZ7DWT3WK6WJDANCNFSM6AAAAAASH54ADQ | unsubscribe ] . You are receiving this because you authored the thread. Message ID: ***@***.***>

[behroozdaneshian]

Hello all,
I still have a problem with extracting daily temperature data for a desired year range(e.g., 2006 to 2012) for a given station ("GHCND:SPE00119783"). The problem is that the created table(annualTemperatureData) does not include data from January 1 to February 28. I have checked the CSV file for this station and data exist for the mentioned time interval. Does anyone know how we can deal with that? I used the following code for that.
annualTemperatureData = [];
for y = 2006:2012
annualTemperatureData = [annualTemperatureData;getdata(n,"GHCND", ...
datetime(strcat(num2str(y),"-01-01")), ...
datetime(strcat(num2str(y),"-12-31")), ...
stationid = "GHCND:SPE00119783", ...
datatypeid = "TAVG", ...
locationid = "FIPS:SP", ...
limit = 1000, ...
units = "standard")];
end

[cfletchg]

Hello Behrooz, NOAA is aware of the issue where the data for January and February is missing and they are working on it. No updates as of yet. Chris From: behroozdaneshian ***@***.***> Sent: Thursday, January 5, 2023 10:43 PM To: mathworks/noaa ***@***.***> Cc: Chris Garvin ***@***.***>; State change ***@***.***> Subject: Re: [mathworks/noaa] MATLAB Interface for NOAA Data-major issue (Issue #1) Hello all, I still have a problem with extracting daily temperature data for a desired year range(e.g., 2006 to 2012) for a given station ("GHCND:SPE00119783"). The problem is that the created table(annualTemperatureData) does not include data from January 1 to February 28. I have checked the CSV file for this station and data exist for the mentioned time interval. Does anyone know how we can deal with that? I used the following code for that. annualTemperatureData = []; for y = 2006:2012 annualTemperatureData = [annualTemperatureData;getdata(n,"GHCND", ... datetime(strcat(num2str(y),"-01-01")), ... datetime(strcat(num2str(y),"-12-31")), ... stationid = "GHCND:SPE00119783", ... datatypeid = "TAVG", ... locationid = "FIPS:SP", ... limit = 1000, ... units = "standard")]; end — Reply to this email directly, view it on GitHub<#1 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AABZPAU3LSBU2ZT2DR6IDSTWQ6ID3ANCNFSM6AAAAAASH54ADQ>. You are receiving this because you modified the open/close state.Message ID: ***@***.******@***.***>>

[steveforczyk]

Thanks for keeping me in the loop From: "Chris Garvin" ***@***.***> To: "mathworks/noaa" ***@***.***> Cc: "Stephen M Forczyk" ***@***.***>, "Author" ***@***.***> Sent: Friday, January 6, 2023 8:06:14 AM Subject: Re: [mathworks/noaa] MATLAB Interface for NOAA Data-major issue (Issue #1) Hello Behrooz, NOAA is aware of the issue where the data for January and February is missing and they are working on it. No updates as of yet. Chris From: behroozdaneshian ***@***.***> Sent: Thursday, January 5 , 2023 10:43 PM To: mathworks/noaa ***@***.***> Cc: Chris Garvin ***@***.***>; State change ***@***.***> Subject: Re: [mathworks/noaa] MATLAB Interface for NOAA Data-major issue (Issue #1) Hello all, I still have a problem with extracting daily temperature data for a desired year range(e.g., 2006 to 2012) for a given station ("GHCND:SPE00119783"). The problem is that the created table(annualTemperatureData) does not include data from January 1 to February 28 . I have checked the CSV file for this station and data exist for the mentioned time interval. Does anyone know how we can deal with that? I used the following code for that. annualTemperatureData = []; for y = 2006:2012 annualTemperatureData = [annualTemperatureData;getdata(n,"GHCND", ... datetime(strcat(num2str(y),"-01-01")), ... datetime(strcat(num2str(y),"-12-31")), ... stationid = "GHCND:SPE00119783", ... datatypeid = "TAVG", ... locationid = "FIPS:SP", ... limit = 1000, ... units = "standard")]; end — Reply to this email directly, view it on GitHub< [ #1 (comment) | #1 (comment) ] >, or unsubscribe< [ https://github.com/notifications/unsubscribe-auth/AABZPAU3LSBU2ZT2DR6IDSTWQ6ID3ANCNFSM6AAAAAASH54ADQ | https://github.com/notifications/unsubscribe-auth/AABZPAU3LSBU2ZT2DR6IDSTWQ6ID3ANCNFSM6AAAAAASH54ADQ ] >. You are receiving this because you modified the open/close state.Message ID: ***@***.******@***.***>> — Reply to this email directly, [ #1 (comment) | view it on GitHub ] , or [ https://github.com/notifications/unsubscribe-auth/AIANZIK4LM2WSF7O7TFC3TDWRARFNANCNFSM6AAAAAASH54ADQ | unsubscribe ] . You are receiving this because you authored the thread. Message ID: < ***@***.*** >

[behroozdaneshian]

Oh my god, so the problem is at the source. Can anyone suggest another way to extract January and February data?

[steveforczyk]

Yes I believe it is in the interface. FYI I also used a Python script to extract the data and it worked but you guessed it no Jan or Feb data until Feb 28. Since the problem occured in using Python or Matlab it seems to be at the next higher level From: "behroozdaneshian" ***@***.***> To: "mathworks/noaa" ***@***.***> Cc: "Stephen M Forczyk" ***@***.***>, "Author" ***@***.***> Sent: Saturday, January 7, 2023 5:22:02 PM Subject: Re: [mathworks/noaa] MATLAB Interface for NOAA Data-major issue (Issue #1) Oh my god, so the problem is at the source. Can anyone suggest another way to extract January and February data? — Reply to this email directly, [ #1 (comment) | view it on GitHub ] , or [ https://github.com/notifications/unsubscribe-auth/AIANZIPH246Z22VYRTCJBETWRH3BVANCNFSM6AAAAAASH54ADQ | unsubscribe ] . You are receiving this because you authored the thread. Message ID: ***@***.***>

[cfletchg]

Apparently, NOAA has a new API available with different endpoints that will address this data issue. I’m investigating but it looks different enough that it cannot just be dropped into the existing interface. From: Stephen M Forczyk ***@***.***> Sent: Sunday, January 8, 2023 9:49 AM To: mathworks/noaa ***@***.***> Cc: Chris Garvin ***@***.***>; State change ***@***.***> Subject: Re: [mathworks/noaa] MATLAB Interface for NOAA Data-major issue (Issue #1) Yes I believe it is in the interface. FYI I also used a Python script to extract the data and it worked but you guessed it no Jan or Feb data until Feb 28. Since the problem occured in using Python or Matlab it seems to be at the next higher level From: "behroozdaneshian" ***@***.***<mailto:***@***.***>> To: "mathworks/noaa" ***@***.***<mailto:***@***.***>> Cc: "Stephen M Forczyk" ***@***.***<mailto:***@***.***>>, "Author" ***@***.***<mailto:***@***.***>> Sent: Saturday, January 7, 2023 5:22:02 PM Subject: Re: [mathworks/noaa] MATLAB Interface for NOAA Data-major issue (Issue #1) Oh my god, so the problem is at the source. Can anyone suggest another way to extract January and February data? — Reply to this email directly, [ #1 (comment)<https://github.com/mathworks/noaa/issues/1#issuecomment-1374650357> | view it on GitHub ] , or [ https://github.com/notifications/unsubscribe-auth/AIANZIPH246Z22VYRTCJBETWRH3BVANCNFSM6AAAAAASH54ADQ<https://github.com/notifications/unsubscribe-auth/AIANZIPH246Z22VYRTCJBETWRH3BVANCNFSM6AAAAAASH54ADQ> | unsubscribe ] . You are receiving this because you authored the thread. Message ID: ***@***.***<mailto:***@***.***>> — Reply to this email directly, view it on GitHub<#1 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AABZPASFJ7V7KPTQVZICLSLWRLHXBANCNFSM6AAAAAASH54ADQ>. You are receiving this because you modified the open/close state.Message ID: ***@***.******@***.***>>

[behroozdaneshian]

I sent them an email regarding this issue, and they told me we are working on API to tackle it. They also suggest v1 API using this URL: https://www.ncei.noaa.gov/access-search#access-apis
However, I could not figure it out.

[cfletchg]

Hopefully, they will address it in the existing API but I’m not sure they are going to. If you know the V1 URL you need, you can access it with webread. But I’m investigating the V1 API as I said. From: behroozdaneshian ***@***.***> Sent: Monday, January 9, 2023 9:57 AM To: mathworks/noaa ***@***.***> Cc: Chris Garvin ***@***.***>; State change ***@***.***> Subject: Re: [mathworks/noaa] MATLAB Interface for NOAA Data-major issue (Issue #1) I sent them an email regarding this issue, and they told me we are working on API to tackle it. They also suggest v1 API using this URL: https://www.ncei.noaa.gov/access-search#access-apis<https://www.ncei.noaa.gov/access-search#access-apis> However, I could not figure it out. — Reply to this email directly, view it on GitHub<#1 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AABZPAVEB4KNYKKDIZIQG6LWRQRNTANCNFSM6AAAAAASH54ADQ>. You are receiving this because you modified the open/close state.Message ID: ***@***.******@***.***>>

[steveforczyk]

Chris: I appreciate any help I can get. I have many other irons in the fire at the moment so I can put this one aside for the moment. I wanted the ground station data as a truth check for some of the satellite data I work with all the time. I am a trust but verify kind of guy! From: "Chris Garvin" ***@***.***> To: "mathworks/noaa" ***@***.***> Cc: "Stephen M Forczyk" ***@***.***>, "Author" ***@***.***> Sent: Monday, January 9, 2023 9:01:08 AM Subject: Re: [mathworks/noaa] MATLAB Interface for NOAA Data-major issue (Issue #1) Hopefully, they will address it in the existing API but I’m not sure they are going to. If you know the V1 URL you need, you can access it with webread. But I’m investigating the V1 API as I said. From: behroozdaneshian ***@***.***> Sent: Monday, January 9, 2023 9:57 AM To: mathworks/noaa ***@***.***> Cc: Chris Garvin ***@***.***>; State change ***@***.***> Subject: Re: [mathworks/noaa] MATLAB Interface for NOAA Data-major issue (Issue #1) I sent them an email regarding this issue, and they told me we are working on API to tackle it. They also suggest v1 API using this URL: https://www.ncei.noaa.gov/access-search#access-apis<https://www.ncei.noaa.gov/access-search#access-apis> However, I could not figure it out. — Reply to this email directly, view it on GitHub<#1 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AABZPAVEB4KNYKKDIZIQG6LWRQRNTANCNFSM6AAAAAASH54ADQ>. You are receiving this because you modified the open/close state.Message ID: ***@***.******@***.***>> — Reply to this email directly, [ #1 (comment) | view it on GitHub ] , or [ https://github.com/notifications/unsubscribe-auth/AIANZIKDFTTAR4K3KIW5YJDWRQR3JANCNFSM6AAAAAASH54ADQ | unsubscribe ] . You are receiving this because you authored the thread. Message ID: ***@***.***>

[cfletchg]

I’ve been looking at what I’ll call the NCEI V1 API. I’ll refer to the API that the MATLAB interface is based on as NOAA V2 API. The NCEI V1 API has a different approach. To request data, you need to know things like the station and boundingBox (location) inputs for your request. The NOAA V2 API allowed one to search for all available stations and locations id’s whereas the NCEI V1 API does not. One can extract things like stations if they know which dataset and optionally which datatype they want. And this type of request takes some post processing of the data which could potentially be “buggy” if some type of unexpected data structure is returned. For the NCEI V1 API, the current easiest way to request data looks something like this options = weboptions("Timeout",200) webread(https://www.ncei.noaa.gov/access/services/data/v1?dataset=daily-summaries&dataTypes=TAVG&stations=USW00014607&startDate=2022-01-01&endDate=2023-01-01&boundingBox=90,-180,-90,180,options) You can see that the dataset, dataTypes, stations, date range and boundingBox are specified. The NOAA V2 API takes the same types of inputs but has a more generalized search mechanism to find these. Granted, one still has to know what the stations, for example, refer to. It’s similar to the noaa getdata call in MATLAB: getdata(n,"GHCND", datetime("2022-01-01")), datetime("2023-01-01")), ... stationid = "GHCND:SPE00119783", ... datatypeid = "TAVG", ... locationid = "FIPS:SP", ... limit = 1000, ... units = "metric")]; except that the station id is a different format and the locationid is not used. And I’m unclear so far on how boundingBox is mapped to the V2 location id’s. NCEI V1 API doesn’t appear to have any holes in the data though. So not much of an answer so far but this is where I am and not sure where it will lead. From: Stephen M Forczyk ***@***.***> Sent: Monday, January 9, 2023 6:13 PM To: mathworks/noaa ***@***.***> Cc: Chris Garvin ***@***.***>; State change ***@***.***> Subject: Re: [mathworks/noaa] MATLAB Interface for NOAA Data-major issue (Issue #1) Chris: I appreciate any help I can get. I have many other irons in the fire at the moment so I can put this one aside for the moment. I wanted the ground station data as a truth check for some of the satellite data I work with all the time. I am a trust but verify kind of guy! From: "Chris Garvin" ***@***.***<mailto:***@***.***>> To: "mathworks/noaa" ***@***.***<mailto:***@***.***>> Cc: "Stephen M Forczyk" ***@***.***<mailto:***@***.***>>, "Author" ***@***.***<mailto:***@***.***>> Sent: Monday, January 9, 2023 9:01:08 AM Subject: Re: [mathworks/noaa] MATLAB Interface for NOAA Data-major issue (Issue #1) Hopefully, they will address it in the existing API but I’m not sure they are going to. If you know the V1 URL you need, you can access it with webread. But I’m investigating the V1 API as I said. From: behroozdaneshian ***@***.***<mailto:***@***.***>> Sent: Monday, January 9, 2023 9:57 AM To: mathworks/noaa ***@***.***<mailto:***@***.***>> Cc: Chris Garvin ***@***.***<mailto:***@***.***>>; State change ***@***.***<mailto:***@***.***>> Subject: Re: [mathworks/noaa] MATLAB Interface for NOAA Data-major issue (Issue #1) I sent them an email regarding this issue, and they told me we are working on API to tackle it. They also suggest v1 API using this URL: https://www.ncei.noaa.gov/access-search#access-apis<https://www.ncei.noaa.gov/access-search#access-apis><https://www.ncei.noaa.gov/access-search#access-apis<https://www.ncei.noaa.gov/access-search#access-apis>> However, I could not figure it out. — Reply to this email directly, view it on GitHub<#1 (comment)<https://github.com/mathworks/noaa/issues/1#issuecomment-1375746550>>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AABZPAVEB4KNYKKDIZIQG6LWRQRNTANCNFSM6AAAAAASH54ADQ<https://github.com/notifications/unsubscribe-auth/AABZPAVEB4KNYKKDIZIQG6LWRQRNTANCNFSM6AAAAAASH54ADQ>>. You are receiving this because you modified the open/close state.Message ID: ***@***.******@***.***<mailto:***@***.******@***.***>>> — Reply to this email directly, [ #1 (comment)<https://github.com/mathworks/noaa/issues/1#issuecomment-1375751691> | view it on GitHub ] , or [ https://github.com/notifications/unsubscribe-auth/AIANZIKDFTTAR4K3KIW5YJDWRQR3JANCNFSM6AAAAAASH54ADQ<https://github.com/notifications/unsubscribe-auth/AIANZIKDFTTAR4K3KIW5YJDWRQR3JANCNFSM6AAAAAASH54ADQ> | unsubscribe ] . You are receiving this because you authored the thread. Message ID: ***@***.***<mailto:***@***.***>> — Reply to this email directly, view it on GitHub<#1 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AABZPAX3NOGRERKQDEDWZCLWRSLPBANCNFSM6AAAAAASH54ADQ>. You are receiving this because you modified the open/close state.Message ID: ***@***.******@***.***>>

[steveforczyk]

Chris: Thanks for the update. Is there an available Matlab interface based on the V1 API? From: "Chris Garvin" ***@***.***> To: "mathworks/noaa" ***@***.***> Cc: "Stephen M Forczyk" ***@***.***>, "Author" ***@***.***> Sent: Thursday, January 12, 2023 3:02:32 PM Subject: Re: [mathworks/noaa] MATLAB Interface for NOAA Data-major issue (Issue #1) I’ve been looking at what I’ll call the NCEI V1 API. I’ll refer to the API that the MATLAB interface is based on as NOAA V2 API. The NCEI V1 API has a different approach. To request data, you need to know things like the station and boundingBox (location) inputs for your request. The NOAA V2 API allowed one to search for all available stations and locations id’s whereas the NCEI V1 API does not. One can extract things like stations if they know which dataset and optionally which datatype they want. And this type of request takes some post processing of the data which could potentially be “buggy” if some type of unexpected data structure is returned. For the NCEI V1 API, the current easiest way to request data looks something like this options = weboptions("Timeout",200) webread(https://www.ncei.noaa.gov/access/services/data/v1?dataset=daily-summaries&dataTypes=TAVG&stations=USW00014607&startDate=2022-01-01&endDate=2023-01-01&boundingBox=90,-180,-90,180,options) You can see that the dataset, dataTypes, stations, date range and boundingBox are specified. The NOAA V2 API takes the same types of inputs but has a more generalized search mechanism to find these. Granted, one still has to know what the stations, for example, refer to. It’s similar to the noaa getdata call in MATLAB: getdata(n,"GHCND", datetime("2022-01-01")), datetime("2023-01-01")), ... stationid = "GHCND:SPE00119783", ... datatypeid = "TAVG", ... locationid = "FIPS:SP", ... limit = 1000, ... units = "metric")]; except that the station id is a different format and the locationid is not used. And I’m unclear so far on how boundingBox is mapped to the V2 location id’s. NCEI V1 API doesn’t appear to have any holes in the data though. So not much of an answer so far but this is where I am and not sure where it will lead. From: Stephen M Forczyk ***@***.***> Sent: Monday, January 9, 2023 6:13 PM To: mathworks/noaa ***@***.***> Cc: Chris Garvin ***@***.***>; State change ***@***.***> Subject: Re: [mathworks/noaa] MATLAB Interface for NOAA Data-major issue (Issue #1) Chris: I appreciate any help I can get. I have many other irons in the fire at the moment so I can put this one aside for the moment. I wanted the ground station data as a truth check for some of the satellite data I work with all the time. I am a trust but verify kind of guy! From: "Chris Garvin" ***@***.***<mailto:***@***.***>> To: "mathworks/noaa" ***@***.***<mailto:***@***.***>> Cc: "Stephen M Forczyk" ***@***.***<mailto:***@***.***>>, "Author" ***@***.***<mailto:***@***.***>> Sent: Monday, January 9, 2023 9:01:08 AM Subject: Re: [mathworks/noaa] MATLAB Interface for NOAA Data-major issue (Issue #1) Hopefully, they will address it in the existing API but I’m not sure they are going to. If you know the V1 URL you need, you can access it with webread. But I’m investigating the V1 API as I said. From: behroozdaneshian ***@***.***<mailto:***@***.***>> Sent: Monday, January 9, 2023 9:57 AM To: mathworks/noaa ***@***.***<mailto:***@***.***>> Cc: Chris Garvin ***@***.***<mailto:***@***.***>>; State change ***@***.***<mailto:***@***.***>> Subject: Re: [mathworks/noaa] MATLAB Interface for NOAA Data-major issue (Issue #1) I sent them an email regarding this issue, and they told me we are working on API to tackle it. They also suggest v1 API using this URL: https://www.ncei.noaa.gov/access-search#access-apis<https://www.ncei.noaa.gov/access-search#access-apis><https://www.ncei.noaa.gov/access-search#access-apis<https://www.ncei.noaa.gov/access-search#access-apis>> However, I could not figure it out. — Reply to this email directly, view it on GitHub<#1 (comment)<https://github.com/mathworks/noaa/issues/1#issuecomment-1375746550>>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AABZPAVEB4KNYKKDIZIQG6LWRQRNTANCNFSM6AAAAAASH54ADQ<https://github.com/notifications/unsubscribe-auth/AABZPAVEB4KNYKKDIZIQG6LWRQRNTANCNFSM6AAAAAASH54ADQ>>. You are receiving this because you modified the open/close state.Message ID: ***@***.******@***.***<mailto:***@***.******@***.***>>> — Reply to this email directly, [ #1 (comment)<https://github.com/mathworks/noaa/issues/1#issuecomment-1375751691> | view it on GitHub ] , or [ https://github.com/notifications/unsubscribe-auth/AIANZIKDFTTAR4K3KIW5YJDWRQR3JANCNFSM6AAAAAASH54ADQ<https://github.com/notifications/unsubscribe-auth/AIANZIKDFTTAR4K3KIW5YJDWRQR3JANCNFSM6AAAAAASH54ADQ> | unsubscribe ] . You are receiving this because you authored the thread. Message ID: ***@***.***<mailto:***@***.***>> — Reply to this email directly, view it on GitHub<#1 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AABZPAX3NOGRERKQDEDWZCLWRSLPBANCNFSM6AAAAAASH54ADQ>. You are receiving this because you modified the open/close state.Message ID: ***@***.******@***.***>> — Reply to this email directly, [ #1 (comment) | view it on GitHub ] , or [ https://github.com/notifications/unsubscribe-auth/AIANZIOGPFCXAWAQALRT6Z3WSBWORANCNFSM6AAAAAASH54ADQ | unsubscribe ] . You are receiving this because you authored the thread. Message ID: ***@***.***>

[cfletchg]

Not currently. I’m looking at that now to determine the feasibility. Using the webread call from MATLAB is the best option right now. You can use the example URL from the previous email and modify it based on the data you need. From: Stephen M Forczyk ***@***.***> Sent: Friday, January 13, 2023 9:14 AM To: mathworks/noaa ***@***.***> Cc: Chris Garvin ***@***.***>; State change ***@***.***> Subject: Re: [mathworks/noaa] MATLAB Interface for NOAA Data-major issue (Issue #1) Chris: Thanks for the update. Is there an available Matlab interface based on the V1 API? From: "Chris Garvin" ***@***.***<mailto:***@***.***>> To: "mathworks/noaa" ***@***.***<mailto:***@***.***>> Cc: "Stephen M Forczyk" ***@***.***<mailto:***@***.***>>, "Author" ***@***.***<mailto:***@***.***>> Sent: Thursday, January 12, 2023 3:02:32 PM Subject: Re: [mathworks/noaa] MATLAB Interface for NOAA Data-major issue (Issue #1) I’ve been looking at what I’ll call the NCEI V1 API. I’ll refer to the API that the MATLAB interface is based on as NOAA V2 API. The NCEI V1 API has a different approach. To request data, you need to know things like the station and boundingBox (location) inputs for your request. The NOAA V2 API allowed one to search for all available stations and locations id’s whereas the NCEI V1 API does not. One can extract things like stations if they know which dataset and optionally which datatype they want. And this type of request takes some post processing of the data which could potentially be “buggy” if some type of unexpected data structure is returned. For the NCEI V1 API, the current easiest way to request data looks something like this options = weboptions("Timeout",200) webread(https://www.ncei.noaa.gov/access/services/data/v1?dataset=daily-summaries&dataTypes=TAVG&stations=USW00014607&startDate=2022-01-01&endDate=2023-01-01&boundingBox=90,-180,-90,180,options<https://www.ncei.noaa.gov/access/services/data/v1?dataset=daily-summaries&dataTypes=TAVG&stations=USW00014607&startDate=2022-01-01&endDate=2023-01-01&boundingBox=90,-180,-90,180,options>) You can see that the dataset, dataTypes, stations, date range and boundingBox are specified. The NOAA V2 API takes the same types of inputs but has a more generalized search mechanism to find these. Granted, one still has to know what the stations, for example, refer to. It’s similar to the noaa getdata call in MATLAB: getdata(n,"GHCND", datetime("2022-01-01")), datetime("2023-01-01")), ... stationid = "GHCND:SPE00119783", ... datatypeid = "TAVG", ... locationid = "FIPS:SP", ... limit = 1000, ... units = "metric")]; except that the station id is a different format and the locationid is not used. And I’m unclear so far on how boundingBox is mapped to the V2 location id’s. NCEI V1 API doesn’t appear to have any holes in the data though. So not much of an answer so far but this is where I am and not sure where it will lead. From: Stephen M Forczyk ***@***.***<mailto:***@***.***>> Sent: Monday, January 9, 2023 6:13 PM To: mathworks/noaa ***@***.***<mailto:***@***.***>> Cc: Chris Garvin ***@***.***<mailto:***@***.***>>; State change ***@***.***<mailto:***@***.***>> Subject: Re: [mathworks/noaa] MATLAB Interface for NOAA Data-major issue (Issue #1) Chris: I appreciate any help I can get. I have many other irons in the fire at the moment so I can put this one aside for the moment. I wanted the ground station data as a truth check for some of the satellite data I work with all the time. I am a trust but verify kind of guy! From: "Chris Garvin" ***@***.***<mailto:***@***.***<mailto:***@***.***%3cmailto:***@***.***>>> To: "mathworks/noaa" ***@***.***<mailto:***@***.***<mailto:***@***.***%3cmailto:***@***.***>>> Cc: "Stephen M Forczyk" ***@***.***<mailto:***@***.***<mailto:***@***.***%3cmailto:***@***.***>>>, "Author" ***@***.***<mailto:***@***.***<mailto:***@***.***%3cmailto:***@***.***>>> Sent: Monday, January 9, 2023 9:01:08 AM Subject: Re: [mathworks/noaa] MATLAB Interface for NOAA Data-major issue (Issue #1) Hopefully, they will address it in the existing API but I’m not sure they are going to. If you know the V1 URL you need, you can access it with webread. But I’m investigating the V1 API as I said. From: behroozdaneshian ***@***.***<mailto:***@***.***<mailto:***@***.***%3cmailto:***@***.***>>> Sent: Monday, January 9, 2023 9:57 AM To: mathworks/noaa ***@***.***<mailto:***@***.***<mailto:***@***.***%3cmailto:***@***.***>>> Cc: Chris Garvin ***@***.***<mailto:***@***.***<mailto:***@***.***%3cmailto:***@***.***>>>; State change ***@***.***<mailto:***@***.***<mailto:***@***.***%3cmailto:***@***.***>>> Subject: Re: [mathworks/noaa] MATLAB Interface for NOAA Data-major issue (Issue #1) I sent them an email regarding this issue, and they told me we are working on API to tackle it. They also suggest v1 API using this URL: https://www.ncei.noaa.gov/access-search#access-apis<https://www.ncei.noaa.gov/access-search#access-apis><https://www.ncei.noaa.gov/access-search#access-apis<https://www.ncei.noaa.gov/access-search#access-apis>><https://www.ncei.noaa.gov/access-search#access-apis<https://www.ncei.noaa.gov/access-search#access-apis><https://www.ncei.noaa.gov/access-search#access-apis<https://www.ncei.noaa.gov/access-search#access-apis>>> However, I could not figure it out. — Reply to this email directly, view it on GitHub<#1 (comment)<https://github.com/mathworks/noaa/issues/1#issuecomment-1375746550><https://github.com/mathworks/noaa/issues/1#issuecomment-1375746550<https://github.com/mathworks/noaa/issues/1#issuecomment-1375746550>>>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AABZPAVEB4KNYKKDIZIQG6LWRQRNTANCNFSM6AAAAAASH54ADQ<https://github.com/notifications/unsubscribe-auth/AABZPAVEB4KNYKKDIZIQG6LWRQRNTANCNFSM6AAAAAASH54ADQ><https://github.com/notifications/unsubscribe-auth/AABZPAVEB4KNYKKDIZIQG6LWRQRNTANCNFSM6AAAAAASH54ADQ<https://github.com/notifications/unsubscribe-auth/AABZPAVEB4KNYKKDIZIQG6LWRQRNTANCNFSM6AAAAAASH54ADQ>>>. You are receiving this because you modified the open/close state.Message ID: ***@***.******@***.***<mailto:***@***.******@***.***<mailto:***@***.******@***.***%3cmailto:***@***.******@***.***>>>> — Reply to this email directly, [ #1 (comment)<https://github.com/mathworks/noaa/issues/1#issuecomment-1375751691><https://github.com/mathworks/noaa/issues/1#issuecomment-1375751691<https://github.com/mathworks/noaa/issues/1#issuecomment-1375751691>> | view it on GitHub ] , or [ https://github.com/notifications/unsubscribe-auth/AIANZIKDFTTAR4K3KIW5YJDWRQR3JANCNFSM6AAAAAASH54ADQ<https://github.com/notifications/unsubscribe-auth/AIANZIKDFTTAR4K3KIW5YJDWRQR3JANCNFSM6AAAAAASH54ADQ><https://github.com/notifications/unsubscribe-auth/AIANZIKDFTTAR4K3KIW5YJDWRQR3JANCNFSM6AAAAAASH54ADQ<https://github.com/notifications/unsubscribe-auth/AIANZIKDFTTAR4K3KIW5YJDWRQR3JANCNFSM6AAAAAASH54ADQ>> | unsubscribe ] . You are receiving this because you authored the thread. Message ID: ***@***.***<mailto:***@***.***<mailto:***@***.***%3cmailto:***@***.***>>> — Reply to this email directly, view it on GitHub<#1 (comment)<https://github.com/mathworks/noaa/issues/1#issuecomment-1376469151>>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AABZPAX3NOGRERKQDEDWZCLWRSLPBANCNFSM6AAAAAASH54ADQ<https://github.com/notifications/unsubscribe-auth/AABZPAX3NOGRERKQDEDWZCLWRSLPBANCNFSM6AAAAAASH54ADQ>>. You are receiving this because you modified the open/close state.Message ID: ***@***.******@***.***<mailto:***@***.******@***.***>>> — Reply to this email directly, [ #1 (comment)<https://github.com/mathworks/noaa/issues/1#issuecomment-1380990054> | view it on GitHub ] , or [ https://github.com/notifications/unsubscribe-auth/AIANZIOGPFCXAWAQALRT6Z3WSBWORANCNFSM6AAAAAASH54ADQ<https://github.com/notifications/unsubscribe-auth/AIANZIOGPFCXAWAQALRT6Z3WSBWORANCNFSM6AAAAAASH54ADQ> | unsubscribe ] . You are receiving this because you authored the thread. Message ID: ***@***.***<mailto:***@***.***>> — Reply to this email directly, view it on GitHub<#1 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AABZPAUOSOJZEC3L47YTB7LWSFPLVANCNFSM6AAAAAASH54ADQ>. You are receiving this because you modified the open/close state.Message ID: ***@***.******@***.***>>