mps9506 / echor Goto Github PK

View Code? Open in Web Editor NEW

7.0 2.0 1.0 42.74 MB

Download EPA ECHO data in R

Home Page: https://mps9506.github.io/echor/

License: Other

R 100.00%

r pollution epa rstats r-package water-quality

echor's People

Contributors

Stargazers

Watchers

Forkers

romainfrancois

echor's Issues

tidyr v1.0.0 breaks vignette

Per emails from Hadley and CRAN, the new version of tidyr (v1.0.0) breaks the package.
https://tidyr.tidyverse.org/dev/news/index.html#breaking-changes

Looks like the vignette is the only thing that needs changing.

urlEncode breaks multiple arguments

using urlEncode in queryList to build api argument calls breaks the way the list is built. URLencode needs to be applied only to the list values so the function can properly paste each value into the string.

downloadDMRs creates a tibble that doesn't unnest easily

The nested tibble from downloadDMRs() relies on the user mapping column types to unnest the dmr column:

library(dplyr)
library(echor)

df <- tibble::tibble(permit = c("TX0021474", "TX0023051", "TXG130053"))
df <- downloadDMRs(df, idColumn = permit, start_date = "01/01/2010",
                   end_date = "12/30/2017", parameter_code = "50050")

df %>% tidyr::unnest(dmr)

Results in the following message:

Error: Column `monitoring_location_code` can't be converted from character to numeric

I have to follow up with the following to unnest the data:

df %>%
  mutate(n = purrr::map(dmr, ~nrow(.))) %>%
  mutate(dmr = purrr::map(dmr, ~mutate(.x,
                                version_nmbr = as.character(version_nmbr),
                                monitoring_location_code = as.character(monitoring_location_code),
                                exceedence_pct = as.character(exceedence_pct),
                                rnc_detection_date = as.character(rnc_detection_date),
                                rnc_resolution_code = as.character(rnc_resolution_code),
                                rnc_resolution_date = as.character(rnc_resolution_date),
                                days_late = as.character(days_late),
                                nodi_code = as.character(nodi_code),
                                dmr_value_nmbr = as.numeric(dmr_value_nmbr),
                                dmr_value_standard_units = as.numeric(dmr_value_standard_units),
                                limit_value_nmbr = as.numeric(limit_value_nmbr),
                                limit_value_standard_units = as.numeric(limit_value_standard_units)))) %>%
  tidyr::unnest(dmr)

The user should not have to figure this out, but if the above code is hardcoded, it will be prone to breaking if the columns returned by the API ever change.

api address changing

ECHO's data service URLs are changing. Please update your code and bookmarks as soon as possible to maintain access to ECHO data. Users will need to adjust their URLs to start with https://echodata.epa.gov instead of https://ofmpub.epa.gov. The service documentation listed below has been changed to use the new URL.

Complete documentation

Documentation and description need to be cleaned up and completed.

ggmap orphaned

ggmap is included in suggests since it is used in the vignette. However, it is listed as an orphaned package and raises a note when checking as cran.

Release echor 0.1.9

Prepare for release:

Submit to CRAN:

usethis::use_version('patch')
devtools::submit_cran()
Approve email

Wait for CRAN...

Accepted 🎉
usethis::use_github_release()

need a getQID function

This is to sort out functional issues required for fixing #79 .

I didn't realize getDownload endpoint restricts downloads to 100,000 rows. I used this endpoint because it returns a really convenient CSV file. The getQID endpoint returns all rows, but are paginated.

The getQID endpoint doesn't tell us how many pages of results there are. It does return the total responses and looking at some test requests, it returns 5,000 rows per page.
The getGeoJson endpoint probably should check if there are more than 100,000 responses before making the request.

Possible approaches:

check number of facilities returned in getFacilities, if > 100,000 make a request through getQID endpoint. if < 100,000 make a request through getDownload. The advantage of this is that we potentially make fewer server requests. There is a minimum of 20 requests at 100,000 records.
make all requests through getQID. The advantage here is if there are any discrepancies between column names returned by the getDownload endpoint and getQID endpoint, users won't see it.

I'm planning on adopting the first approach.

checklist

Add getQID() endpoint with url endpoint for each of the services.
Add getQID() to echoWaterGetFacilityInfo(), echoAirGetFacilityInfo(), and echoSDWGetSystems()
Check number of responses before making the getGeoJSON() call.

sf dependency

Explore using https://github.com/SymbolixAU/geojsonsf to convert geojson to sf instead of st_read. Would eliminate the heavy dependency on sf.

facility functions need q_columns argument

the various *GetFacilityInfo() functions need an argument to specify which columns are returned.
Should provide sensible defaults.

invalid GeoJSON

Returning simple features is causing problems:

> x <- echoAirGetFacilityInfo(xmin = '-96.407563',
+                        ymin = '30.554395',
+                        xmax = '-96.25947',
+                        ymax = '30.751984',
+                        output = 'sf')
Request failed [500]. Retrying in 1 seconds...
Request failed [500]. Retrying in 2.6 seconds...
There was a server error, try again later.
 Error: No 'type' member at object index 0 - invalid GeoJSON

Handle CAA reports with less than 10yrs of data

Need a more flexible way of handling data returned by echoGetCAAPR. Currently fails if the full record of data is not returned by ECHO.

code cleanup

update to personal email for next submission, need to confirm with cran from work email
reduce dependencies, move vignettes to pkgdown articles
tidyeval can be updated to use rlang data pronouns
add url to Description
delete JOSS paper files
README badges need to be wrapped in comments tags to get put in sidebar in the pkgdown website
...

Encoding errors

Github Actions rendered readme and pkgdown has encoding errors in the in code box. eg:

library(echor)

## echoWaterGetFacilityInfo() will return a dataframe or simple features (sf) dataframe.

df <- echoWaterGetFacilityInfo(output = "df", 
                               xmin = '-96.387509', 
                               ymin = '30.583572', 
                               xmax = '-96.281422', 
                               ymax = '30.640008',
                               p_ptype = "NPD")

head(df)
#> �[90m# A tibble: 3 x 26�[39m
#>   CWPName SourceID CWPStreet CWPCity CWPState CWPStateDistrict CWPZip
#>   �[3m�[90m<chr>�[39m�[23m   �[3m�[90m<chr>�[39m�[23m    �[3m�[90m<chr>�[39m�[23m     �[3m�[90m<chr>�[39m�[23m   �[3m�[90m<chr>�[39m�[23m    �[3m�[90m<chr>�[39m�[23m            �[3m�[90m<chr>�[39m�[23m 
#> �[90m1�[39m CENTRA… TX00027… 222 IREL… COLLEG… TX       09               77843 
#> �[90m2�[39m HEAT T… TX01065… 0.25MI S… COLLEG… TX       09               77845 
#> �[90m3�[39m TURKEY… TX00624… 3000FT W… BRYAN   TX       09               77807 
#> �[90m# … with 19 more variables: MasterExternalPermitNmbr �[3m�[90m<chr>�[90m�[23m, RegistryID �[3m�[90m<chr>�[90m�[23m,�[39m
#> �[90m#   CWPCounty �[3m�[90m<chr>�[90m�[23m, CWPEPARegion �[3m�[90m<chr>�[90m�[23m, FacDerivedHuc �[3m�[90m<chr>�[90m�[23m, FacLat �[3m�[90m<dbl>�[90m�[23m,�[39m
#> �[90m#   FacLong �[3m�[90m<dbl>�[90m�[23m, CWPTotalDesignFlowNmbr �[3m�[90m<dbl>�[90m�[23m,�[39m
#> �[90m#   CWPActualAverageFlowNmbr �[3m�[90m<dbl>�[90m�[23m, ReceivingMs4Name �[3m�[90m<chr>�[90m�[23m,�[39m
#> �[90m#   AssociatedPollutant �[3m�[90m<chr>�[90m�[23m, MsgpPermitType �[3m�[90m<chr>�[90m�[23m, CWPPermitStatusDesc �[3m�[90m<chr>�[90m�[23m,�[39m
#> �[90m#   CWPPermitTypeDesc �[3m�[90m<chr>�[90m�[23m, CWPIssueDate �[3m�[90m<date>�[90m�[23m, CWPEffectiveDate �[3m�[90m<date>�[90m�[23m,�[39m
#> �[90m#   CWPExpirationDate �[3m�[90m<date>�[90m�[23m, CWPSNCStatusDate �[3m�[90m<date>�[90m�[23m, StateAuthGen �[3m�[90m<chr>�[90m�[23m�[39m

Deal with clusters

If you query a large area that returns many values, ECHO returns a cluster with associated qid. The clusters break functional returns for users and do not provide useful info, in my opinion. The qid can be used to retrieve plant info using /cwa_rest_services.get_download

Need to modify echoWaterGetFacilityInfo() to handle clusters, then pass a query using the qid with new function that calls cwa_res_services.get_download on EPA ECHO.

I also need to figure out the best practice to inform users that the function is returning one or the other.

Return df with parameters and parameter codes

CRAN checks - vignette error

Some new errors on CRAN:

Check: re-building of vignette outputs
Result: WARN
    Error(s) in re-building vignettes:
     ...
    --- re-building 'introduction.Rmd' using rmarkdown
    Quitting from lines 310-315 (introduction.Rmd)
    Error: processing vignette 'introduction.Rmd' failed with diagnostics:
    Expecting '}'
    --- failed re-building 'introduction.Rmd'
    
    SUMMARY: processing the following file failed:
     'introduction.Rmd'
    
    Error: Vignette re-building failed.
    Execution halted

Fix echoGetEffluent example in vignette/readme

The example in the readme and vignette are returning empty dataframes.

Issues from CRAN submission

please write package names and software names in single quotes (e.g. 'ECHO') in title and description.
Please explain all acronyms (e.g. EPA) in your description to avoid misunderstandings.
Please replace \dontrun{} by \donttest{} or unwap the examples if they can be executed in less than 5 sec per Rd-file.

Fix and resubmit.

Use single function to call CAA, SDW, and CWA functions

To simplify functions for users, we can use a single function to call echoAirGetFacilityInfo(), echoWaterGetFacilityInfo() and echoSDWGetFacilityInfo().

For example:

echoGetFacilities(program = "caa", output = "df", ...)

Same for echoGetCAAPR() and echoGetEffluent():

echoGetReports(program = "caa", p_id = "xyz", ...)

Readme

Need a good readme.

Add metadata function

/cwa_rest_services.metadata provides metadata returned as JSON.

This would be prudent to add so users can look up metadata in R without having to link to webpages to find relevant info.

checks

The following checks don't pass (https://cloud.r-project.org/web/checks/check_results_echor.html)

using R version 3.6.0 (2019-04-26)
using platform: x86_64-apple-darwin15.6.0 (64-bit)
using session charset: UTF-8
checking for file ‘echor/DESCRIPTION’ ... OK
checking extension type ... Package
this is package ‘echor’ version ‘0.1.3’
package encoding: UTF-8
checking package namespace information ... OK
checking package dependencies ... OK
checking if this is a source package ... OK
checking if there is a namespace ... OK
checking for executable files ... OK
checking for hidden files and directories ... OK
checking for portable file names ... OK
checking for sufficient/correct file permissions ... OK
checking whether package ‘echor’ can be installed ... [4s/4s] OK
checking installed package size ... OK
checking package directory ... OK
checking ‘build’ directory ... OK
checking DESCRIPTION meta-information ... OK
checking top-level files ... OK
checking for left-over files ... OK
checking index information ... OK
checking package subdirectories ... OK
checking R files for non-ASCII characters ... OK
checking R files for syntax errors ... OK
checking whether the package can be loaded ... OK
checking whether the package can be loaded with stated dependencies ... OK
checking whether the package can be unloaded cleanly ... OK
checking whether the namespace can be loaded with stated dependencies ... OK
checking whether the namespace can be unloaded cleanly ... OK
checking loading without being on the library search path ... OK
checking dependencies in R code ... NOTE
Namespace in Imports field not imported from: ‘lubridate’
  All declared Imports should be used.
checking S3 generic/method consistency ... OK
checking replacement functions ... OK
checking foreign function calls ... OK
checking R code for possible problems ... [3s/4s] OK
checking Rd files ... OK
checking Rd metadata ... OK
checking Rd cross-references ... OK
checking for missing documentation entries ... OK
checking for code/documentation mismatches ... OK
checking Rd \usage sections ... OK
checking Rd contents ... OK
checking for unstated dependencies in examples ... OK
checking installed files from ‘inst/doc’ ... OK
checking files in ‘vignettes’ ... OK
checking examples ... [1s/1s] OK
checking for unstated dependencies in ‘tests’ ... OK
checking tests ... [2s/2s] OK
  Running ‘testthat.R’ [1s/1s]
checking for unstated dependencies in vignettes ... OK
checking package vignettes in ‘inst/doc’ ... OK
checking running R code from vignettes ... NONE
checking re-building of vignette outputs ... [3s/23s] OK
checking PDF version of manual ... OK
DONE
Status: 1 NOTE

spelling check

check_spelling seems to cause issues in build test:

> if(requireNamespace('spelling', quietly = TRUE))
+   spelling::spell_check_test(vignettes = TRUE, error = FALSE,
+                              skip_on_cran = TRUE)
Error in read_xml.raw(charToRaw(enc2utf8(x)), "UTF-8", ..., as_html = as_html,  : 
  PCDATA invalid Char value 27 [9]
Calls: <Anonymous> ... xml_find_all -> <Anonymous> -> read_xml.character -> read_xml.raw
Execution halted

Might be best to remove it and run spellchecks manually when writing documentation and vignettes.

`dots_values()` is getting soft-deprecated

No hurry as it won't cause warnings to end users yet. But could you please look into using dots_list() or list2() instead?

echoGetCAAPR doesn't pass tests

Running tests on new functions echoGetCAAPR raises following notes:

checking R code for possible problems ... NOTE
echoGetCAAPR: no visible binding for global variable 'Year'
echoGetCAAPR: no visible binding for global variable 'Discharge'
echoGetCAAPR: no visible binding for global variable 'Year1'
echoGetCAAPR: no visible binding for global variable 'Year10'
Undefined global functions or variables:
  Discharge Year Year1 Year10

Probably due to using dplyr and tidyr in this function.

Consider doing this with base R methods or understand how to use these undefined variables.

Corporate Compliance Screener

More new endpoints:
https://echo.epa.gov/tools/web-services/compliance-screener#/Corporate%20Compliance%20Screener

echoGetEffluent returns list in a list

The dataframe returned by echoGetEffluent has list within a list. Ideally, a clean dataframe/tibble should be returned. It might be better to call download_effluent_chart with this function.

Examples returning empty dataframes

Impacted examples:

echoWaterGetFacilityInfo()

use httptest to build vignettes

Execution halted in loop with echoGetEffluent() due to curl_fetch_memory() error

Long batch processing jobs can fail when connection is reset by peer. Error occurs in curl_fetch_memory call -- this may be a curl issue, or this may be handleable in this module, unsure which is the approach. I am going to look into the code herein to see if another approach exists that can handle resets gracefully.

Error in curl::curl_fetch_memory(url, handle = handle) :
  SSL read: error:00000000:lib(0):func(0):reason(0), errno 104
Calls: echoGetEffluent ... request_fetch -> request_fetch.write_memory -> <Anonymous> -> .Call

Release echor 0.1.7

Prepare for release:

Submit to CRAN:

usethis::use_version('patch')
devtools::submit_cran()
Approve email

Wait for CRAN...

progress_estimated is depreciated

Suggest using progress.
Do I want to import this for one function?

echo returns qcolumns 1 and 2 everytime

https://echo.epa.gov/tools/web-services/facility-search-water#!/Facility_Information/get_cwa_rest_services_get_download

no matter what qcolumns are specified, echo always provides columns 1 and 2.

Need to do some list matching to see if the user includes 1 and 2 in the q columns list, and always include 1 and 2 in the col_types argument...

New Loading Tool endpoints

Documentation is thin, but there are a bunch of new endpoints here:
https://echo.epa.gov/tools/web-services/loading-tool

Add helper function to download many DMR effluent reports

A helper function to using purrr::map to download DMR effluent reports would be useful since we can only download one at a time. Something like:

df <- tibble(PermitId = c("TX0021474", "TX0023051", "TXG130053"))

# create the progress bar with a dplyr function. 
pb <- progress_estimated(length(df$PermitId))

df <- df %>%
mutate(reports = purrr::pmap(., ~ {
    # update the progress bar (tick()) and print progress (print())
        pb$tick()$print()
    
        echoGetEffluent(
          p_id = ..2,
          parameter = "50050",
          start_date = "01/01/2008",
          end_date = "12/30/2017")
    }
  ))

Concerns: Might be ripe for abusive behavior of ECHO server.

downloadDMRs returns error

MWE doesn't work:

df <- data_frame("p_id" = c('TX0119407', 'TX0132187', 'TX040237'))
df <- downloadDMRs(df, p_id)

Error in echoGetEffluent(p_id = ..1, ...) : 
  formal argument "p_id" matched by multiple actual arguments

CRAN checks donttest

Also on CRAN an additional issue list:

* checking examples with --run-donttest ... [3s/27s] ERROR
Running examples in ‘echor-Ex.R’ failed
The error most likely occurred in:

> ### Name: echoGetCAAPR
> ### Title: Download EPA ECHO emissions inventory report data
> ### Aliases: echoGetCAAPR
> 
> ### ** Examples
> 
> ## No test: 
> ## This example requires an internet connection to run
> 
> echoGetCAAPR(p_id = '110000350174')
Error: Can't subset columns that don't exist.
✖ Column `Year1` doesn't exist.
Backtrace:
     █
  1. └─echor::echoGetCAAPR(p_id = "110000350174")
  2.   ├─tidyr::gather_(...)
  3.   └─tidyr:::gather_.data.frame(...)
  4.     ├─tidyr::gather(...)
  5.     └─tidyr:::gather.data.frame(...)
  6.       ├─base::unname(tidyselect::vars_select(tbl_vars(data), !!!quos))
  7.       └─tidyselect::vars_select(tbl_vars(data), !!!quos)
  8.         └─tidyselect:::eval_select_impl(...)
  9.           ├─tidyselect:::with_subscript_errors(...)
 10.           │ ├─base::tryCatch(...)
 11.           │ │ └─base:::tryCatchList(expr, classes, parentenv, handlers)
 12.           │ │   └─base:::tryCatchOne(expr, names, parentenv, handlers[[1L]])
 13.           │ │     └─base:::doTryCatch(return(expr), name, parentenv, handler)
 14.           │ └─tidyselect:::instrument_base_errors(expr)
 15.           │   └─base::withCallingHand
Execution halted

Tests for air facilities functions

Write new tests for:
echoGetCAAPR
echoAirGetFacilityInfo

get_download does not support queries returning more than 100k records

I'm having trouble running queries that return more than 100k results. For example, when I run the following:

facilities <-
  echoWaterGetFacilityInfo(output = 'df',
                           qcolumns = "1",
                           p_st = "GA")

I get a dataframe that looks like this:

Is this expected behavior?

What seems like is happening is that the original query has a result set of around 122k records, and the get_download endpoint returns an error when requesting a result of more than 100k records. Trying this query directly via the ECHO Swagger:

https://echodata.epa.gov/echo/cwa_rest_services.get_download?output=CSV&qid=MY_QUERY_ID&qcolumns=1

I get the following:

<b>FILE OUTPUT FAILED</b>
<br />
<b>ERROR MESSAGE:</b> This query of 122923 exceeds the maximum rows allowed for a download of 100000.  Please modify your selection.

I think what should happen in this case is that echor should fall back to using the get_qid endpoint, which seems to allow pagination.

CRAN Check

Functions are erroring on HTTP 500 and not returning a "graceful" message instead of error or warning.

* using log directory ‘/data/blackswan/ripley/R/packages/tests-devel/echor.Rcheck’
* using R Under development (unstable) (2023-06-12 r84534)
* using platform: x86_64-pc-linux-gnu
* R was compiled by
    gcc (GCC) 11.3.1 20220421 (Red Hat 11.3.1-2)
    GNU Fortran (GCC) 11.3.1 20220421 (Red Hat 11.3.1-2)
* running under: Fedora 34 (Workstation Edition)
* using session charset: UTF-8
* checking for file ‘echor/DESCRIPTION’ ... OK
* checking extension type ... Package
* this is package ‘echor’ version ‘0.1.8’
* package encoding: UTF-8
* checking package namespace information ... OK
* checking package dependencies ... OK
* checking if this is a source package ... OK
* checking if there is a namespace ... OK
* checking for executable files ... OK
* checking for hidden files and directories ... OK
* checking for portable file names ... OK
* checking for sufficient/correct file permissions ... OK
* checking whether package ‘echor’ can be installed ... OK
* checking package directory ... OK
* checking DESCRIPTION meta-information ... OK
* checking top-level files ... OK
* checking for left-over files ... OK
* checking index information ... OK
* checking package subdirectories ... OK
* checking R files for non-ASCII characters ... OK
* checking R files for syntax errors ... OK
* checking whether the package can be loaded ... OK
* checking whether the package can be loaded with stated dependencies ... OK
* checking whether the package can be unloaded cleanly ... OK
* checking whether the namespace can be loaded with stated dependencies ... OK
* checking whether the namespace can be unloaded cleanly ... OK
* checking loading without being on the library search path ... OK
* checking dependencies in R code ... OK
* checking S3 generic/method consistency ... OK
* checking replacement functions ... OK
* checking foreign function calls ... OK
* checking R code for possible problems ... OK
* checking Rd files ... OK
* checking Rd metadata ... OK
* checking Rd line widths ... OK
* checking Rd cross-references ... OK
* checking for missing documentation entries ... OK
* checking for code/documentation mismatches ... OK
* checking Rd \usage sections ... OK
* checking Rd contents ... OK
* checking for unstated dependencies in examples ... OK
* checking examples ... OK
* checking examples with --run-donttest ... [2s/18s] ERROR
Running examples in ‘echor-Ex.R’ failed
The error most likely occurred in:

> ### Name: echoAirGetFacilityInfo
> ### Title: Downloads EPA ECHO permitted air emitter information
> ### Aliases: echoAirGetFacilityInfo
> 
> ### ** Examples
> 
> ## No test: 
> ## These examples require an internet connection to run
> 
> ## Retrieve table of facilities by bounding box
> echoAirGetFacilityInfo(xmin = '-96.407563',
+ ymin = '30.554395',
+ xmax = '-96.25947',
+ ymax = '30.751984',
+ output = 'df')
# A tibble: 40 × 7
   AIRName                    SourceID AIRStreet AIRCity AIRState FacLat FacLong
   <chr>                      <chr>    <chr>     <chr>   <chr>     <dbl>   <dbl>
 1 AGGIE CLEANERS             0600000… 111 COLL… COLLEG… TX         30.6   -96.3
 2 ALENCO, DIV OF RELIANT BL… TX00000… 615 CARS… BRYAN   TX         30.6   -96.4
 3 ALL SEASONS 1 HR CLEANERS  0600000… 2501 TEX… COLLEG… TX         30.6   -96.3
 4 BLUEBONNET PAVING          TX00000… HWY. 60,… COLLEG… TX         30.6   -96.3
 5 BRIARCREST DRY CLEANERS    0600000… 1887 BRI… BRYAN   TX         30.7   -96.3
 6 BRYAN CERAMICS PLANT       TX00000… 1500 IND… BRYAN   TX         30.7   -96.4
 7 BRYAN CLEANERS & LAUNDRY   0600000… 1803 HOL… COLLEG… TX         30.6   -96.3
 8 BRYAN HICKS GAS PLANT      TX00000… 3747 OLD… BRYAN   TX         30.7   -96.3
 9 CITGO PETROLEUM CORPORATI… TX00000… 1714 FIN… BRYAN   TX         30.7   -96.4
10 CITY OF BRYAN              TX00000… 1.5 MI W… BRYAN   TX         30.6   -96.4
# ℹ 30 more rows
> 
> ## Retrieve a simple features dataframe by bounding box
> spatialdata <- echoAirGetFacilityInfo(xmin = '-96.407563',
+ ymin = '30.554395',
+ xmax = '-96.25947',
+ ymax = '30.751984',
+ output = 'sf')
Request failed [500]. Retrying in 1 seconds...
Request failed [500]. Retrying in 1.5 seconds...
Error in resp_check(request) : Internal Server Error (HTTP 500).
Calls: echoAirGetFacilityInfo -> getGeoJson -> resp_check -> stop_for_status
Execution halted
* checking for unstated dependencies in ‘tests’ ... OK
* checking tests ...
  Running ‘testthat.R’
 OK
* checking PDF version of manual ... OK
* checking for non-standard things in the check directory ... OK
* checking for detritus in the temp directory ... OK
* checking for new files in some other directories ... OK
* DONE

Status: 1 ERROR
See
  ‘/data/blackswan/ripley/R/packages/tests-devel/echor.Rcheck/00check.log’
for details.

Command exited with non-zero status 1
Time 0:55.23, 32.75 + 5.62

Fix notes and warn on CRAN

Need to remove lazydata on description
Not 100% what is going on here, but check plots created in the vignette causing the error: https://www.r-project.org/nosvn/R.check/r-release-macos-arm64/echor-00check.html
Old release in windows is failing but I have no idea why: https://www.r-project.org/nosvn/R.check/r-oldrel-windows-ix86+x86_64/echor-00check.html

Release echor 0.1.5

Prepare for release:

devtools::build_readme()
Check current CRAN check results
devtools::check(remote = TRUE, manual = TRUE)
devtools::check_win_devel()
rhub::check_for_cran(env_vars = c(R_CHECK_FORCE_SUGGESTS = "true", R_CHECK_CRAN_INCOMING_USE_ASPELL = "true", R_COMPILE_AND_INSTALL_PACKAGES = "always"), platforms = c("windows-x86_64-release", "windows-x86_64-devel", "macos-highsierra-release-cran")
revdepcheck::revdep_check(num_workers = 4)
Update cran-comments.md
Polish NEWS

Submit to CRAN:

usethis::use_version('patch')
devtools::submit_cran()
Approve email

Wait for CRAN...

Accepted 🎉
usethis::use_github_release()
usethis::use_dev_version()

Release echor 0.1.8

Prepare for release:

Submit to CRAN:

usethis::use_version('patch')
devtools::submit_cran()
Approve email

Wait for CRAN...

Accepted 🎉
git push
usethis::use_github_release()
git push

Release echor 0.1.6

Prepare for release:

Submit to CRAN:

usethis::use_version('patch')
devtools::submit_cran()
Approve email

Wait for CRAN...

Accepted 🎉
usethis::use_github_release()
usethis::use_dev_version()

Description file malformed

Check raises:

Malformed Description field: should contain one or more complete sentences.

Should be easy fix.

sdw endpoints are restricted to results <= 100,000

The effluent endpoint returns multipage data when more than 100,000 records are returned. The SDW endpoints return an error when more than 100,000 records are returned:

curl -X GET "https://echodata.epa.gov/echo/sdw_rest_services.get_systems?p_st=CA%2CGA%2CFL%2COH%2COR%2CTX%2CVA%2CWA" -H  "accept: application/json"

{
  "Results": {
    "Error": {
      "ErrorMessage": "Rows Returned would be 101380. Queryset Limit would be exceeded - please make search parameters more selective."
    }
  }
}

the get_download endpoint still returns a full csv for <100,000 rows. Would make sense return a message when queryset is exceeded and remove the code for parsing pages returned by get_qid endpoint.

mps9506 / echor Goto Github PK

echor's People

Contributors

Stargazers

Watchers

Forkers

echor's Issues

checklist

Recommend Projects

Recommend Topics

Recommend Org

Jobs