ropensci / chirps Goto Github PK

View Code? Open in Web Editor NEW

30.0 7.0 12.0 3.65 MB

API Client for CHIRPS

Home Page: https://docs.ropensci.org/chirps/

License: MIT License

R 85.96% TeX 14.04%

climatology precipitation-data rstats chirps

chirps's Introduction

chirps: API Client for CHIRPS and CHIRTS

Overview

chirps provides the API Client for the Climate Hazards Center 'CHIRPS' and 'CHIRTS'. The 'CHIRPS' data is a quasi-global (50°S – 50°N) high-resolution (0.05 arc-degrees) rainfall data set, which incorporates satellite imagery and in-situ station data to create gridded rainfall time series for trend analysis and seasonal drought monitoring. 'CHIRTS' is a quasi-global (60°S – 70°N), high-resolution data set of daily maximum and minimum temperatures. For more details on 'CHIRPS' and 'CHIRTS' data please visit its official home page https://www.chc.ucsb.edu/data.

Quick start

From CRAN

The stable version is available through CRAN.

install.packages("chirps")

From GitHub

A development version that may have new features or bug fixes is available through GitHub.

library("remotes")

install_github("ropensci/chirps", build_vignettes = TRUE)

Example

Fetch CHIRPS data from three points across the Tapajós National Forest (Brazil) from in Jan-2017. The default procedure will download the COG files from the CHIRPS server and handle it internally using the package terra. This is more interesting when dealing with hundreds of points and days. Data can be returned as a matrix using the argument as.matrix = TRUE.

library("chirps")

lonlat <- data.frame(lon = c(-55.0281,-54.9857, -55.0714),
                     lat = c(-2.8094, -2.8756, -3.5279))

dates <- c("2017-01-01", "2017-01-31")

dat <- get_chirps(lonlat, dates, server = "CHC", as.matrix = FALSE)

For a faster download of few datapoints (~ 10 datapoints), the argument server = "ClimateSERV" can be used

library("chirps")

lonlat <- data.frame(lon = c(-55.0281,-54.9857, -55.0714),
                     lat = c(-2.8094, -2.8756, -3.5279))

dates <- c("2017-01-01", "2017-01-31")

dat <- get_chirps(lonlat, dates, server = "ClimateSERV", as.matrix = FALSE)

Going further

The full functionality of chirps is illustrated in the package vignette. The vignette can be found on the package website or from within R once the package has been installed, e.g. via:

vignette("Overview", package = "chirps")

Use of CHIRPS data

While chirps does not redistribute the data or provide it in any way, we encourage users to cite Funk et al. (2015) when using CHIRPS and Funk et al. (2019) when using CHIRTS

Funk C., Peterson P., Landsfeld M., … Michaelsen J. (2015). The climate hazards infrared precipitation with stations—a new environmental record for monitoring extremes. Scientific Data, 2, 150066. https://doi.org/10.1038/sdata.2015.66

Funk, C., Peterson, P., Peterson, S., … Mata, N. (2019). A high-resolution 1983–2016 TMAX climate data record based on infrared temperatures and stations by the climate hazard center. Journal of Climate, 32(17), 5639–5658. https://doi.org/10.1175/JCLI-D-18-0698.1

chirps's People

Contributors

Stargazers

Watchers

Forkers

patperu agrdatasci martin85geo anacarolinafxavier19 ani-ghosh robertsaldivar macdonaldmichael1 david-mage caroline120 ecor jhannie495 reduarte

chirps's Issues

Correct vignette

Vignette errors out upon knitting.

Don't know how to automatically pick scale for object of type function. Defaulting to continuous.
Quitting from lines 88-105 (Overview.Rmd) 
Error: Aesthetics must be valid data columns. Problematic aesthetic(s): x = date. 
Did you mistype the name of a data column or forget to add stat()?

Execution halted

get_chirps is extremely slow

I have been experimenting with get_chirps today and finding really slow speeds, both locally and on servers.

lonlat<-data.frame(lon=c(-4.314916,-4.225084),lat=c(31.30144 ,31.37854))
dates<-as.character(c("1981-01-02","1981-12-31"))
system.time(data<-chirps::get_chirps(lonlat,
dates=dates,
server = "CHC"))

this took 250sec, but is only 728 rows data

Incorrect conditions in .validate_dates (internal functions)

At the moment it is not possible to get values from first and last dates as conditions in .validate_dates are strict :

......................
# no older than past date
cond2 <- xmin > past

# no later then present date
cond3 <- xmax < present
......................

At least for the condition on past date, I believe this should be greater or equal.

Best,
Victor

1 NOTE on package check

 There are ::: calls to the package's namespace in its code. A package
    almost never needs to use ::: for its own objects:
    ‘.get_CHIRTS_tiles_CHC’

Tests need to be more specific and CRAN friendly

Tests requiring connectivity need to use skip_on_cran().

Some tests could be written to be more specific for checking results from the query to ensure that the function is working properly.

prepare for CRAN submission

Check for any possible issue before sending to CRAN. What to you think @adamhsparks ?

Offer alternative datasets from ClimateServ, like IMERG

Given that ClimateServ's precipitation data has stopped updating around May 14th, and that IMERG has recently been added, and that, according to this paper and its lit review, IMERG is more accurate than CHIRPS (but less accurate than JAXA's GSMaP), I would suggest making this package more flexible in order to download other datasets, like corrected CHIRPS and IMERG.

I wouldn't change the logo, though. It works quite nice for stickers and stuff like that.

Subscript out of bounds.

Example case No 4: input SpatExtent fails

Dear chirps package developers

When I try example case 4 (with SpatExtent as object argument) from https://docs.ropensci.org/chirps/reference/get_chirps.html as follows:

library(chirps)
library(terra)

# Case 4: input SpatExtent and return a raster within the extent
area <- ext(c(-66, -64, -6, -4))

dates <- c("2017-12-15", "2017-12-31")

r4 <- get_chirps(area, dates, server = "CHC")

I get the following error message:

Error in as.data.frame.default(object) :
cannot coerce class ‘structure("SpatExtent", package = "terra")’ to a data.frame

This version of example case 4 is not present in the reference from cran so I'm guessing it is no longer supported. Being able to download chirps data in an area of interest would, however, be a very useful feature!

Would you have a recommendation on how the chirps r package could be used to obtain data within an area of interest?

Many greetings

Using \donttest for examples is no longer suggested

CRAN decided to start testing donttest{} examples without telling anyone until they test and your package fails.

I've moved nasapower, which failed, to use the new examplesIf from the development version of roxygen2. You just need to install it from GitHub, e.g. remotes::install_github("r-lib/roxygen2")
and then the example portion becomes:

#' @examplesIf interactive()
example_goes_here()

According to CRAN policy, server failures should not be R errors

CRAN has a policy (which I don't agree with) that server response failures should fail "gracefully" without an error message. To me, I'd prefer the script to stop and give me an informative message as we've done in chirps. I know I have a collaborator who contacted me after I made this change as well because they were using a tryCatch() to wrap a function from bomrang and this broke that functionality.

However, that's not what CRAN wants.

I've a tryCatch() function we can look at and discuss in bomrang that might be useful to refer to. It returns an invisible NULL if the server times out after checking network functionality and server responses, etc.

https://github.com/ropensci/bomrang/blob/77f14342188ea1e48362eb0bfdb3c60dd9510ab4/R/internal_functions.R#L13

Others in rOpenSci or the reviewers might be good to consult with as well.

Odd vignette on CRAN

Somehow the vignette on CRAN is odd. The citations keys are not working and authors are not showing. Check for the reasons on that issue.

https://cran.r-project.org/web/packages/chirps/vignettes/Overview.html

Chirps data fetching not working

Hi, I am a mac user with RStudio version 4. I am finding that the data fetching script in the Readme is not able to fetch the data. I am running the following commands (directly from the Readme):

lonlat <- data.frame(lon = c(-55.0281,-54.9857, -55.0714),
lat = c(-2.8094, -2.8756, -3.5279))

dates <- c("2017-01-01", "2017-12-31")

dat <- get_chirps(lonlat, dates)

At this point, get_chirps times out. I have not been able to access the chirps ftp server either (ftp://ftp.chg.ucsb.edu/pub/org/chg/products/CHIRPS-2.0/global_daily/tifs/p05/)

Timeout when trying to get data for Peru

Minimal reproducible example below (which was originally of Honduras, hence object name) - with withTimeout() to show timeout (I waited over a minute but still got no response):

library(chirps)
library(dplyr)
#> 
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#> 
#>     filter, lag
#> The following objects are masked from 'package:base':
#> 
#>     intersect, setdiff, setequal, union
library(sf)
#> Linking to GEOS 3.8.0, GDAL 3.0.2, PROJ 6.2.1
countries = spData::world
honduras = countries %>% 
  filter(name_long == "Peru")
sample_sites = st_sample(honduras, 3) %>% 
  st_as_sf()
#> although coordinates are longitude/latitude, st_intersects assumes that they are planar
#> although coordinates are longitude/latitude, st_intersects assumes that they are planar
#> although coordinates are longitude/latitude, st_intersects assumes that they are planar
#> although coordinates are longitude/latitude, st_intersects assumes that they are planar
getOption('timeout')
#> [1] 60
R.utils::withTimeout({
  dat = get_chirps(sample_sites, dates =c("2013-01-01","2013-02-01"))
}, timeout = 10)
#> Warning in st_buffer.sfc(st_geometry(x), dist, nQuadSegs, endCapStyle =
#> endCapStyle, : st_buffer does not correctly buffer longitude/latitude data
#> dist is assumed to be in decimal degrees (arc_degrees).
#> Error: 
#> Something went wrong with the query, no data were returned. Please see <https://climateserv.servirglobal.net/> for potential server issues.

^{Created on 2020-02-02 by the reprex package (v0.3.0)}

Session info

devtools::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value                       
#>  version  R version 3.6.2 (2019-12-12)
#>  os       Ubuntu 18.04.3 LTS          
#>  system   x86_64, linux-gnu           
#>  ui       X11                         
#>  language en_GB:en                    
#>  collate  en_GB.UTF-8                 
#>  ctype    en_GB.UTF-8                 
#>  tz       Europe/London               
#>  date     2020-02-02                  
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package     * version    date       lib
#>  assertthat    0.2.1      2019-03-21 [2]
#>  backports     1.1.5      2019-10-02 [1]
#>  callr         3.4.0      2019-12-09 [1]
#>  chirps      * 0.0.6      2020-02-02 [1]
#>  class         7.3-15     2019-01-01 [2]
#>  classInt      0.4-3      2019-12-04 [1]
#>  cli           2.0.1      2020-01-08 [1]
#>  crayon        1.3.4      2017-09-16 [2]
#>  crul          0.9.0      2019-11-06 [2]
#>  curl          4.3        2019-12-02 [2]
#>  DBI           1.1.0      2019-12-15 [2]
#>  desc          1.2.0      2018-05-01 [2]
#>  devtools      2.2.1      2019-09-24 [1]
#>  digest        0.6.23     2019-11-23 [1]
#>  dplyr       * 0.8.3      2019-07-04 [2]
#>  e1071         1.7-3      2019-11-26 [2]
#>  ellipsis      0.3.0      2019-09-20 [3]
#>  evaluate      0.14       2019-05-28 [2]
#>  fansi         0.4.1      2020-01-08 [1]
#>  fs            1.3.1      2019-05-06 [2]
#>  glue          1.3.1      2019-03-12 [2]
#>  highr         0.8        2019-03-20 [3]
#>  htmltools     0.4.0.9002 2020-01-25 [1]
#>  httpcode      0.2.0      2016-11-14 [2]
#>  jsonlite      1.6        2018-12-07 [2]
#>  KernSmooth    2.23-16    2019-10-15 [4]
#>  knitr         1.26       2019-11-12 [1]
#>  lwgeom        0.1-7      2019-05-06 [1]
#>  magrittr      1.5        2014-11-22 [2]
#>  memoise       1.1.0      2017-04-21 [3]
#>  pillar        1.4.3      2019-12-20 [1]
#>  pkgbuild      1.0.6      2019-10-09 [2]
#>  pkgconfig     2.0.3      2019-09-22 [2]
#>  pkgload       1.0.2      2018-10-29 [3]
#>  prettyunits   1.1.0      2020-01-09 [1]
#>  processx      3.4.1      2019-07-18 [2]
#>  ps            1.3.0      2018-12-21 [2]
#>  purrr         0.3.3      2019-10-18 [1]
#>  R.methodsS3   1.7.1      2016-02-16 [2]
#>  R.oo          1.23.0     2019-11-03 [1]
#>  R.utils       2.9.2      2019-12-08 [1]
#>  R6            2.4.1      2019-11-12 [2]
#>  Rcpp          1.0.3      2019-11-08 [1]
#>  remotes       2.1.0      2019-06-24 [1]
#>  rlang         0.4.4      2020-01-28 [1]
#>  rmarkdown     2.0.6      2020-01-13 [1]
#>  rprojroot     1.3-2      2018-01-03 [2]
#>  sessioninfo   1.1.1      2018-11-05 [3]
#>  sf          * 0.8-1      2020-01-28 [1]
#>  spData        0.3.2      2020-01-13 [1]
#>  spDataLarge   0.3.1      2020-01-12 [1]
#>  stringi       1.4.5      2020-01-11 [1]
#>  stringr       1.4.0      2019-02-10 [2]
#>  testthat      2.3.1      2019-12-01 [2]
#>  tibble        2.1.3      2019-06-06 [2]
#>  tidyselect    1.0.0      2020-01-27 [1]
#>  triebeard     0.3.0      2016-08-04 [2]
#>  units         0.6-5      2019-10-08 [1]
#>  urltools      1.7.3      2019-04-14 [2]
#>  usethis       1.5.1.9000 2020-01-31 [1]
#>  withr         2.1.2      2018-03-15 [2]
#>  xfun          0.12       2020-01-13 [1]
#>  yaml          2.2.0      2018-07-25 [3]
#>  source                                     
#>  CRAN (R 3.6.0)                             
#>  CRAN (R 3.6.1)                             
#>  CRAN (R 3.6.1)                             
#>  Github (agrobioinfoservices/chirps@1858861)
#>  standard (@7.3-15)                         
#>  Github (r-spatial/classInt@7bc8a82)        
#>  CRAN (R 3.6.2)                             
#>  standard (@1.3.4)                          
#>  CRAN (R 3.6.1)                             
#>  CRAN (R 3.6.2)                             
#>  CRAN (R 3.6.2)                             
#>  standard (@1.2.0)                          
#>  CRAN (R 3.6.1)                             
#>  CRAN (R 3.6.1)                             
#>  CRAN (R 3.6.1)                             
#>  CRAN (R 3.6.1)                             
#>  CRAN (R 3.6.1)                             
#>  CRAN (R 3.6.0)                             
#>  CRAN (R 3.6.2)                             
#>  CRAN (R 3.6.0)                             
#>  CRAN (R 3.5.3)                             
#>  CRAN (R 3.5.3)                             
#>  Github (rstudio/htmltools@e07546c)         
#>  CRAN (R 3.5.3)                             
#>  standard (@1.6)                            
#>  CRAN (R 3.6.1)                             
#>  CRAN (R 3.6.1)                             
#>  CRAN (R 3.6.1)                             
#>  CRAN (R 3.5.2)                             
#>  CRAN (R 3.5.0)                             
#>  CRAN (R 3.6.2)                             
#>  CRAN (R 3.6.1)                             
#>  CRAN (R 3.6.1)                             
#>  CRAN (R 3.5.1)                             
#>  CRAN (R 3.6.2)                             
#>  CRAN (R 3.6.1)                             
#>  CRAN (R 3.5.3)                             
#>  CRAN (R 3.6.1)                             
#>  CRAN (R 3.5.2)                             
#>  CRAN (R 3.6.1)                             
#>  CRAN (R 3.6.1)                             
#>  CRAN (R 3.6.1)                             
#>  CRAN (R 3.6.1)                             
#>  CRAN (R 3.6.1)                             
#>  CRAN (R 3.6.2)                             
#>  Github (rstudio/rmarkdown@1c859f4)         
#>  CRAN (R 3.5.3)                             
#>  CRAN (R 3.5.1)                             
#>  CRAN (R 3.6.2)                             
#>  Github (nowosad/spData@13d5ead)            
#>  Github (nowosad/spDataLarge@1f52612)       
#>  CRAN (R 3.6.2)                             
#>  standard (@1.4.0)                          
#>  CRAN (R 3.6.1)                             
#>  CRAN (R 3.6.0)                             
#>  CRAN (R 3.6.2)                             
#>  CRAN (R 3.5.3)                             
#>  CRAN (R 3.6.1)                             
#>  CRAN (R 3.6.0)                             
#>  Github (r-lib/usethis@dbac27b)             
#>  CRAN (R 3.5.3)                             
#>  CRAN (R 3.6.2)                             
#>  CRAN (R 3.5.1)                             
#> 
#> [1] /home/robin/R/x86_64-pc-linux-gnu-library/3.6
#> [2] /usr/local/lib/R/site-library
#> [3] /usr/lib/R/site-library
#> [4] /usr/lib/R/library

Great idea for a package, FYI here's some analysis of CHIRPS data that may be worth linking to, great example of using it with the stars package: https://github.com/loreabad6/chirps-stars

CHIRTS temperature data is available

Funk et al recently released CHIRTS (temperature). I will keep track on how it goes and look for any API service to be implemented

https://www.chc.ucsb.edu/data/chirtsdaily

ClimateSERV server results in a failure

e.g.,

> dates = c("2017-12-15","2017-12-31")
> lonlat = c(121.25562, 14.6774)
> names(lonlat) <- c("lon", "lat")

> get_chirps(object = data.frame(as.list(lonlat)),
             dates = dates,
             server = "ClimateSERV")

Fetching data from ClimateSERV server 

Getting your request...

 Error in data.frame(..., check.names = FALSE) : 
arguments imply differing number of rows: 0, 121

get_chips local

Maybe we could write a function to download chirps from the ftp server as an alternative service to get_chirps(). What do you think @adamhsparks ?

Error: try to increase the buffer area with 'dist'

I was using Chirps to get precipitation data of India but I keep on running into the following error:

Fetching data from ClimateSERV 

Getting your request...

Error in .GET(gjson = gj, dates = dates_inter, operation = operation,  : 
  Failed to get valid values,
         try to increase the buffer area with 'dist'

Do you know what might be causing this. The code used is from another issue and is given below:

countries = spData::world
honduras = countries %>% 
    filter(name_long == "India")
sample_sites = st_sample(honduras, 3) %>% 
    st_as_sf()
dat = get_chirps(sample_sites, dates =c("2013-01-01","2013-12-01"), server = "ClimateSERV")

## Also tried using longitude and latitude obtained from https://www.geodatos.net/en/coordinates/india but same error persists
location <- data.frame(long = 78.96288, lat = 20.593684)
d <- get_chirps(location, dates = c("2019-01-01", "2019-12-31"),
                server = "ClimateSERV")

What am I doing wrong?

Error: lexical error: invalid string in json text.

This error occur when data from a large time series is requested (> 1 year). The problem may be in parsing the json file within the function. When I run the code line by line it works perfectly, and returns the chirps data. It also works in the browser.

When chirps is required for a time series of ~ 1 year it works well. But this might be a problem for users trying to capture a full picture of a time series.

set.seed(1232)
lonlat <- data.frame(lon = runif(2, -55, -54), lat = runif(2, -3, -2.7))

dates <- c("1982-01-01","2000-12-31")

get_chirps(lonlat, dates)


 Error: lexical error: invalid string in json text.
                                       need to send id
                     (right here) ------^ 
5. parse_string(txt, bigint_as_char) 
4. parseJSON(txt, bigint_as_char) 
3. parse_and_simplify(txt = txt, simplifyVector = simplifyVector, 
    simplifyDataFrame = simplifyDataFrame, simplifyMatrix = simplifyMatrix, 
    flatten = flatten, ...) 
2. jsonlite::fromJSON(d) at get_chirps.R#154
1. get_chirps(lonlat, dates)

date 1981-01-01 is not available

Hello!

First of all, thanks for the package. It's very usefull.

I realized that day 1981-01-01 is not available. Its just one date, so i don´t know if matters to you.

Best whishes

Not possible to get data for only one or two days

This line checking the dates,

chirps/R/internal_functions.R

Line 303 in 7f60a8e

cond1 <- as.integer(xmax - xmin) > 1

works such that if I wanted to fetch data for 2015-01-15 - 2015-01-16 chirps will fail with the error message that the dates are out of sync with available data or that they are reversed (twisted).

get_chirps as a S3 method

From R. Bivand comments.

Write get_chirps as a S3 method that can behave different depending on which input is provided. sf object, geojson or data.frame.

Error: Internal server Error HTTP 500

I have tried to download some data using the following code:

library("chirps")
lonlat <- data.frame(lon = c(-55.0281,-54.9857, -55.0714), lat = c(-2.8094, -2.8756, -3.5279))
dates <- c("2017-01-01", "2017-12-31")
dat <- get_chirps(lonlat, dates)

which results in the following error message:

Error: Internal server Error (HTTP 500)

I thought the problem came from my laptop, but I tried with two different and got the same issues. Do you know where it might come from?

Thank you!

package title

For discussion.

Given that the data are mainly known by the acronym, CHIRPS, should we consider shortening the package title in DESCRIPTION to make it more readable and easier to identify?

Current:
API Client for the Climate Hazards Group InfraRed Precipitation with Station Data

Proposed:
API Client for CHIRPS

or
CHIRPS API Client

I'm bringing this up because I honestly wouldn't know what the package was from the current title given that I only knew the data as "CHIRPS".

Review use of supressWarnings()

In queries to the server it is very useful to capture warnings and communication from the server. If we are suppressing them, that's not the most ideal.

I'm all for slimming dependencies but using {httr} or {crul} isn't a bad thing here, I don't think, since they're designed for this and allow for further functionality like checking the server for a response before sending the query and stopping the function to let the user know the server isn't responding, etc.

For comment and discussion.

If I were your reviewer I'd suggest using one of these packages to remove this portion of the code and gain added functionality (web mocking, etc.).

Extracting time series

extnt <- as(extent(c(xmin= 31, xmax= 49, ymin= 2, ymax= 20)), 'SpatialPolygons')
crs(extnt) <- "+proj=longlat +datum=WGS84 +no_defs"
area <- ext(extnt)
dates <- c("2017-12-15","2017-12-31")
get_chirps(area, dates, server = "CHC", as.raster = TRUE)

this worked well for me to extract data for the given extent. However, I could not able to extract sample points from the domain.
#43
spt <- st_sample(area, 30)
spt <- st_as_sf(spt)
dft <- get_chirps(spt, dates = c("1990-01-01","2018-12-31"))
p_ind <- precip_indices(dft, timeseries = TRUE, intervals = 30)

I hope you found a way to solve this as well.
Thank you in advance for any help or input!

All the best,

Markos

get_chirts() does not allow fetching only one day

> dates <- "2010-12-15"
> lonlat <- data.frame(lon = c(-55.0281,-54.9857),
                      lat = c(-2.8094, -2.8756))

> temp1 <- get_chirts(lonlat, dates, var = "Tmax")

> Error in seq.int(0, to0 - from, by) : 'to' must be a finite number

Make it easier to load the test data outside of tests?

E.g. by putting it in tests/testthat/ and using testthat::test_path("test_data.rda") to find the right path (which works inside and outside of tests).

chirps stuck at the example.

I don't know if i am doing something wrong, but i am stuck at the example for the past 30 minutes.

Is there something that i can check?

Thanks.

Represent time series as spatial object using {stars}

From R. Bivand comments.

Check for a reasonable way to represent the times series using stars package. Also enable precip_indices to return a layered object that can be passed to a geographical analysis.

Documentation examples don't run

I've made a few corrections to the get_chirps() @examples section, but the S3 method for class sf point does not work and need to be fixed before resubmitting our response to the reviewers.

The object, tp_points is not defined but is called on line 72.

I replaced example() with data() on line 68.

Check data from ClimateSERV against CHC and prepare for new version

I new version should arrive in mid-2024 and I will check some questions about the discrepancies on the two data sources ClimateSERV vs CHC (no example code here for now). Also work in fixing the series of issues reported over the last year.

Tests fail

I was troubleshooting some of the tests when I found discrepancies in the precip indices.

This is from

chirps/tests/testthat/test-get_chirps.R

Line 13 in 7f60a8e

test_that("default method", {

> x
 [1]  0.00  0.00  0.00  0.00 20.18 38.35  0.00 38.35  0.00  0.00
> chirps$chirps
 [1]  9.26  0.00  0.00  0.00 18.51 38.35  0.00 38.35  0.00  0.00

I didn't go further as it's getting late here. Just flagging this to follow up later.

Prepare for submission to ropensci

The package is almost ready for submission to ropensci. This is a Milestone proposed to R. Bivand in his course on "Analysing Spatial Data". The package will be also submitted to JOSS. Here some things to do.

To do:

Write vignette (done!)
Write paper
Check consistency with ropensci templates
Check text and typos

Use CHIRPS CoG tiles

Hey,
@kauedesousa, @adamhsparks
Thanks to Pete Peterson, CHIRPS daily global tiles are now available as Cloud Optimized GeoTiffs (CoG) that can provide an alternative to ClimateSERV API and significantly improve the data access time, as well as support more generic request formats. Here is an example. Please feel free to integrate it with the current functions if you find it useful.

#' Read CoG tiles from ftp://ftp.chc.ucsb.edu/
#'
#' @param dates character, vector of start date and end date for processing interval, 
#'  in the format ("MM-DD-YYYY")
#' @param y spatial, see details
#' @param as.raster logical, If TRUE returns \code{\link[terra]{SpatRaster}} with \code{\link[terra]{crop}} by y, ignoring any other arguments
#' @param ... options, see \code{\link[terra]{extract}}
#' @return A SpatRaster object if \code{as.raster=TRUE}, else matrix, list, or data.frame 
#' @details
#' y any allowed format from \code{\link[terra]{cropr}} or \code{\link[terra]{extract}}
#' Available file description are provided in  
#' https://data.chc.ucsb.edu/products/CHIRPS-2.0/README-CHIRPS.txt
#' 
#' interval: supported intervals are daily, pentad, dekad, monthly, 2-monthly, 3-monthly, annual
#' currently hard coded to daily 
#' 
#' @examples
#' Case 1: return summary tables
#' dates <- c("2017-12-15","2017-12-31")
#' lonlat <- data.frame(lon = c(-55.0281,-54.9857), lat = c(-2.8094, -2.8756))
#' val <- get_chrips_out(dates, y = lonlat, fun = mean)
#' 
#' Case 2: return raster
#' f <- system.file("ex/lux.shp", package="terra")
#' v <- vect(f)
#' r <- get_chrips_out(dates, y = v, as.raster = TRUE)
#'                        
#' @noRd

.get_CHIRPS_tiles_CHC <- function(dates, resolution, ...){
  # setup file names
  seqdate <- seq.Date(as.Date(dates[1]), as.Date(dates[2]), by = "day")
  years <- format(seqdate, format="%Y")
  dates <- gsub("-","\\.",seqdate)
  fnames <- file.path(years, paste0("chirps-v2.0.", dates, ".cog"))
  
  # hardcoded now, but need to change in the future with more CoG availability
  coverage <- "global"
  interval <- "daily"
  format <- "cogs"
  
  # check for resolution
  if(!missing("resolution")){
    if(!resolution %in% c(.05, .25)){
      stop("resolution must be .05 deg or .25 deg")
    }
    } else {
      resolution <- .05
      print("no resolution provided, trying .05 deg")
    }
  
  resolution <- gsub("p0.", "p", paste0("p", resolution))
  u <- file.path("https://data.chc.ucsb.edu/products/CHIRPS-2.0", 
                 paste0(coverage, "_", interval), format, resolution, fnames)
  u1 <- file.path("/vsicurl", u)
  r <- terra::rast(u1)
  # time(r) <- seqdate
  return(r)
}

get_chrips_out <- function(dates, y, as.raster = FALSE, ...){
  # get CHIRPS CoG files
  rr <- .get_CHIRPS_tiles_CHC(dates, resolution = .05)
  
  if(as.raster){
    rout <- crop(rr, y)
    return(rout)
  } else {
    vals <- extract(rr, y, ...)
    vals$ID <- NULL
    return(vals)
  }
}

CHC data not available on specific date

Hello dear ropensci/chirps team!
I am facing an issue with a specific date when downloading CHIRPS data from CHC. The issue seems to be related to 2018-04-01.
Using the code presented below, I've been able to download data from 2010 up to march 2018. Also, I tested staring with 2018-04-02 on, and the data was retrieved. I also tried from ClimateSERV, with no sucess...

Do you have any suggestions or solution here?
Thank you in advance!

mt <-readOGR(file) \n
area <- st_bbox(mt)
area <- as.vector(area)
area <- terra::ext(area)

dates <- c("2018-04-01", "2018-04-10")
rain_data <- get_chirps(area, dates, server = "CHC")

Error: [rast] file does not exist: /vsicurl/https://data.chc.ucsb.edu/products/CHIRPS-2.0/global_daily/cogs/p05/2018/chirps-v2.0.2018.04.01.cog
In addition: Warning message:
`/vsicurl/https://data.chc.ucsb.edu/products/CHIRPS-2.0/global_daily/cogs/p05/2018/chirps-v2.0.2018.04.01.cog' not recognized as a supported file format. (GDAL error 4)

as.raster = TRUE doesn't return a raster object

lonlat <-
  structure(list(
    lon = c(-60.03947, -54.7838828),
    lat = c(-3.031387,
            -2.4221716)
  ),
  class = "data.frame",
  row.names = c(NA, -2L))
dates <- c("2000-01-01", "2000-01-05")
x_raster <-
  get_chirps(
    data.frame(lonlat),
    dates = dates,
    server = "ClimateSERV",
    as.raster = TRUE
  )

results in

x_raster
       id    lon   lat       date chirps
    <int>  <dbl> <dbl>     <date>  <dbl>
1:      1 -60.04 -3.03 2000-01-01   0.00
2:      1 -60.04 -3.03 2000-01-02   0.00
3:      1 -60.04 -3.03 2000-01-03   0.00
4:      1 -60.04 -3.03 2000-01-04   0.00
5:      1 -60.04 -3.03 2000-01-05  20.18
6:      2 -54.78 -2.42 2000-01-01  38.35
7:      2 -54.78 -2.42 2000-01-02   0.00
8:      2 -54.78 -2.42 2000-01-03  38.35
9:      2 -54.78 -2.42 2000-01-04   0.00
10:     2 -54.78 -2.42 2000-01-05   0.00

A data.frame object.

Use tryCatch() for downloads

It would be good to use a tryCatch() when sending the query and downloading files to give users a clear message about what's going on.

Chirps/UCSB changed its urls - get_chirps returns null for rainfall

In the last couple weeks, the url changed for chirps. No get_chirps does not return any values for rainfall. It returns -9999 . This is the new https url, as I understand it:
https://data.chc.ucsb.edu/products/CHIRPS-2.0/

get_chirps using a SpatRaster?

I'm working on writing tests to improve the test coverage for the package (and hopefully stop the automated e-mails that the tests failed).

Right now I'm working on get_chirps() for the SpatRaster method but I'm getting an error.

 r <- rast(system.file("ex/elev.tif", package = "terra"))
 x_raster <- get_chirps(r, dates, server = "CHC", as.raster = TRUE)

Results in an error.

Error in `[.data.frame`(lonlat, , 2) : undefined columns selected

Am I just using the function incorrectly?

get_chirts() fails with uninformative message

Trying to fetch CHIRTS data for years that are unavailable does not return an informative message.

Error in chirps:::.get_CHIRTS_tiles_CHC(dates, var, ...) :
  unique(years) %in% yrange is not TRUE

A combination of if() and stop() should be used to provide the user with an informative message.

Documentation error with R under development

##[warning]Non-file package-anchored link(s) in documentation object 'dataframe_to_geojson.Rd': ‘[sf]{st_buffer}’

It's because st_buffer() contains help for several functions, https://stackoverflow.com/questions/62420464/non-file-package-anchored-links-in-documentation-object.

I've attempted to fix this. I think I have it right. AgrDataSci@f7ee261

Interest in adding CHIRPS-GEFS &/or Prelim?

Hi, thanks for the nice package!

I was wondering if you had considered adding functionality to pull other chirps products like chirps-gefs and/or chirps preliminary? If there is interest I'd be happy to contribute - if not, no worries.

Thanks and Let me know

Move CI from Travis to Actions?

I was thinking to move the CI here to GitHub using Actions.

There are issues with installing some of the geo packages for the devel version of R with that though, but you can easily test Windows, macOS and Linux in one spot.

Are you OK with this, @kauedesousa?

Using get_chirps over a SFC, possible?

Hello everyone and thank you very much for all of your amazing work,

I am trying to pull the chirps data for Ethiopia and I am struggling to advance.

This is my first time using CHIRPS, but I am really interested in working myself into the subject. My goal is to get CHIRPS data for ETHIOPIA, I will want to finally de-trend the precipitation averages (either daily or monthly) in order to find abnormalities, in order to identify regional droughts as the basis for me research. Hence, I would ideally have both the precipitation on a pixel by pixel basis and will also be able to connect the pixel to the respective districts.

I am using: https://geodata.lib.berkeley.edu/catalog/stanford-kz352sx7513 : as base input of Ethiopian districts over which I want get the chirps data.

#Problem
I am reading in the ETHIPIA District Shapefile above as a SFC_Multipolygon, with the intention of pulling the CHIRPS data for each district over a given period of time. Am I correct in the assumption that the "get_chirps" command is currently only able to pull data for geographic points or is it somehow able to pull the CHIRPS data for entire districts?

I have been trying around a lot but can't figure which approach would work, if you guys have any insights or inputs I would be incredibly grateful. Even if the answer is an affirmation that what I want to do is not possible with the get_chirps command. (Would have to manually load in all the CHIRPS data images.)

The code (ETHIOPIA = The data input):

ETHIOPIA_shp <- st_read(ETHIOPIA)

ETHIOPIAREP_shp <-st_transform(ETHIOPIA_shp, CRS(OGP))

st_write(
ETHIOPIAREP_shp,
ETHIOPIAREP,
format="GTiff", append=FALSE)

SPDF <- readOGR(dsn=ETHIOPIAREPP,layer="ETHIOPIAREP")

dates <- c("2017-12-15", "2017-12-31")

get_chirps(SPDF, dates, operation = 5)

yields the error :

Fehler in [.data.frame(lonlat, , 2) : undefined columns selected

and if I use:

get_chirps(ETHIOPIAREP_shp, dates, operation = 5)

in the last line, it runs forever without end. This is where I think it might not be possible to run it for multipolygons.

I hope this question / issue is not to far away from the intended use of get_chirps and that I am not wasting your time.

Thank you in advance for any help or input!

All the best,
Marius

data(1).zip

outputs as data.table

get_chirps and precip_indices may return large data sets that can cost some memory. Check if is wise and acceptable by users to return a data.table object instead of a tbl_df.

S3 method to concatenate geojson objects

The functions data.frame_to_geojson() and sf_to_geojson() could be simplified with as.geojson(). This may provide a clean documentation and easy debugging pipeline.