jeffcnz / hillr

An R Package for interacting with a Hilltop time-series server

hillr

A R package for interacting with a Hilltop time series server.
More information on using Hilltop Servers can be found here This package implements many of the features available on a Hilltop server, but not all options are available.

Features

Usage

List of Sites

To get a list of sites available from the endpoint

sites <- getHilltopSites(endpoint = "https://data.hbrc.govt.nz/Envirodata/EMARDiscrete.hts?")

Measurements At A Site

To get a list of measurements at a site

measurements <- getHilltopMeasurements(endpoint = "https://data.hbrc.govt.nz/Envirodata/EMARDiscrete.hts?",
                                                   site = "Maraetotara Lagoon")

Data

To get some data for a site you'll need to know the Hilltop server endpoint, site name, and measurement. The getHilltopData function does basic requests, but to request multiple sites and or measurements, and to get additional metadata you'll need to use the fullGetHilltopData function.

Generally the fullGetHilltopData function is the preferred option.

Discrete (Sample Data)

eColi <- getHilltopData(endpoint = "https://data.hbrc.govt.nz/Envirodata/EMARDiscrete.hts?",
                       site = "Maraetotara Lagoon",
                       measurement = "E. Coli",
                       from = "1/1/2018",
                       to = "1/10/2018")

To get all the metadata for a measurement, or to get data from multiple sites and measurements

data <- fullGetHilltopData(endpoint = "https://data.hbrc.govt.nz/Envirodata/EMARDiscrete.hts?",
                           sites = c("Maraetotara Lagoon", "Ngaruroro River at Fernhill"),
                           measurements = c("E. Coli", "Nitrate Nitrogen"),
                           from = "1/1/2017",
                           to = "1/10/2017",
                           option = "WQ")

Requirements

Installation

In R make sure that you have devtools installed. Remove any preinstalled versions of hillr, then run

devtools::install_github("jeffcnz/hillr")

Authors

• Jeff Cooke

License

This project is licensed under the MIT License

hillr's Issues

Connection timed out

I was testing the hillr package to download available discharge observations during cyclone Gabrielle and the R package returns a connection timeout when running the list of sites command from the README:

sites <- getHilltopSites(endpoint = "https://data.hbrc.govt.nz/Envirodata/EMARDiscrete.hts?")

does the server need a registration or something?

Gap Tolerance not able to be set in request

For continuous data where there is a gap marked in the data the aggregation may not get done. Hilltop Server can accept a GapTolerance interval to allow it to provide data if the gap is less than the specified interval.
Implementing this functionality would be useful.

Date for summary stats can be confusing

The date returned when an interval is specified and a statistic is the end of the interval, so monthly totals would be timestamped as the first of the following month, daily totals would be timestamped as the following day (the result is for the preceeding interval).
The server can return data with just a date, rather than a date and a time and this date is the last day in the interval, ie more intuitive.
Would be good to implement this functionality.

fullGetHilltopData() Error and not returning a dataframe when given invalid site name

When using fullGetHilltopData(), providing the sites argument with a string vector containing a string that is not a valid request-able site, will result in an error and no data frame returned. Even if you have other valid sites in the sites vector.

`
library(hillr)

wSites<- c("Waiiti Stream at Waihau Road","Waihau Stream at Waihau Road","Mohaka River D/S Ripia River","TEST1")
wMeasurements <- c("Total Nitrogen", "Turbidity (Field)","Total Phosphorus")

webService <- "http://data.hbrc.govt.nz/Envirodata/EMARtest.hts?"

requestData <- fullGetHilltopData(endpoint = webService, sites = wSites, measurements = wMeasurements, option = "WQ")
`

In the above code, "TEST1" is not a valid site to request.

Producing:

"Error in subset.default(wqSampDf, select = -c(Site, Measurement, Units)) :
argument "subset" is missing, with no default"

Doesn't return actual data when there are duplicate entries from the server.

When there are duplicate attributes on a measurement the dataframe returned contains the count of the elements, not the attribute values.