pepfar-datim / datimutils Goto Github PK

https://github.com/pepfar-datim/data-pack-commons/blob/546ade9e7af04ec305e6132455ed9ed626b2b8b6/data-raw/SnuxImDistMain.R#L57

non_mil_types_of_org_units <- 
      datapackcommons::getMetadata("dimensions", 
                                   "id:eq:mINJi7rR1a6", 
                                   "items[name,id]") %>% 
      tidyr::unnest(c("items")) %>% 
      dplyr::filter(name != "Military") %>% 
      .[["id"]]

https://github.com/pepfar-datim/data-pack-commons/blob/546ade9e7af04ec305e6132455ed9ed626b2b8b6/data-raw/SnuxImDistMain.R#L86

mech_codes <- datapackcommons::getMetadata("categories",
                                            "id:eq:SH885jaRe0o",
                                            "categoryOptions[id,code]") %>% 
    .[["categoryOptions"]] %>% 
    .[[1]] %>% 
    dplyr::rename(mechanism_co_uid = "id", mechanism_code = "code")

Testing

Show that the old code and new code produce equivalent output.

Description

@jason-p-pickering @sam-bao - Please add any suggested functions to include in this package (what it should do, not a name suggestion) for its initial launch:

• login to DATIM using username and pw
• export dimension data/metadata (e.g., categoryOptions, dataElements, organisationUnits) from endpoint, and allow easy, magrittr-friendly filtering and field selection.
• Pull a SQL view
• perform basic query against Analytics API
• Export list of sites for supplied country id, specifying between facility and community
• Export list of mechanisms for supplied country id
• Export list of PSNUs (optional inclusion of _Military here), or any SNU level for supplied country id
• Export list of form-valid dataElements, categoryOptionCombos, and categoryOptions (classified by category) for supplied dataset and fiscal year and/or quarter. (HAS to link to the code list SQL View to make sure this list represents what is ACTUALLY valid in forms for a given dataset and year.)

Acceptance criteria and must have scope

The goal is to accumulate upfront as many ideas for potential functions as are possible, and then to later convert these into epics or issues in our backlog.

Stakeholders

Would be good to run this list past ICPI users as they seem to be the largest potential user base. These functions should also allow offload of any generic DATIM API interactions from datapackr and datimvalidation.

Timeline

Goal is to complete first pass of this list by Aug 30, 2019.

The datapackr package and apps make a few calls to necessary sqlViews in DATIM. We have implemented a few different approaches to getting this, but it would make sense to centralize how we retrieve them from the server.

Based on the DHIS2 developer documentation, the prototype function should look something like this

getSQLView<-function(id,criteria,vars,filters) {...}

id: The UID of the sqlView

criteria: A simple server side filter which can be specified. Should accept a list and collapse it.
/api/sqlViews/{id}/data?criteria=col1:value1&criteria=col2:value2

vars: Should accept a named list or a data frame consisting of key-value pairs
/api/sqlViews/{id}/data?var=key1:value1&var=key2:value2

filters: : Should function the same as getMetadata, allowing for multiple filters and operators, or alternatively, a user constructed filter string.
/api/sqlViews/w3UxFykyHFy/data.json?filter=orgunit_level:eq:2&filter=orgunit_name:ilike:bo

Retrieval of sqlViews appears to be quicker and easier to implement using the CSV response.

glue::glue("{d2_session$base_url}api/sqlViews/IMg2pQJRHCr/data.csv") %>% 
     httr::GET(., handle = d2_session$handle )  %>% 
     httr::content(.,"text") %>% 
     readr::read_csv(.)

The JSON response could also be used and does contain a bit metadata information, and could also be considered to be used.

FYI @sam-bao

@maxwellchandler

This line still doesn't seem to be hit by out unit tests.

Hi,
I have installed both datimvalidation and datimutils R packages for pepfar data validation purposes. When I try to use the logintodatim function, however, I get the error Error: lexical error: invalid char in json text. <!DOCTYPE HTML> <html class="lo (right here) ------^

This is how I used it: loginToDATIM(base_url = "https://dev-de.datim.org/dhis-web-commons/security/login.action", username = "xxx", password = "xxxxxxx"). The base url from the manual says I should use http://dev-de.datim.org which gives an error Error in curl::curl_fetch_memory(url, handle = handle) : Could not resolve host: dev-de.datim.orgapi

With this error, I changed to the resolved URL which is https://dev-de.datim.org/dhis-web-commons/security/login.action but then faces the lexical error explained. I am stuck and cannot make any progress; any help would be appreciated. Just to add, I have an existing account in Datim.

Augustine

1. Create a method like getDataStore(namespace,key) which would return a datastore object.
2. Create a method to update datastore objects (PATCH)
3. Create a method to replace a datastore object (PUT)
4. Create a a method to delete a datastore object (DELETE)