unhcr / koboloader Goto Github PK

The indicator calculation sheet in an important element of reproducibility for the analysis.

However this worksheet also includes some elements of R code that might not be easy to implement for everyone -

We should probably have a shiny in to facilitate the creation of indicators

Like #6 , based on an additional column to be documented in the xlsform or indicator sheet, it shoudl be possible to add some predictive model capacity.

The target to predict could be defined and then algorithm will run to find out the best predictor and algorithm...

Double check that all feature form the equivalent package for SPSS are implemented
https://github.com/PovertyAction/odkmeta/issues

Output of variable in date format doesnt display

"Date question in data frame: MainDataFrame"

Context

In the crunchin result, all questions related to date doesnt display

Problem

No display of the results

Expected result

describe what you would expect to have resulted from this process

Current result

No displyed result

# insert reprex here

Possible Fix

not obligatory, but suggest fixes or reasons for the bug

name of issue screenshot

if relevant, include a screenshot

Shoudl open the xlsform and insert missing column used by kobo_dico: https://github.com/unhcr/koboloadeR/blob/master/R/kobo_dico.R#l42:L82

Please briefly describe your context, your problem and what output you expect.

Please include if possible a minimal reproducible example (AKA a reprex). If you've never heard of a reprex before, start by reading https://www.tidyverse.org/help/#reprex.

Context

Want to start the shiny app from Rstudio (opensSuse tumbleweed, Rstudio Version 1.2.5033 R version 3.6.3 (2020-02-29)

Problem

follow the instruction to create a new kobo project unsing https://unhcr.github.io/koboloadeR/ Project walkthrough

Expected result

the koboloadeR shiny app opens

Current result

Error message that several shiny functions not found. Does not open the app.

#  reprex
library (koboloadeR)
kobo_projectinit() 
kobo_shiny("app_main_koboloadeR.R")

Possible Fix

library(shinydashboard)
library(shinyalert)
library(rhandsontable)

• expected behaviour: these packages should be loaded automatically

In a commit to "fix issues for CRAN Registration", in the function get_me located in utils.R, a GET function call was inadvertently changed to get; GET is from the httr package and is used for downloading URLs, whereas get is a base function with a completely different purpose, namely to get a variable from a specified environment. A mishap when applying code style settings?

Context

I tried using both kobo_datasets and kobo_data_downloader with the current version. They worked in an older version I downloaded from mrdwab/koboloadeR.

Problem

Error message.

Expected result

Downloading of the data/listing of the datasets, which worked in version 0.1.2.

Current result

invalid 'envir' argument

Possible Fix

In utils.R, in function get_me, change get to httr::GET.

See here: http://r-pkgs.had.co.nz/vignettes.html

My plan is to use the koboloadeR package to download datasets stored on UNHCR's KoBo server. I managed without any problems when using KoBo Toolbox and HumanitarianReponse servers. However, I could not get to access any data stored on UNHCR's KoBo server.

I ran the following code unsuccessfully:
kobo_datasets(user = "username:password", api = "unhcr")

I also tried with "https://kobo.unhcr.org/api/v1/data" and "https://kobocat.unhcr.org/api/v1/data" as API arguments. None of these options worked. What is the correct API argument for the UNHCR server?

Once issue #3 is fixed, it should be possible to convert the 3 scripts below into functions directly within the package:

https://github.com/unhcr/koboloadeR/blob/master/inst/script/1-loaddata.R
https://github.com/unhcr/koboloadeR/blob/master/inst/script/3-create-indicators.R
https://github.com/unhcr/koboloadeR/blob/master/inst/script/4-generate-report.R

Please briefly describe your context, your problem and what output you expect.

Please include if possible a minimal reproducible example (AKA a reprex). If you've never heard of a reprex before, start by reading https://www.tidyverse.org/help/#reprex.

Context

Installing koboloadeR from RStudio Cloud (R version 4.2.0)

Problem

Ran remotes::install_github("unhcr/koboloadeR", Ncpus=4) from project walkthrough

Expected result

successful koboloadeR installation

Current result

package 'dummies' reported as not available

remotes::install_github("unhcr/koboloadeR", Ncpus = 4)
#> Downloading GitHub repo unhcr/koboloadeR@HEAD
#> Skipping 1 packages not available: dummies
#> * checking for file ‘/tmp/RtmpZpDcOa/remotes18a447f62e95/unhcr-koboloadeR-93f0d63/DESCRIPTION’ ... OK
#> * preparing ‘koboloadeR’:
#> * checking DESCRIPTION meta-information ... OK
#> * checking for LF line-endings in source and make files and shell scripts
#> * checking for empty or unneeded directories
#> Omitted ‘LazyData’ from DESCRIPTION
#> * building ‘koboloadeR_0.1.9.tar.gz’
#> Warning in sprintf(gettext(fmt, domain = domain, trim = trim), ...) :
#>   one argument not used by format 'invalid uid value replaced by that for user 'nobody''
#> Warning: invalid uid value replaced by that for user 'nobody'
#> Installing package into '/cloud/lib/x86_64-pc-linux-gnu-library/4.2'
#> (as 'lib' is unspecified)
#> Warning in i.p(...): installation of package '/tmp/RtmpZpDcOa/file18a439061160/
#> koboloadeR_0.1.9.tar.gz' had non-zero exit status

Possible Fix

not obligatory, but suggest fixes or reasons for the bug

Shiny App window disappear when you click "Build Data Dictionary"

If indicators is not classified as sensitive, the value should be pushed to an on line repository (CKAN based) such as http://data.humdata.org/

This probably requires to:

1. tag the variable according to HXL: http://hxlstandard.org/
2. upload it through the API: Replicating or embedding the python work here https://github.com/OCHA-DAP/hdx-python-api

This is a major refactoring of the package - but would be likely a lot more user-friendly...

This would imply to specify them under the variable column within the xlsform...

Potential consequences include:

• specific visualization - cf Likert package: https://github.com/jbryer/likert

• Specific type of correlation test to be performed - in relation with ticket: #19
  

I am using in Windows and
install_github("unhcr/koboloadeR", ref = "gh-pages")
fails due to ..

it searches rJava in users/USER/Documents/R/win-library/3.5/rJava/libs/x64/rJava.dll
path back slash (/)
But the file exists in C:\Users\USER\Documents\R\win-library\3.5\rJava\libs\x64
(front slash )

DDI: https://www.ddialliance.org/ is the standard for survey meta-data and is used in survey catalog such as NADA: http://www.ihsn.org/software/nada

At the end of the data crunching process, we should have a function to generate a DDI file for the survey.

see some reference here: https://cran.r-project.org/web/packages/DDIwR/DDIwR.pdf
https://www.user2017.brussels/uploads/useR2017-brussel-nakano.pdf --

Error "missing values are not allowed in subscripted assignments of data frames"
Instance ID and Cluster ID unable to move to next screen

Started building some raw map output from data + dictionary:
https://github.com/unhcr/koboloadeR/blob/master/R/kobo_map_cat.R
https://github.com/unhcr/koboloadeR/blob/master/R/kobo_map_int.R

This should be included in the report generation scripts

when I try to install with this "remotes::install_github("unhcr/koboloadeR", Ncpus=4) "
I receive this message "ERROR: dependency 'simFrame' is not available for package 'koboloadeR'

But when I go on the web page "cran.r-project.org/web/packages/simFrame" it tell me:
Package ‘simFrame’ was removed from the CRAN repository.
Formerly available versions can be obtained from the archive.
Archived on 2021-07-31 as check problems were not corrected in time.
A summary of the most recent check results can be obtained from the check results archive.
Please use the canonical form https://CRAN.R-project.org/package=simFrame to link to this page.

Is it necessary this dependency?

Thank You
Pascal

Please briefly describe your context, your problem and what output you expect.

Please include if possible a minimal reproducible example (AKA a reprex). If you've never heard of a reprex before, start by reading https://www.tidyverse.org/help/#reprex.

Context

I am using kobotoolbox to collect household level data on a daily basis to inform a Community based Surveillance System. If a household member gets sick, his/her symptoms are entered using a "Repeat"-type module. Similarly, if a household member gives birth, data related to this birth is entered using a "Repeat"-type module.

Problem

I use the kobo toolbox online interface to download the hierarchical data in csv files per module (as in https://unhcr.github.io/koboloadeR/docs/articles/Getting_data.html#using-the-online-interface-1). I create instanceID variable in all csv files. When I use the kobo_load_data function it never loads any of the "repeat"-type modules.

Current result

The log says :

loading Birth_report file ..
[1] "There was an error in the data processing step!!! \n\n"
$message
[1] "unused argument (stringsAsFactors = F)"

$call
readr::read_csv(paste0(mainDir, "/data-raw/", configInfoOrigin[configInfoOrigin$name ==
dbr, "path"]), stringsAsFactors = F)

attr(,"class")
[1] "try-error"

Possible Fix

I think maybe it has to do with the fact that for some households there are not any sick members or births to report, and maybe that is causing problems when loading the data, but I cannot find the reason nor how to fix it. Thanks

Need to add error bar based on sample definition when sampling is used...
http://www.sthda.com/english/wiki/ggplot2-error-bars-quick-start-guide-r-software-and-data-visualization

A project is managing multiple files:

• data - one of multiple frame
• cleaning log
• weight
• form
• indicators

Now this is managed through a series of frame / stored as csv

We could use the same approach then sdcMicro by creating a specifc class object with all those elements stored in a structured way...
https://github.com/sdcTools/sdcMicro/blob/master/R/0classes.r

It would be great that when label are long that they are not cut automatically, this needs some time to redo the labelling each time

Context

provide more detailed introduction to the issue itself and why it is relevant

Problem

ordered list the process to finding and recreating the issue, example below

Expected result

The label is not cut wherever the lenght is.

Current result

describe what you you currently experience from this process, and thereby explain the bug

# insert reprex here

Possible Fix

not obligatory, but suggest fixes or reasons for the bug

name of issue screenshot

if relevant, include a screenshot

For text questions, need to add more capacity see http://www.sthda.com/english/wiki/text-mining-and-word-cloud-fundamentals-in-r-5-simple-steps-you-should-know

See here: http://r-pkgs.had.co.nz/tests.html

It should be possible to also define within the xlsform -- or indicator calculation sheet - the variables that could be used together to build some clustering

The output would be part of the final Rmd and could allow to compare different clustering algorithm (hierarchical clustering, k-means clustering, latent-class modeling...)

Map variables to a structural equation model:

1. risk (in relation with basing needs)
2. coping (in relation with access to services and livelihood)
3. resilience (in relation with community inclusion and access to right)

CFA will allow to select the right indicators from the dataset to fit the model and path modeling would then allow to get the weight to be used for each indicators in order to build a scoring formula for each those 3 latent variables.
proposed approach is integrate Lavaan package within the workflow

For some Multiple choice questions the output doesnt show up,

Multiple choice question
No responses or the same answer was given (only one modality) recorded for this question.

Context

Crunching data from Kobo, no specific error.

Problem

Multiple choice question
No responses or the same answer was given (only one modality) recorded for this question.

Expected result

Display of the multiple choice result

Current result

For some Multiple choice questions the output doesnt show up,

# insert reprex here

Possible Fix

not obligatory, but suggest fixes or reasons for the bug

name of issue screenshot

if relevant, include a screenshot

Ability to take out at the beginning of the RMD and output the section on Crucnhing step, and Dataset description

Context

provide more detailed introduction to the issue itself and why it is relevant

Problem

ordered list the process to finding and recreating the issue, example below

Expected result

Be able to choose at the beginning if needed to include Crucnhing step, and Dataset description

Current result

describe what you you currently experience from this process, and thereby explain the bug

  koboloadeR::kobo_crunching_report(form,
                        output ="docx", ## other options are  "docx", "pptx", "aspx"
                        render = "TRUE", ## can put to false in case you just need to refresh the Rmd based on your xlsform config
                        lang = "eng", ## other options is to have report language in Spanish "esp" or french "fre"
                        unhcRstyle = "TRUE", ## put false to have a non-UNHCR neutral theme
                        use_pct = "TRUE", ## use count rather than % in charts label
                        add_error_bar = "TRUE"  ## put false to supress error bar
                        )

Possible Fix

App a paremeter in this function to be able to include or not those sections.

name of issue screenshot

if relevant, include a screenshot

Different cases to consider:

1. Discretize a value (using cut)
2. Re categorize a categorical variable by re coding modalities (using fct_recode)
3. Sum up different numeric or integer variables
4. Calculate min, max or avg value for multiple integer or numeric variables
5. Set condition on specific variables (using ifelse)
6. Calculate ratio by dividing 2 numeric or integer variables
7. aggregate variables from nested frame (inc ase of hierarchical data set using dcast)

Indicators creation can be chained.
The interface to be developed in shiny for non expert users - allowing to build the corresponding R code and to place the newly created indicator within the same analysis plan that the collected variables.

Adding cleaning log capacity to edit data frames directly in the app, as well as record the changes.
Could also include excluding questions from the analysis: for example, ID numbers, phones numbers, etc. The histograms generation can crash the shiny app.

Hi. thank for this great work. There is a plan to put koboloadeR in CRAN?
I want to use it in shiny app but i don't know if it will work after deploy.

Currently the data crunching report only deals with Chisquare test between select_one variable;
https://github.com/unhcr/koboloadeR/blob/gh-pages/inst/script/3-generate-report.R#L500:L625

More correlation test could be handled...

Ideally the configuration of the project should be entirely done in https://github.com/unhcr/koboloadeR/blob/master/inst/script/0-config.R

When the survey includes repeat element and nested data frame, the links between frames needs to be configured in order to map properly the frame with dictionary generated from kobo_dico: https://github.com/unhcr/koboloadeR/blob/master/R/kobo_dico.R

As the nested frame naming follows some logic inherited from the regular export from ODK briefcase, it should be possible to extract the required configuration

This not yet implemented in the config file -- and the various nested frame still needs to be configures manually throughout the workflow (load data - calculate indicator - generate report).

tidyverse is not recommended as a package dependency; better to load directly the few components we need like ggplot, dplyr. Should speed up installation by a lot. Also I think CRAN really doesn't like it

See some brainstorming on the expected scope for the function:
https://github.com/unhcr/koboloadeR/blob/master/R/kobo_anonymise.R