csm-applications / csm-crowdslr Goto Github PK

O coeficiente kappa de Cohen ( κ ) é uma estatística usada para medir a confiabilidade interexaminador (e também a confiabilidade intraexaminador ) para itens qualitativos (categóricos). Essa medida é considerada uma medida mais robusta do que o cálculo de concordância percentual simples, visto que k leva em consideração a possibilidade de a concordância ocorrer por acaso. Apesar disso, esse coeficiente é bastante aceito[1]

Assim como sugerido anteriormente por revisores externos seria interessante implementar esse coeficiente para medir a concordância entre os crowdworkers.

In Systematic Literature Reviews primary studies selection is usually divided into smaller steps. It happens because many irrelevant studies returns on Electronic Database search, therefore, when you divide the studies selection evaluating only title, abstract and keywords it is possible to eliminate studies that clearly are not relevant for your review. After the first step, we can perform a full-text analysis and evaluate their relevance for our SLR.

For now, CrowdSLR is not capable of creating an interface for full text selection i.e., only the title and abstract are inserted in Json for evaluation. Therefore, it is necessary to improve this project by allowing crowdsource access and select studies based on full text.

For now, password security is hardcoded and no encrypting technique is used. Therefore, while exposing our tool for many anonymous workers there is a risk of cyber atacks.

It is necessary to improve security techniques and block sensitive actions to avoid problems in research