dhis2 / who-data-quality-app

Data Quality App by WHO

WHO Data Quality Tool

License

© Copyright 2017 the World Health Organization (WHO). This software is distributed under the terms of the GNU General Public License version 3 (GPL Version 3), copied verbatim in the file “COPYING”. In applying this license, WHO does not waive any of the privileges and immunities enjoyed by WHO under national or international law or submit to any national court jurisdiction.

Getting started

Install dependencies

To install app dependencies:

npm install

Compile to zip

To compile the app to a .zip file:

npm run zip

Start dev server

To start the webpack development server:

npm start

By default, webpack will start on port 8081, and assumes DHIS2 is running on http://localhost:8080/dhis with admin:district as the user and password.

A different DHIS2 instance can be used to develop against by setting the DHIS2_HOME environmental variable (e.g. export DHIS2_HOME=/Users/Olav/dhis2/dq_d2home). Inside the folder DHIS2_HOME points to, create a file config with the url and base64-encoded username:password like this:

exports.baseUrl = 'http://localhost:8090/dqdev';
exports.authorization = 'Basic YWRtaW46ZGlzdHJpY3QK='; //Based64 encoded

who-data-quality-app's Issues

How to use the tool to review Domain 3 and Domain 4

I have yet to see a country DHIS2 instance which has the data/data elements/indicators required to assess Domain 3 (consistency of routine estimates with survey estimates) and Domain 4 (External consistency of live birth estimates). The user manual should include some brief instructions on how to configure and populate the required indicators for this.

Consistency - detailed chart should use shortest relevant periods

The detail charts for one particular orgunits in the consistency module should show "shortest possible" period for a variable, not selected period only. E.g. if doing an analysis by year of a monthly data element, the detailed chart should show the months rather than years.

Improved method for identifying outliers

Identifying outliers:

A more sophisticated method to identify outliers should be developed. Preferably the HB method (demonstrated at the DHIS 2 academy in August)

A graph in addition to tables to identify outliers

Review handling of aggregationtype in d2Data

Review and improve handling of aggregationtype in the internal d2Data service.

Consistency analysis module improvements

1. Change default on 'criteria' from 33% to 0% or remove it altogether.

2. We have not been able to identify the difference between "Totals" and "Details". If there is no difference, the option should be removed.

3. 'District' as default on 'Disaggregation' under 'Orgunit'.

4. Add the opportunity to compare a specific month (eg April 2017) with the previous releases of April (April 2016, April 2015, April 2014, ..., April N) during 'Period'.

5. Some variables should not be aggregated on month to get the year. Eg. doctors, beds, etc. Is it possible to enter a functionality where you can get an average so that the numbers do not become x12? The error may be more in the main program?

6. Remove "dropout" / redefine "dropout" unless this is a known term in DHIS2. As we see it, "dropout" should refer to whether unique devices / people drop out of or continue a program (panel data)

Link to data set report

Ability to select a data value and open the corresponding data set report. Only works with orgunit "leaves".

Admin - improve layout of quality parameter table

Improve the layout of the quality parameter table in the admin module, to use two rows.

Dropout rate visualisation - show name of orgunit on hover

When hovering over dropout rate visualisations, show the name of the orgunit for each bar.

Outlier custom date range period type

When selecting custom date ranges for the analysis checks, monthly period type could be used as the default (with option of changing to dates/weeks when that is needed).

Outlier - option to ignore outliers

Scatterplots - show % of overall value for each value

In the scatterplots, show the percentage each particular value represents.

Problems with aggregation of reporting completeness

With several country instances (e.g. Liberia) when reporting completeness is aggregated over multiple years (e.g. for the "Overall score" in section 1d of the Annual Report), the result is substantially greater than 100%. My guess is that DHIS2 administrators need guidance on how to define the aggregation for reporting completeness.

Quarterly data on dashboard need more than 1 year/4 quarters of data

Quarterly data on dashboard have limited value since only 4 periods are shown, should extended to e.g. 3 years/4 quarters.

Favourites for outliers and consistency modules

Analysis modules - disable "Analyse" button while data is being fetched

Outliers - dynamic thresholds by facility size

"Dynamic" threshold for outliers according to the averagely reported value, similar to DHIS 1.4

Smaller deviations allowed for bigger facilities, and the smallest facilities potentially ignored.

Would have a similar effect to the HB-method proposed by SSB.

d2select indicator/data element - ability to restore groups

When loading a data element or indicator programatically to the d2select components, a group should be auto-selected as well.

Outlier - review escaping of commas in names for CSV downloads

Report/module for doing all checks for *one* indicator

A report or module for reviewing all DQ metrics related to one indicator (rather than by domain (dashboard) or group (annual report))

Histogram of reported values

For a particular data element (operand), make a histogram with the typical reported values.

Outlier - show completeness in visualisation

When showing the visualisation for an orgunits and variable in the outlier module, it would be useful to include the trend in completeness as well.

Standard reports - customisable reports combining various outputs

Functionality for combining various DQ metrics/outputs in reports, which can be accessed similarly to the annual report.

Completeness module

Add in-depth completeness module. Could include the completeness-related outputs currently on the dashboard and in the annual report, but also additional functions.

Examples:

• analysis of the completeness of each variable by datasets
• empty but complete dataset, and datasets that are filled by not complete

Permit the administrator to select which sub-sections appear in the annual report

As it is, the report is too lengthy and users will find it hard to focus on the most important findings. As one example, I find the entire "Summary" section to be unhelpful. I like to skip it and jump to the detailed findings section.

Dashboard improvements

Consistency - time tab
Better if the graph on the right has default level on district and not region. It would also be an advantage if the "guides lines" that distinguish between percentage deviations in scatter plots were removed. They give too much attention to small units.

Consistency - data tab

Is it correct to use the term dropout? It's not referring to same people (or is this a common way to use the term in DHIS2?)

Use dynamic graphs for "Consistency data"; In particular, histograms show little information when the y-axis is kept at 100% maximum at all times. By adjusting the maximum level on the y-axis of eg 5% over the highest observed percentage, the graph will be more intuitive. In that case, it is also important to point out that the y-axis is not 100%, but at X%.

Outliers tab

Consider dropping this tab. The outlier part does not work optimally, and much of the information here is better presented in the «consitency-time» tab. A good replacement can be to enter a "tables" tab. Where there is a table with the most important variables. Where the line starts at national level and where you can drill down.

Dashboard performance

Dashboard can be slow to load, in particular the time consistency. To improved the (perceived) waiting time, fetch the quickest chart types first and display results as they are returned.

Outlier - add summary statistics for outlier query

Add a summary result when doing a DQ analysis. Could show e.g. distribution of outliers by

• orgunit
• period
• indicator
  depending on the parameters.

Admin - move descriptions/info to column headers

Move descriptions/info on the meaning of different columns to the column headers, as is done in the outlier results. Applies to the numerator parameter settings for example.

Improvements for completeness estimation (Dashboard)

Dashbord 'Completeness' should display information for three years back, similar to the 'Consistency-time' tab.

Support modification of numerator mapping

After a particular Numerator has already been mapped, if you want to revise the mapping, you click on "edit".

The bottom half of the resulting dialogue box (Data mapping, Dataset for completeness, Variable for Completeness) is empty. It would be nice if the bottom half of this dialogue box showed the existing mapping so that you did not have to again search for each of the proper settings. This way it would be easier to re-set just one of 3 things being mapped or to switch the numerator from non-core to core.

Annual report - Denominator for «% divergent orgunits»

In the annual report, the value to use for denominator when calculation % of orgunits with divergent score should be review. Currently only orgunits with at least on non-null completeness is included in the denominator (under the assumption that only those report on the given reporting form). This should probably be changed to include all orgunits.

Fiscal year support in annual report

This would be relevant for several countries including Liberia and Ethiopia

Consistency over time - more options for choosing reference periods

Option to choose both consecutive periods, and same periods previous year (and a combination). Could present option as button-groups, one line per year, where any periods can be used for reference.

d2Data - review handling of aggregationtype

Review the current handling of aggregationType in the d2data service.

Internationalize app

Currently, only English is supported. Extend support for other languages using standard DHIS2 i18n frameworks.

Consistency - show the number and % or orgunits outside the threshold

Add info on the number and percentage of orgunits that are outside the given DQ threshold

How to use the Completeness Dashboard to drill down and explore trends in completeness

This is a place holder.

I am experimenting with methods for drilling down and reviewing facility-level trends in reporting completeness.

External consistency - allow comparisons of raw data

Add support in the external comparison to compare "raw" data e.g. from parallel systems, in addition to survey results.

2.28 compatibility

Upgrade to api version 2.26 and test with 2.28

Consistency - make result responsive

Disable orgunit level/group selection when selected orgunit is leaf

Disable orgunit level/group selection when selected orgunit is a leaf.

Consistency - option to open pre-defined relations

Admin - improve filter on indicators in numerator mapping

If mapping a numerator to an indicator, only indicators with denominator = 1 should be options.

Calendar year extends from December to November

With the Analysis tools and the Data Export for Excel tool, when I select by year and 2016 I get data for the months from December 2015 to November 2016.

Sort icon should change depending or order

Currently, the sort icon does not indicate the sort order (a-z or z-a), this might be useful.

Version number in app

The version number of the app should be logged or displayed for example on the "About" page. This would make troubleshooting easier.

Only show data elements where the data element type can be aggregated

Only show data elements where the data element type is one which can be aggregated. For example, leave out text and date type data elements. Applies both to the in-depth analysis modules and the numerator/denominator mapping.

Further explanation required for setting reference lines

When using Administration to configure the Tool:
a) Under "Numerator relations", with "Type" it is not self-evident what is meant by "Equal across orgunits" as opposed to "A = B". With the Liberia case, "Equal across orgunits" appears to place the reference line correctly (with a slope equal to the nationwide ratio) whereas "A=B" places the reference line at A=B. The user manual +/- the webpage itself should explain this.
b) Under "Numerator quality parameters", with "Compare orgunit consistency with" it is not self-evident what is meant by "Overall result" versus "Expected result". With the Liberia case, "Overall result" appears to correctly place the reference line. The user manual +/- the webpage itself should elaborate on this.
c) Under "Numerator quality parameters", what difference does it make whether "Expected trend" is set to "Increasing" versus "Decreasing" versus "Constant"? How can the graph in the bottom left of sections 2d be interpreted to decide whether the Expected trend is Increasing versus Decreasing versus Constant?
Expected trend.docx

Consistency - indicator consistency option to highlight those relevant for single periods

Certain indicator consistency checks might be relevant when looking at a single period (e.g. month), for example reported stock vs services. Others require a longer time period, such as dropout rates. If it was possible to specify this, the dashboard could show data for single months/quarters for these indicators.

Dashboard - improve year over year chart labels

• when hovering, show the appropriate year only
• show orgunit
• review if the series names can be made clearer