avniproject / avni-reporting Goto Github PK

This is a spike card where we need to look at problems with our Jasper Reports solution.

• The reporting solution must mention Avni instead of Jasper Reports, so that there is more trust in the application. What level of whitelabeling is possible?
• Current reports and graphs look ugly. How easy is it to style templates to reports that can make it look better?
• Can we show goal lines on the graphs?
• Currently, every implementation has multiple folders show up. How can we change this and show just the list of reports (not the data source etc)
• Is it possible to create a dashboard?
• Is there an ability to send notifications from Jasper Reports?

https://avni.freshdesk.com/a/tickets/763

Error -

https://docs.google.com/document/d/1Xl6AFNMsmn9Jom5NQcbA1zJDSdMfP2dBBmTTJrDIupg/edit#heading=h.b1slaempagvf

Reports not working -

• https://reporting.avniproject.org/question/641-prevalence-of-substance-misuse-save-with-line-listing?year=2019 -View was missing for this report

• https://reporting.avniproject.org/question/625-prevalence-of-anemia-with-line-listing?year=2018 -View was missing for this report

• https://reporting.avniproject.org/question/627-menstrual-hygeine-practices-with-line-listing?year=2019 -View was missing for this report

• https://reporting.avniproject.org/question/630-school-dropout-ratio-with-line-listing?year=2019 -View was missing for this report

• https://reporting.avniproject.org/question/489-ifa-distribution-consumption-mildanemia

• https://reporting.avniproject.org/question/487-ifa-distribution-consumption-normalanemia

• https://reporting.avniproject.org/question/737-completed-visits

• https://reporting.avniproject.org/question/740-lab-test-report

• For latest value in observation pick value from not null latest observation.
• All / most reports having number should have total.

Metabase Reports - Taking time more than 2 mins

• Adolescent Extract -
  - Annual Visit - 3:40 mins
  - Quarterly Visit - 5:00 mins
• Adolescent Reports -
  - Views are missing.
• CHETNA- SNEHA
  - No Report
• ASHWINI
  - No Report
• CK
  - No Report
• IHMP
  - No Report
• JSS MP Sickle Cell
  - Lab Test Report - https://reporting.avniproject.org/question/740 (View is not working)
  - List of Relative - Screening not done - https://reporting.avniproject.org/question/305 - 4:50 mins
  - New enrolment (Screened) Vs Diseased - After 6:40 Mins gives the message “Question is taking too long”.
  - Sample Status (Pending Sample) - After 6:20 Mins gives the message “Question is taking too long”.
  - Sample pending for electrophoresis - Derived from Sample status report
• JSS Phulwari
  - Weight for Age grade - Phulwari children - https://reporting.avniproject.org/question/201 (Syntax error)
• JSSCP Phulwari
  - No Report
• Preconception
  - Monthly monitoring longitudinal report - https://reporting.avniproject.org/question/670 - 2:30 mins

https://docs.google.com/spreadsheets/d/162niLHjrvaCrSeU1NwwgAknK6fYC3IAx79_mmcjuZ9M/edit#gid=381423043

This is more of a log of what was done, so that we have a record of it somewhere.

Tasks done

1. Ensure all production connections from Metabase have sync turned off by default.
   
   

2. Add alerts and create tickets whenever read db CPU goes high.
   See https://avni.freshdesk.com/a/tickets/783 for an example ticket that is created.

3. On avni-server, improve performance on views that get created - Change view generation to start using the function get_coded_string_value.

4. Fix views based on 3 above for Hasiru Dala (KA and TN), JNPCT and MDSR.

5. Stop working on this for a while since things are much better.

• common resources
• implementation specific resources

Implementations -

• Adolescent - Sewa Rural
• Chetna
• CK - Already DONE
• IHMP
• Preconception
• JNPCT
• Hasiru dala
• Sickle Cell
• Phulwari - PHNI
• Phulwari - JSSCP
• MDSR

Create a report that shows all the visits are overdue for more than one months.

Jasper has configurable security settings to prevent SQL injection and other attacks.

We are observing unpredictability around this configuration which is blocking queries that start with the 'WITH' keyword. This was observed during the initial spike around Jasper. It mysteriously resolved itself after reverting to the default security configuration. The security configuration was not changed since but the issue reoccurred again today (June 9, 2020).

This security configuration has been turned off in order for work on reports to proceed but this needs to be looked into and setup correctly.

Optimise few JSS queries that are taking long time to execute which are spiking the CPU on the read database.

1. https://avni.freshdesk.com/a/tickets/798
   DB user: jss
   Log file - Search for text: "duration" (value over 50000ms)
   Reports using these queries: https://reporting.avniproject.org/question/143

2. https://avni.freshdesk.com/a/tickets/802
   DB user: jsscp
   Log file - Timestamp:11:40:16
   Reports using this query: https://reporting.avniproject.org/question/520 https://reporting.avniproject.org/question/476 https://reporting.avniproject.org/question/940, Jasper JSSCP Aggregate - Antenatal services during pregnancy.

3. https://avni.freshdesk.com/a/tickets/804
   DB user: jsscp (Query structure is similar to case 1 above)
   Log file - Timestamp:04:49:36
   Reports using this query: https://reporting.avniproject.org/question/940 https://reporting.avniproject.org/question/683

Steps to optimize:
Refer to this doc