lambda-science / impatient

IMPatienT🗂️: an integrated web application to digitize, process and explore multimodal patient data. Demo version deployed at: https://impatient.lbgi.fr/

IMPatienT 🗂️: an integrated web application to digitize, process and explore multimodal patient data.

IMPatienT 🗂️ (Integrated digital Multimodal PATIENt daTa) is a web application developped in the MYO-xIA project for patient data digitization and exploration. It features a standard vocabulary creator, optical character recognition (OCR), natural language processing (NLP), image annotation and segmentation using machine learning, interactive visualizations and automatic diagnosis suggestion.

A demo version is currently deployed at: https://impatient.lbgi.fr/ This project is free and open-source under the AGPL license, feel free to fork and contribute to the development. Several guides are available at the bottom of this page for the production deployment and developer mode.

Contact:

The main maintainer is: Corentin Meyer - PhD in Biomedical AI [email protected]

Citing IMPatienT🗂️

[placeholder]

Partners

IMPatienT is developped and used in collaboration with the Morphological Unit of the Institute of Myology of Paris. A production instance is deployed to help discovering new relevant features for congenital myopathies classification and diagnosis.

IMPatienT🗂️ Abstract

Background
Medical acts, such as imaging, generally lead to the production of several medical text reports that describe the relevant findings. Such processes induce multimodality in patient data by linking image data to free-text data and consequently, multimodal data have become central to drive research and improve diagnosis of patients. However, the exploitation of patient data is challenging as the ecosystem of available analysis tools is fragmented depending on the type of data (images, text, genetic sequences), the task to be performed (digitization, processing, exploration) and the domain of interest (clinical phenotype, histology…). To address the challenges, the analysis tools need to be integrated in a simple, comprehensive, and flexible platform.
Results
Here, we present IMPatienT (Integrated digital Multimodal PATIENt daTa), a free and open-source web application to digitize, process and explore multimodal patient data. IMPatienT has a modular architecture, including four components to: (i) create a standard vocabulary for a domain, (ii) digitize and process free-text data by mapping it to a set of standard terms, (iii) annotate images and perform image segmentation, and (iv) generate an automatic visualization dashboard to provide insight on the data and perform automatic diagnosis suggestions. Finally, we demonstrate the usefulness of IMPatienT on a corpus of 40 simulated muscle biopsy reports of congenital myopathy patients.
Conclusions
IMPatienT is a platform to digitize, process and explore patient data that can handle image and free-text data. As it relies on a user-designed vocabulary, it can be adapted to fit any domain of research and can be used as a patient registry for exploratory data analysis (EDA). A demo instance of the application is available at https://impatient.lbgi.fr.

Setup guides

(DOCKER) Developper Mode Setup (to contribute)

See the wiki page: Developper Mode Setup (DOCKER)

(DOCKER) Deploy to production & Maintain IMPatienT

See the wiki page: Deploy and maintain (DOCKER)

(LINUX) Deploy to production & Maintain IMPatienT

See the wiki page: Deploy and maintain (LINUX)

(LINUX) Developper Mode Setup

See the wiki page: Developper Mode Setup (LINUX)

impatient's Issues

Delete Legacy Code

Delete legacy code, config files and text that will not be use for publication.

Check all translation in code, pages and ontology.

Check all translation in code, pages and ontology to prepare for publication

Modify On Image Upload Doesn't Add Image File

Improve Visualization Module

• Download table as CSV
• Automatic Graph by age / diag / muscle
• Automatic correlation Matrix

Img anot loading template is bugged because of login asking

Dash Image annotation: upload image & select & save

Find a way in the image annotation module to upload an image, select the image from DB and save/load the annotations.
One possible way would be a separate page for image uploading.

Change footer link

Visualisation improvement

Add title, axis colors to plots.
Add BOQA Results ?

Make sure report digitalization in translated and working fine (NO GENE error)

Module 4: Enhance

Add BOQA Results and UNCLEAR reclassification.
For each report store prediction in DB so not always calculated

[FEATURE] META Improvement

Backlog of ideas to for META improvement (structure, deployment...):

• Add Admin Interface to create accounts
• Remove NGINX Cache or try with production settings (to fix images issues ?)
• Might add a role-base / user-based gestion of data

Modified patient flag

Translata Ontology

[FEATURE] Improving Vocabulary Module

Backlog of ideas to improve the vocabulary module:

• Switch lang button should become FRA/ENG buttons (two sepearates buttons), might use JSTree metadata field to check the current vocab language
• Add a "image annotation" tickbox as nodes properties (boolean) to show or not on image annotation module

Add Encryption to the Database

All input in database should be encrypted using AES-256 bits.

1. Set the secret key in the .env file
2. Make sure all form encrypt data before saving to DB.
3. Use secret key to read data

Reference:

• Generate key: https://www.allkeysgenerator.com/Random/Security-Encryption-Key-Generator.aspx
• CBC Algo with IV patient: https://python-guide-pt-br.readthedocs.io/fr/latest/scenarios/crypto.html

Full Gene list in menu

Visit

• Mutation Field + Phenotype field
• Re-use model for image
• Tiff file convertion

[FEATURE] Invert onto should update patients for viz

Title

Module 3: Enhance

Modify Show Segmentation to Button
Modify Blurring parameters to a single value
Modify tag list to Standard vocabulary part "image tagging"
If classifier is not none load it

[FEATURE] Import existing ontology such as HPO in module 1.

Title.

Module 2: Enhance

Per trms genes: show all and marks linked as colored
Changer info terms de champs à HTML classique
Ajouter champs gene suspecté, modifier diagnosed as tag list

Make sure ontology edition is working as expected

[FEATURE] Improving Image Module

Backlog of ideas to improve the image module:

• Find out why sometimes annotations don't show up on refresh, same question for image
• Find out why the layout doesn't get updated when new terms are added (remove cache ?)
• Fix the form image upload issue when modifying data (New Form ?)
• Load previous model for new images ?
• Simplify code base

Load Genes informations in DB

Auto stats for BOQA after new patients

Migrate to Conda ENV

Would probably be better to migrate to a conda env for more cross-platform compatibility.

Annot saving: save bluring parameters features

Also load CLF and trigger the classif

Module 1: Simplify Fields

no distinction gene / gene datamined
sort genes
show un-asigned genes
find complete gene list
synonymes + other lang

Image Annotation URL when not coming from index

Clicking on Image Annotation Link when not being on the root URL is bugged.

[FEATURE] Improving Dashboard Module

Backlog of ideas to improve the dashboard module:

• Don't re-do all calculations and viz when visiting the page, only trigger it on ontology change or new reports.

[FEATURE] Improving Report Module

Backlog of ideas to improve the report module:

• Add a red X button to vocabulary terms overview
• Add RegEx to prefill age, biopsy ID and date
• Hide the debug modal
• Add a modified patient hook in SQLite DB to redo visualisations automatically (and not always)

[Security] IMPatienT v1.5.0 Stored Cross-Site Scripting (XSS) - CVE-2023-23637

A Security Advisory has been raised for IMPatienT v1.5.0 (CVE-2023-23637):

Description:
IMPatienT v1.5.0 allows Stored Cross-Site Scripting (XSS) via onmouseover in certain text fields within a PATCH /modify_onto request.
This may allow attackers to steal Protected Health Information (PHI).

Suggested Fix:
Consider sanitizing user input parameters by removing all non-compliant characters. Additionally, you could consider encoding the user input using HTML or URL methods.

Reference:
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-23637
https://nvd.nist.gov/vuln/detail/CVE-2023-23637
https://owasp.org/www-project-top-ten/2017/A7_2017-Cross-Site_Scripting_(XSS)

Payload:

PATCH /modify_onto HTTP/1.1
Host: 127.0.0.1:5000
Content-Length: 2218
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.5414.75 Safari/537.36
Origin: http://127.0.0.1:5000/
Referer: http://127.0.0.1:5000/ontocreate
Connection: close

[{"id":"MHO:000001","text":"Sample Keyword","icon":true,"li_attr":{"id":"MHO:000001"},"a_attr":{"href":"#","id":"MHO:000001_anchor"},"state":{"loaded":true,"opened":true,"selected":false,"disabled":false},"data":{"description":"","synonymes":"","phenotype_datamined":"","gene_datamined":"","alternative_language":"Sample Keyword","hex_color":"#c7ef34","hpo_datamined":"","correlates_with":"","image_annotation":false},"parent":"#"},{"id":"MHO:000004","text":"Keyword Image Annotation","icon":true,"li_attr":{"id":"MHO:000004"},"a_attr":{"href":"#","id":"MHO:000004_anchor"},"state":{"loaded":true,"opened":false,"selected":true,"disabled":false},"data":{"description":"","synonymes":"","phenotype_datamined":"UNCLEAR","gene_datamined":"N/A","alternative_language":"","correlates_with":"","image_annotation":true,"hex_color":"#77e3a4","hpo_datamined":""},"parent":"MHO:000001"},{"id":"MHO:000005","text":"Keyword Image Annotation 2<a onmouseover=alert('XSS')>XSS</a>","icon":true,"li_attr":{"id":"MHO:000005"},"a_attr":{"href":"#","id":"MHO:000005_anchor"},"state":{"loaded":true,"opened":false,"selected":false,"disabled":false},"data":{"description":"","synonymes":"","phenotype_datamined":"","gene_datamined":"","alternative_language":"","correlates_with":"","image_annotation":true,"hex_color":"#094f6a","hpo_datamined":""},"parent":"MHO:000001"},{"id":"MHO:000002","text":"Sample Keyword Child","icon":true,"li_attr":{"id":"MHO:000002"},"a_attr":{"href":"#","id":"MHO:000002_anchor"},"state":{"loaded":true,"opened":false,"selected":false,"disabled":false},"data":{"description":"","synonymes":"","phenotype_datamined":"","gene_datamined":"","alternative_language":"","correlates_with":"","image_annotation":false,"hex_color":"#14cd17","hpo_datamined":""},"parent":"MHO:000001"},{"id":"MHO:000003","text":"Sample Keyword Child 2","icon":true,"li_attr":{"id":"MHO:000003"},"a_attr":{"href":"#","id":"MHO:000003_anchor"},"state":{"loaded":true,"opened":false,"selected":false,"disabled":false},"data":{"description":"","synonymes":"","phenotype_datamined":"","gene_datamined":"","alternative_language":"","correlates_with":"","image_annotation":false,"hex_color":"#d9eab9","hpo_datamined":""},"parent":"MHO:000001"}]