berntpopp / sysndd

GitHub Repository for the SysNDD database, web app and api used to curate gene disease relationships in neurodevelopmental disorders (NDD).

SysNDD is the expert curated database of gene-inheritance-disease relationships in neurodevelopmental disorders (NDD).

The SysNDD GitHub repository

This repository is for development of our SysNDD web application (app), application programming interface (api) and relational database (db). Browse the sub-foldes to view the respective readme files and source code.

Table of contents

• Quick start 🏁
• Documentation 📝
• Contributing and community 👥
• Bugs and feature requests 🪲 & 🌟
• Creators 👩‍🔬
• Support and Funding 🤗
• Credits and acknowledgement 👍
• Copyright and license ©️

Quick start

The SysNDD installation depends on docker and docker-compose. If these are installed the project can be installed locally using the provided shell script:

bash deployment.sh "<config.tar.gz>"

• A dummy config file for local deployment will be provided in this repository.
• Data and a script to populate the MySQL database will be provided in this repository.

Documentation

Please explore The SysNDD Documentation hosted on GitHub pages and build with bookdown.

To help you get the most out of the SysNDD website, we are putting together a series of video tutorials.

Contributing and community

To contribute in curating novel entries to our database you can register for a new reviewer/ curator account on the SysNDD page.

Ask questions, report bugs and chat about SysNDD in general using our Github discussions page.

Bugs and feature requests

If you have technical problems using SysNDD or requests regarding the data or functionality, please contact us at support [at] sysndd.org.

Creators and contributors

Bernt Popp (SysNDD)

• https://twitter.com/berntpopp
• https://github.com/berntpopp
• https://orcid.org/0000-0002-3679-1081
• https://scholar.google.com/citations?user=Uvhu3t0AAAAJ
• https://www.berntpopp.com

Christiane Zweier (SysID, SysNDD)

• https://orcid.org/0000-0001-8002-2020
• https://scholar.google.com/citations?user=KE0N1r8AAAAJ

Annette Schenck (SysID)

• https://twitter.com/annette_schenck
• https://orcid.org/0000-0002-6918-3314
• https://www.schencklab.com

Melek Firat Altay (SysNDD)

• https://twitter.com/firataltay
• https://github.com/altay-epfl
• https://orcid.org/0000-0002-8174-5631
• https://linkedin.com/in/melek-firat-altay

Support and Funding

The current SysNDD database development is supported by:

• DFG (Deutsche Forschungsgemeinschaft) grant PO2366/2-1 to Bernt Popp.
• DFG (Deutsche Forschungsgemeinschaft) grant ZW184/6-1 to Christiane Zweier.
• ITHACA ERN through Alain Verloes . The previous SysID database and data curation was supported by:
• The European Union’s FP7 large scale integrated network GenCoDys (HEALTH-241995) Martijn A Huynen . and Annette Schenck.
• VIDI and TOP grants (917-96-346, 912-12-109) from The Netherlands Organisation for Scientific Research (NWO) to Annette Schenck.
• DFG (Deutsche Forschungsgemeinschaft) grants ZW184/1-1 and -2 to Christiane Zweier.
• the IZKF (Interdisziplinäres Zentrum für Klinische Forschung) Erlangen to Christiane Zweier.
• ZonMw grant (NWO, 907-00-365) to Tjitske Kleefstra.

Credits and acknowledgement

We acknowledge Martijn Huynen and members of the Huynen and Schenck groups at the Radboud University Medical Center Nijmegen, The Netherlands, for building SysID and supporting it for many years. We would also like to thank all past users for using SysID and for constructive feedback, thus making the sometimes tedious updates and re-organization into the new SysNDD database worthwhile. Since recently, Alain Verloes and ERN ITHACA provide valuable encouragement and support by initiating and supporting the data integration with Orphanet and helping with the recruitment of expert curators.

Copyright and license

• All code from this project is licensed under the "MIT No Attribution" (MIT-0) License - see the LICENSE.md file for details.
• The project data, website and api usage are licensed under the "Attribution 4.0 International" (CC BY 4.0) License - see the https://creativecommons.org/licenses/by/4.0/ for details.

sysndd's Issues

Functionality to search for new GeneReview articles

Add functionality to search for new GeneReview articles related to SysNDD entities and allow users to add these to the respective entity from a tabular view.

Feature request: Admin View for Managing Phenotypes and Related Data

Description

Create a new tab in the Admin section to add and administer phenotypes, inheritance modes, variant types, etc.

Expected Outcome

• A user-friendly interface in the Admin section.
• Features for adding and managing phenotypes, inheritance modes, variant types, etc.

Milestone

Admin section views

Tags

• enhancement
• admin
• user interface

Feature request: GeneReviews Update Enhancement

Description

Implement a process to identify genes with existing GeneReviews articles and update the system accordingly. As an immediate solution, create an Excel table linking to GeneReviews articles for genes lacking this information.

Expected Outcome

• Updated records of genes with GeneReviews articles.
• An Excel table as a quick reference for genes without GeneReviews entries.

Token update cant be done from modals

The menu bar cant be accessed from a modal so when the logout warnig is shown, the suer needs to close the modal to refereh the token.

Solution:
Add a refresh button to the logout warnig toast.

Feature request: Implement Variant Ontology Computation from Synopsis Text

Description

Develop a feature to compute variant ontology by extracting key terms (e.g., missense, splice, in-frame, deletion, variants, mutation) from synopsis text. Integrate data sources like OMIM/ClinVar for enhanced accuracy.

Expected Outcome

• Automated ontology computation for variants.
• Integration with OMIM/ClinVar to enrich variant data.

Rename entity_quadruple constraint to entity_triple

The entity_quadruple constraint in ndd_entity is actually a triple and should be renamed accordingly ("entity_triple").

Functionality to initiate Publication Table update/ check

Add functionality to update the publication table.
This should recompute the values and update the table.
It will need added functionality in both the API and APP

TODO:

• create an Endpoint for publication table update using the database script, the EP should be authenticated and only Admin users should access it
• create a view in the Admin tree to initiate the publication upd

Generate disease ontology update functionality

Convert the OMIM disease_ontology_set Update Script into an Endpoint and make an Admin View to update the table from the frontend.

TODO:

• define logic and possible data structure for entities that loose their ontology link through update
• convert the disease_ontology_set Update Script into an API function, improve calculation speed or use promises
• make an Admin View to update the table

Automatically build the documentation using GitHub actions

Configure automatic builds of the documentation using GitHub actions.

Examples:

• https://medium.com/@delucmat/how-to-publish-bookdown-projects-with-github-actions-on-github-pages-6e6aecc7331e
• https://github.com/rstudio/bookdown/actions
• https://lucacozzuto.medium.com/automatic-reports-with-bookdown-and-github-actions-3c19498394aa

Make an endpoint to update the disease_ontology_set

Convert the OMIM disease_ontology_set Update Script into an Endpoint and make an Admin View to update the table from the frontend.

TODO:

• define logic and possible data structure for entities that loose their ontology link through update
• convert the disease_ontology_set Update Script into an API function, improve calculation speed or use promises
• make an Admin View to update the table

BUG: Network error wihen fetching the last commit

The last commit is fetched on every route change. This seems to cause an Network error:

And the status being displayed as offline wrongly:

Implement a retry mechanism and a different online check (maybe using the API) and cache the last commit per session. Possibly get the last commit through the own API where it could be cached.

Bug: Unapproved users can be approved in admin view without setting the password and the abbreviation

Unapproved users can be approved in admin view without setting the password and the abbreviation. The approval status should be highlighted and if the admin approved through this way, the respective routines should be started.

Feature request: Database Creation Scripts Enhancement and Repository Update

Description

Enhance and update database creation scripts. Key tasks include replacing hardcoded values like OMIM links or API keys with config file values, creating a master script to execute others, commenting scripts, writing a README, adding SQLite version of the initial SysID import for reproducibility, and making scripts independent of the working directory.

Expected Outcome

• Updated database creation scripts in the GitHub repository.
• Enhanced documentation and setup process.

Tags

• enhancement
• database
• documentation

Feature request: Implement User Rights Management Interface

Overview

The SysNDD application requires a new interface for managing user rights. This feature will allow administrators to modify user attributes, excluding user_id.

Requirements

• Tab/View for User Management: Create a new tab or view named "Manage User Rights."
• User Table Adjustments: Enable modifications to user fields including user_name, password, email, orcid, abbreviation, first_name, family_name, user_role, comment, terms_agreed, approved, rereview_request, created_at, and password_reset_date.
• API Endpoint Development: Develop and integrate new endpoints such as api/user/update (PUT for JSON objects) and api/user/delete (DELETE requests).
• Interactive User Management Table: Include a table with functionalities like role changes, user approval, and deletion.

Existing Resources

• Adapt and extend functionalities from the current ApproveUser.vue component, which includes mechanisms for user role changes and approval toggles.
• Utilize backend logic in tableMethodsMixin.js for handling table data and user interactions.
• Reference the api/user/change_role endpoint as a model for additional backend functionalities.

Additional Considerations

• Focus on security and data integrity in implementation.
• Ensure user-friendly navigation and experience within the new interface.
• Comprehensive testing of all new features and endpoints for reliability.

Feature request: Direct Approval Option in Create & Modify Entity Pages

Description

Modify the 'Create & Modify entity pages' to include a direct approval option for specific user groups. Implement API-side credentials and rights check.

Expected Outcome

• A switch button to directly approve entries.
• Restricted visibility based on user group permissions.

Feature request: Add 'Removal' Button in Status Modal

Description

Implement a feature to allow removal of an entity if suggested. This should be a button in the status modal in approval mode.

Expected Outcome

• A 'Removal' button in the status modal.
• Allows users to easily perform the removal of an entity.

Bug: double loading authentication

The login with jwt is apparently done twice as it currently shows the not allowed error 500 twice.

Feature request: API Endpoint for Pubtator Queries and Gene List Generation

Summary

Develop an API endpoint that interacts with Pubtator to query and generate a list of genes related to intellectual disability that are not yet annotated in SysNDD.

Description

The new API endpoint should facilitate the automatic retrieval of gene information from Pubtator. This includes performing searches, parsing results, and comparing against our database to list genes that have not been annotated in SysNDD.

Example Searches

https://www.ncbi.nlm.nih.gov/research/pubtator/?view=docsum&query=intellectual+disability+AND+gene
https://www.ncbi.nlm.nih.gov/research/pubtator-api/publications/search?page=1&q=intellectual+disability+AND+gene

Requirements

• Develop a system to save query results in the database with minimal tables or views indicating whether the gene is already annotated in SysNDD.
• Create a curation system to review publications and determine if a gene found via the API should be included in SysNDD (and specify reasons).
• Implement functionality to display this information in the Analysis view.
• Formulate an efficient search query to find most of our old publications and analyze their inclusion.

Additional Notes

• This feature is expected to improve our database's completeness by integrating the latest research publications.
• Consider the potential volume of data and optimize for performance.

Feature: DB database version

This should be in a table in the database using semantic versioning and ideally include the last db folder related commit.
Also display in App and API.

Add the ability to search, filter, and/or download variant type annotations.

The variant annotation is currently only displayed on individual entity pages and has no search functionality.
It would be interesting to add this to the Excel download tables and enable searching for it per user request.

Options:
Make an own view like the phenotype view to allow searching for variant annotations
Simply include the variant annotation in the response and flatten it for Excel download.

Feature: add API version

This should be shown in the swagger frontend using semantic versioning and the last commit affecting the API folder.

Ideally this should also:

• have a endpoint to retrieve version
• be displayed in the frontend

Docs: Explain the "not applicable" category in detail and give examples

The "not applicable" category needs to be better explained

Bug: HPO and Variant ontology terms are not saved with review

Error 500 when saving a review with message HPO_term not found.

BUG: deactivated entity causes error when creating it again

No new entry can be created for the following genes: NSD2, ATP1A3

Problem debugging and description:

• api/entity/create fails if the ndd_entity table already has a deactivated entry with the same combination ("entity_quadruple" constraint).
• constraint is important to not duplicate entities and should not be removed
• the problem is likely caused either by manual renaming of an entity or during the OMIM update, where old entities were temporarily renamed to MONDO terms

Possible solutions and improvements for stability:
add a button to reactivate a deactivated entity in the create entity view
check the presence of the entity after it has been composed in the create entity view; add warning and button if it exists
Design an admin view which allows editing the table entries directly

Other TODO:
fail more gracefully by showing the error message in the toast message or console and not only the code
rename entity_quadruple constraint to entity_triple (see #5)

Bug: logout check causes repeated warning toast

When the jwt token expired it seems like it is not deleted. It is then checked and an error toast 500 is shown repeatedly.

BUG: ontologies with questionmark causes 404 page

OMIM entries with questionmark cuase 404 page.
Examples:
https://sysndd.dbmr.unibe.ch/Search/coloboma

→ https://sysndd.dbmr.unibe.ch/Ontology/?Coloboma, ocular

Problem is inconsistent URl encoding/decoding in API and App especially for questionmarks

Bug: Inconsistent Entity-Batch Grouping for Specific Genes in sysndd Webtool

The sysndd webtool currently exhibits a bug related to the grouping of entities for specific genes within the same batch. This issue has been identified for several genes, including GNAO1, GRIN1, and IFT172. The problem arises when conducting a re-review of the data, as not all entities associated with a specific gene are included in the same batch.

Bug: Preserve Re-Reviewer Identity When Changing Reviews

Description

Modify the review process to preserve the identity of the re-reviewer when changing reviews, unless substantial changes are made by the curator.

Expected Outcome

• Implementation of a default active checkmark to prevent overwriting the re-reviewer.
• Adaptations in the API and frontend to support this feature.

Bug: Review Concept of Disease Renaming Requiring No Approval

Description

Investigate and potentially redefine the process regarding disease renaming, particularly whether it requires approval.

Expected Outcome

• Clarification on the current process.
• Introduction of approval checks if deemed necessary.

Feature request: Develop Combined Status & Review Modal

Description

Create a modal for modifying both the status and review of an entity. This aims to simplify the workflow for updates.

Expected Outcome

• A combined modal for status and review adjustments.
• Direct approve option for streamlined processing.

Feature request: Functionality to initiate gene table update/check

Add functionality to update non_alt_loci_set.
This should recompute the values and update the table.
It will need added functionality in both the API and APP

TODO:

• create an Endpoint for table update using the database script, the EP should be authenticated and only Admin users should access it
• create a view in the Admin tree to initiate the update

Feature: add download possibility to all plots

Add buttons for download to all plots.

Feature: add app version to frontend

Add and display the current version of the webapp in the frontend.
This should use semantic versioning and the last commit.

Bug: Admin level users can not change to curation view in re-review

Admin level suers can not activate the curation view in re-review.
Likely fiaxable in UI.

Docs: Write a description on how to check and report Bugs using the console of Chromium browsers and minimal problem report

Write a description on how to check and report Bugs using the console of Chromium browsers and minimal problem report

Bug: APP/API - Entities over time by gene are not correct right now, too many

Likely caused by some flags not beeing filtered somehow

Both for gene and Symbol

Bug: Re-Review in approval although not submitted

Description

In the re-review process for lamb1, there seems to be an issue. The entity lamb1 is appearing in the approval list prematurely, before it has been officially submitted for approval by the reviewer.

Steps to Reproduce

1. Navigate to the re-review section.
2. Observe the appearance of lamb1 in the approval list.
3. Confirm whether lamb1 has been submitted for review.

Expected Behavior

Entities should only appear in the approval list post-submission for re-review.

Actual Behavior

The entity lamb1 is visible in the approval list before submission, indicating a potential flaw in the submission or listing process.

Additional Context

This issue may affect the workflow and efficiency of the re-review process, necessitating a prompt fix.

Bug: Unsubmitted and Approved Entries Disappear

Description

Resolve the bug where entries can be unsubmitted and approved simultaneously, causing them to disappear from the database.

Steps to Reproduce

1. Unsubmit an entry.
2. Approve the same entry without resubmitting.

Expected Outcome

• Prevention mechanisms against this scenario.
• Entries should not disappear when this sequence of actions occurs.

Bug: PMID not accepted when there's a Space

Description

When entering a PMIDs in the genereviews section, if there's a space in the PMID, the system fails to accept it

Steps to Reproduce

1. Go to the re-review section.
2. Input a PMID with an appended space (e.g., PMID: 12345678 ).
3. Attempt to submit.

Expected Behavior

The system should trim any spaces in the PMID.

Feature: add offline check

Add an indicator for offline status of the app.

Feature request: Make Username Input More Prominent in Assign Re-Review

Description

Improve the visibility and prominence of the username input field in the Assign Re-Review interface.

Expected Outcome

• Enhanced user interface for the username input field in the re-review assignment process.

Bug: Can not create new entities for some genes

Examples:
NSD2, ATP1A3

Bug: Unsubmitted Reviews Appearing in Approve Review View

Description

Bug where reviews unsubmitted in curation re_review mode still appear in the approve review view. Investigate and rectify the error.

Expected Outcome

• Corrected logic so unsubmitted reviews do not appear in the approval view.

Refactor tabular components

Multiple tabular view components are implemented within the views and repeat code. Functioning and control schemes for the four attached components are remarkably similar. Using the DRY principle, these should be simplified these by externalizing similar and repetitive code.

TODO:

• Create a base table component
• Use mixins for shared logic
• Utilize scoped slots for custom content
• Centralize API calls

Move genereviews functions from publication functions scripts

Move all genereviews related functions from the publication functions script to the genereviews-functions.R script.
Refactor where needed.

Docs: Update or add the description for the variant ontology curation

Describe how variant ontology curation is done

Feature request: Implement Download Feature for D3 Plots as PNG

Description

Enhance the user interface by adding a feature to download D3 plots as PNG images. This functionality will enable users to easily save and share visual data representations.

Expected Outcome

• A download button or link associated with each D3 plot.
• Capability for users to download plots as PNG images.
• Ensure the downloaded PNG maintains the quality and resolution of the plot.

Implementation Notes

• Explore D3's native capabilities or additional libraries for converting SVG to PNG.
• Test the feature across different browsers for compatibility.
• Maintain the aesthetic and resolution integrity of the plots in the PNG format.

Feature: automate CSR file creation and certificate signing process

Currently we have to generate the CSR file manually each year which is slow and causes delays absed on the availability of persons performing the CSR generation and sending of the fiels over mail.
It would be great to integrate a script that automatically generates the CSR file once per year and send it to the signing autority receiving the certifacte back and restarting the server with the new key.

Ideas:

• https://security.stackexchange.com/questions/104352/automate-certificate-signing-in-linux

BUG: missing entries in ndd_entity_view due to unapproved status cause modal crash

Entries in the curation - approve review view cannot be opened for editing (pencil icon), and it then hangs:
Examples: SLC33A1, PEX6, SLC12A6, SLC2A1
Affected entity_ids: 4074- 4081

Problem debugging and description:

• ndd_entity_view joins ndd_entity table and ndd_entity_status_approved_view
• if the status in ndd_entity_status_approved_view is not approved, it will not appear in ndd_entity_view
• this causes the filter (e.g. /api/entity?filter=equals(entity_id,4081)) to return an empty data object
• this in turn causes the entity_info object to be populated with empty data in the function getEntity in the ApproveReview.vue view

Possible solutions and improvements for stability:

• change the ndd_entity_view to display unapproved entities; add a standard filter to only show the approved status entries (caution: this is a non authenticated endpoint)
• QUICK FIX: approve the status before approving the review; document this
• do not use this endpoint here; instead, use one that is authenticated to retrieve data directly from ndd_entity
  → Should endpoints with unapproved data generally be authenticated?

Other TODO:

• implement a warning in the filter EPs if there is no data returned
• gracefully fail if no data is returned and show the warning
• getEntity in ApproveStatus.vue also returns no data for these entries and fails when opening the modal

Docs: add a description how to find the PMID for a GeneReviews article

Add to the GitHub Pages