intermine / intermine-data-browser-tool

InterMine Data Browser: a tool for exploring semi-homogeneous biological datasets

import { Meta } from '@storybook/addon-docs/blocks'

Intermine Data Browser

The goal of this project is to implement a faceted search tool to display the data from InterMine database, allowing the users to search easily within the different mines available around InterMine without the requirement of having an extensive knowledge of the data model.

For the project organization, please refer to InterMine

Quick Links

• Check out the latest deployed version
• Read the latest stable documentation docs
• Set up the project to contribute

Required Ontology Concepts (per mine) for each filter

Filter	Ontology Concepts
GO Annotation	goAnnotation.ontologyTerm.name
Dataset Name	dataSets.name
Pathway Name	pathways.name
Organism short name	Gene.organism.shortName
Location	locations.start + locations.end + locations.locatedOn.primaryIdentifier
Diseases (OMIM)	diseases.name
ClinVar	alleles.clinicalSignificance + alleles.type
Protein Localisation	proteinAtlasExpression.cellType + proteinAtlasExpression.tissue.name + proteinAtlasExpression.expressionType + proteinAtlasExpression.level + proteinAtlasExpression.reliability
Protein Domain Name	proteins.proteinDomainRegions.proteinDomain.name
Interactions	interactions.participant2.symbol + interactions.details.type + interactions.details.dataSets.name
Expression	atlasExpression.pValue + atlasExpression.tStatistic + atlasExpression.expression + atlasExpression.dataSets.name

Contributing Guidelines

1. Find a thing to fix/implement in Issues or come up with your own idea, create a discussion issue for it and get a feedback.

2. Fork the repository.

3. Create your new feature branch.

   git checkout -b my-new-feature

4. Install the packages.

This project uses yarn berry for package management. Make sure to have it globally installed before proceeding.

npm install -g yarn

If you run yarn -v, the version displayed will be v1*. Yet, this project will automatically use v2 berry.

5. Commit your changes.

git commit -am 'Add some feature

6. Push to the newly created feature branch.

git push origin my-new-feature

7. Create a new Pull Request

intermine-data-browser-tool's Issues

Class filters along top - allow additional filters

Right now we have Gene and Protein across the top of the tool. Theoretically we might wish to browse classes other than Gene and Protein.

This will need to be done defensively similar to the left-side filters - i.e. check the model and don't add filters for classes that don't exist. In this case it should be slightly easier, though as all classes will be root-level entities in the model.

So, suggested implementation:

• Show any classes tagged preferredBagType (see example) along the top bar (as we currently do with gene and protein)
• Add model to allow the user to add/remove any other class as a view type (store this in localstorage so the user doesn't have to re-add every time they revisit).

"Location" Filter shows error

Whenever I select the Location filter and enter the following details:
Chromosome: 12
Start: 1 End: 1000
And when I press the GO button. I get the following error in the console:

Uncaught TypeError: Cannot read property 'min' of undefined
error at common.min.js file

See which constraints have been added

Maybe a button to display the current constraints?

Create a log of filters added, each time a filter is added a text is added. Maybe something like:

1. Organism -> Homo sapiens
2. Pathway -> Signal transduction

Something to tell if you have added the API key or not for the current mine

Connected/not connected icon with tooltip?

Improve Getting Started or BUILD.md

More details to the Building docs can be added for better documentation. We can change the BUILD.md to getting-started.md containing more detailed instructions on setting up the repository, running it locally on the machine and other related information about processes and workflows.

Red/Green buttons in filters - change colours

Can we change the red/green colour pairing to something a little less bright and noticable? I think it's okay to leave the "go" button bright - it's the item we want to call attention to - but probably the reset should be muted, perhaps a grey of some sort.

As a side note, red/green is not a great pairing of colours in general - it's hard for colour blind users to differentiate, and colour blindness affects around 1 in 12 (8%) of the male european population

Further reading: https://www.smashingmagazine.com/2016/06/improving-color-accessibility-for-color-blind-users/

Gulp 3 is broken on Node 12

Gulp throws the following error when using node v12:

fs.js:27
const { Math, Object, Reflect } = primordials;
                                  ^

ReferenceError: primordials is not defined
    at fs.js:27:35

CORS problems

CORS error when changing to TargetMine:

Remove Graphs box if current view is not Gene or Protein

Update screenshots in the readme to reflect latest version (1.2) look

Update screenshots in the README.md to reflect latest version (1.2) look

Reset all button on top

Next to the mine selector?

Add select to the Views manager for adding a new view

Currently, in order to add a new custom view to the browser, it needs to write it in the corresponding input field in the Views manager:

This leads to a problem, which is that the user may write a view (class/facet) that actually doesn't exist in the current mine. On this extent, the idea is to change this input field by a select one, that gets populated with the available views (classes/facets) in the current mine. This auto-filled select feature is already available, for instance, in the GO Annotation filter:

So the idea is to replicate and adapt accordingly this code.

This issue directly closes this one and #23

Footer text moves out of the box in some devices

The text in the footer moves out of the container on some of the defined screen sizes. Here's the screenshot for the issue:

Update the dependencies

It seems that the dependencies have not been updated for a long time and it may be better if we stick to the latest versions. For the record, I've attached the screenshot of the dependencies which can be updated.

Force to show always the data from the organism you’ve chosen

Always force the organism constraint

Phenotypes (HPO/MPO) filter

It would be interesting to add a new sidebar filter, to filter by phenotypes. This would be very interesting to be able to analyze only those genes/proteins where the human had some specific characteristics, such as nocturia or abnormality of body height.

The idea is to have an autocomplete input box, in the same line as, for instance, the pathways filter, making it easier for the user to search for a specific phenotype. This filter should allow multiple terms at the same time, something that the pathways filter already supports.

The concrete facet of interest is "HPO.Term". An example XML query showing all phenotypes available in HumanMine is:

<query model="genomic" view="HPOTerm.identifier HPOTerm.name HPOTerm.ontology.name HPOTerm.description" sortOrder="HPOTerm.identifier ASC" >
</query>

This is a good first issue for prospective GSoC students, as it allows to get acquainted with handling constraints on intermine-tables.

Top-right button to show lists and filter by them

It is interesting to add a button on the top-right of the browser, just beside the "Add view" and "Add API key" ones, where a user can click and get the list of the lists it has stored in it's account.

This should check that the user has actually added a valid API key for the current mine in the "Add API key" section. This is stored as a LocalStorage object, so it should be easy to check if the key exists.

Then, add a button on the side of each list name, that says "Select", that just adds the constraint of showing that list genes in the im-table.

request module is not included in package.json

request module is not included in package.json as a result request module is not installed by the npm. As a result, npm fails to compile the code.

add windows setup guide

this issue is for @officialkome - they've set up an amazing windows tutorial on wordpress in #78 but would like to convert it to markdown :)

Notify on errors when loading the default filters

Sometimes the user get stuck in a mine that is giving errors, without the possibility to continue using the browser. To avoid this, notify that there is a problem and move to HumanMine

Write new tests from scratch that query using im.js

Description

Currently the tests fail due to the fact that HumanMine is HTTPS and we have issues with CORS

Steps to Reproduce

Run npm test

Expected Behaviour

All tests should pass.

Actual Behaviour

All the tests fail.

Settings

• OS: Ubuntu 18.04 LTS

Screenshots

Update year in Footer

Description

The year in the footer of the web app reads as "@ GSoC 2018". It needs to be updated to "@ GSoC 2020".

Steps to Reproduce

The correction needs to be made in 'footer.pug' file.

Add "current class" filters to left filter bar

For now this would simply be adding Gene and Protein filters (dependent on the active tab) to left - probably these will work ok all mines!

• This will mean we can allow IN queries if user is authenticated
• Also allow LOOKUP query with multiple identifiers pasted in or a single identifier (autocomplete possible here? Maybe not wise?)

This should be dynamic and not hard coded to Gene and Protein classes. I would suggest fixing this after #8 is implemented.

Support selection from multiple lists

Currently the user can specify only one list from which he or she can view Genes/Proteins etc. The aim of this issue is to enable a user to select from more than one list at a time (much like using "ONE OF" operator).
Here are the caveats:

• The imjs query interface supports adding only one of the list at a time.
• Selection from multiple lists can be supported using custom constraintLogic (like A or B where A and B denote the individual list constraints). But imjs query at the moment does not support changing the constraintLogic once it is set. See issue intermine/imjs#18
  For a more verbose description, refer to the discussion on PR #31

Stop using express for handling the requests and instead do all in client-side with imjs

Stop using express for handling the requests and instead do all in client-side with imjs

Buttons on the navigation bar overlap in some devices

The screenshots attached below itself explain the issue.

This is because the navigation bar is not completely responsive.

Usability - make it clearer how you select a specific gene or list

It's not entirely obvious to a new user how to filter by single gene (or other entity) or gene lists. From Gos:

I added an API key for my humanMine account but I couldn't see how to find my personal
list, nor search for specific gene etc.

I'll admit it took me a few seconds to figure it out as well. Any ideas on how we could make it more obvious?

Restore the dependencies' version used before commit #52

After updating some dependencies, the tool seems to generate errors mainly due to code breaking changes. To resolve them, we need to go back to the previous versions of the dependencies used before the commit #52 .

Server Started message on console

As of now, when we run npm start, we don't get any console message saying that the server is started already and that's why one keeps waiting on that step.
Adding an alert message Server Started at Port: XYZ at the end of all bundling processes will look good.

I'm working on this.

Add views: shouldn't be possible to add non-existent views

It looks like any free text is allowed in the add view dialogue :)

CSS for some ANDable filters is a little odd

See screenshot: words are overflowing the text box:

present in FF and Chrome

Show column summaries automatically above the im-table

Right now we have a single visualisation, the pie chart. Gos pointed out that we actually already have quite a few nice charts present in im-tables - the column summaries. Not all summaries are innately interesting, so I don't think we want to show each column summary.

This needs to be generic and work for tables that are neither gene nor protein.

Boring graphs:

example:, the Gene page has gene symbol, name, primary and secondary ID - all of which are unique strings or null, which doesn't give us any thing interesting.

Interesting graphs:

gene.length is an integer and can be plotted on a bar graph (interesting)
gene.organism is categorical data that is interesting in a pie or bar.

The same goes for proteins - length and organism name are probably interesting, the rest isn't.

implementation

This could literally take the imtables column summary graphs and place them in the div above the imtable (cloning the elements maybe?) or it could be that we simply add a bar chart that automatically picks up on data types but has nothing to do with the imtables column summaries. We'll need to think a little.

Suggest we only implement this after #8

Graphs for custom views

If I add a pathway view along the top, the graph displayer box is present but empty.

API modal - close button is too close to the save button.

Could we move close over to the left of the modal so it doesn't get clicked accidentally and/or as an x at the top right of the modal? 🦄

Make the organisms filter a multiple selector

Choose several organisms at the same time, when no organism is selected is like having no organism filter applied.

Organism priorities for organisms graph and filter using config file

Allow to say which organisms should be first in the plot and filter, and then add the ones with highest count of items (as happens now).

mine dropdown - clicking on the arrow doesn't work

Clicking on the arrow on the left of the dropdown doesn't do anything on FF or Chrome - you have to click on the word "humanmine" to get a response

Expression filter: correct sub-filters depending on dataset

Illumina bodymap should display Tissue and FPKM value sub-filters, whereas E-MTAB-62 should have the ones currently being displayed: P-value, T-statistic and UP/DOWN/NONDE.

Steps to fix this:

1. Change layout of the filter to be one cell per row, instead of spliting grid box in 2x2. So have a 4x1.

2. When changing dataset, which is gonna be the first row, alter the next rows accordingly (hide, show with CSS?)

Unwanted grey color background and extra row in the header

There's an unwanted space with grey background above the header, and there is a grey background under the menu.

Ideally we would like to remove that unwanted header spacing and use the same bbackground color (white?) everywhere.

Screenshot:

Add Disease Name extra sub-filter to the ClinVar filter

Basically like in the HumanMine template

inform user of a bad api key

If I paste a bad API key into the API key modal, the save as list button disappears from the imtable, which is great. Is the API key used (yet) in any other place in the app?

What might be even better is if we tell the user the API key isn't any good. e.g. something like

Error: API key provided for HumanMine is not a valid API key - please check you have entered it correctly.

Usability: add textual description to the API key and "Change views" buttons

Very few icons have automatically clear meaning to users, so it is usually best to add a text description as well as the icon itself. You can read more about icon usability on the neilsen norman group blog: https://www.nngroup.com/articles/icon-usability/

dev server of browser tool

Currently, the npm run dev command in project executes the command gulp && webpack --config webpack.config.js. We're missing webpack dependency in package.json and the config file webpack.config.js as well.

Add preferredOrganisms optional array to mineConfigs

Add preferredOrganisms optional array to mineConfigs. Load them first always in the pie chart and in the left side organism filter

Some sort of "loading state" when applying filters

Originally from @yochannah :

"When I run a chromosome search, it takes several seconds before the im-tables loader bar appears, during which time I find myself wondering if I actually pressed the Go! button at all. Is there a way to make the im-tables loading bar load earlier, or some other way to indicate to the user that the browser is thinking?"

Allow free text (write and press the enter key) and display something if there are not results

Allow free text (write and press the enter key) and display something if there are not results (and avoid the filter to be applied)

Make the app fully responsive

It would be ideal to make the app fully responsive, especially header, menu, footer, etc.

Modularize the project using React components or Vue for better scalability

We would like to modularize the project using React components for better scalability.

Protein Domain Name filter is giving filterQuery error

mine filters - refactor to use pathquery

We would like to create a webservice to allow mine admins to explicitly state which filters will be available in the tool for their mine. Right now our filters are hard-coded to certain functions. This won't scale well for other InterMines - instead, we should consider something like this:

{
  mineUrl: "http://www.kittenmine.com/kittenmine" //I think URL is probably the safest way to uniquely identify a mine
  customFilters: [{
      filterQuery: {
        "constraintLogic": "A and B",
        "from": "Gene",
        "select": [
          "alleles.clinicalSignificance"
        ],
        "orderBy": [{
          "path": "alleles.clinicalSignificance",
          "direction": "ASC"
        }],
        "where": [{
            "path": "alleles.type",
            "op": "=",
            "value": "XXX",
            "code": "A"
          },
          {
            "path": "alleles.clinicalSignificance",
            "op": "=",
            "value": "YYY",
            "code": "B"
          }
        ]
      },
      filterName: "CLINVAR"
    },
    {
      filterQuery: {
        "constraintLogic": "A and B",
        "from": "Gene",
        "select": [
          "alleles.clinicalSignificance"
        ],
        "orderBy": [{
          "path": "alleles.clinicalSignificance",
          "direction": "ASC"
        }],
        "where": [{
            "path": "alleles.type",
            "op": "=",
            "value": "XXX",
            "code": "A"
          },
          {
            "path": "alleles.clinicalSignificance",
            "op": "=",
            "value": "YYY",
            "code": "B"
          }
        ]
      },
      filterName: "ClinVar"
    }, {
      filterQuery: {
        "from": "Gene",
        "select": [
          "diseases.name"
        ],
        "orderBy": [{
          "path": "diseases.name",
          "direction": "ASC"
        }]
      },
  "where": [
    {
      "path": "diseases.name",
      "op": "=",
      "value": "ZZZ",
      "code": "A"
    }
  ]
    },
    filterName: "Diseases"
  ]
}