neuroanatomy / neuroweblab Goto Github PK

Context

We are computational neuroanatomy lab working at Institut Pasteur in Paris. Through the years we have developed several Web projects, aimed at building collaborative neuroscience tools. There is in particular 2 projects – Microdraw and BrainBox – that we would like to improve.

Microdraw is a tool for viewing and working with very high-resolution microscopy images. It uses OpenSeadragon for fast, low band-width, access to very large images (the only limit is storage space), and Paper.js for vectorial annotations.

BrainBox is a tool for viewing and working with 3D MRI data. Using websockets, it allows several users to segment (draw) on top of the MRI data simultaneously. This is useful for creating 3D models of specific brain structures, or for quality-controlling and fixing segmentations obtained with automatic tools. This video provides a short description of BrainBox (a very old version of the tool).

Both Microdraw and BrainBox allow neuroscientists to visualise and work with data that's already available in the Web. Together, they provide access to tens of thousands of datasets, and terabytes of data, which become living matter for scientific collaboration.

The challenge

Microdraw and BrainBox share a common set of functionalities, for allowing users to create user profiles, create projects, add collaborators and assign permissions. Although the interfaces look similar, and the code is also similar, both code-bases were developed in parallel, with many parts which are redundant, incompatible, or which have a better implementation in one tool but not the other.

We would like to extract this common core, and refactor it as an independent module: the NeuroWebLab. That would allow us to simplify and improve the maintenance of Microdraw and BrainBox, but also to re-use that common core for our other Web applications and tools. Other neuroscientists could then also use NeuroWebLab, contributing to the creation of an open collaborative environment for working with neuroscientific data in the Web.

The project can be split into smaller objectives, which could potentially be done by different people:

1. Refactor the /project/settings page (where you add collaborators, datasets, annotation types, etc.)
2. Refactor the /project page (where you see the list of datasets to the left and are able to work on the image to the right)
3. Refactor the data access control and authorisations
4. Consolidate the DB structure and the API (harmonise db.user/db.users, db.project/db.projects, etc.)
5. General devops/automatisation, including the development of additional tests.

Technologies and skills required for the project

The server-side code of Microdraw and BrainBox uses Node, Express and MongoDB. The client side code uses Javascript, and although there are still many traces of jQuery, we're aiming at using exclusively ES6 code and Vue. Real-time interaction relies on Websockets. We use ESlint for coding style, Mocha and Chai for unit tests and integration tests, and continuous integration through GitHub actions. Our npm scripts use Bash.

In particular, it is very important to have good knowledge of Mocha and Chai, because that is the only way in which we can make sure that the extraction of NeuroWebLab from Microdraw and BrainBox is not breaking some other functionalities. The coverage of the tests is not bad, but it's far from complete. It would be important to write tests for critical functionalities.

Our team

We are a small neuroscience lab, and we shouldn't be writing code, but writing papers. However, we are convinced that collaborative work on open data is essential for making real progress in our field. We are 3 people in our team working in maintaining and improving Microdraw and BrainBox: @katjaq, @ntraut and @r03ert0. These are the tools we use daily. They are also used by many collaborators world-wide.

Admin

We collaborate with Just One Giant Lab for this project, and through them we're able to hire devs as freelancers. In France, we have hired freelancers through Malt.fr, but we're happy to look into other similar platforms.

sub task of #2

per discussion with @r03ert0 on mattermost:

a big chunk of the DB handling from microdraw (MD) and brainbox (BB) is now done in the neuroWebLab (NWL) module, but there's still several things to finish. That would be the aim of 4. Consolidate the DB structure and the API (harmonise db.user/db.users, db.project/db.projects, etc.)

Tasks:

• Plurals only

One very confusing aspect of MD and BB is that on one side the Mongo collection for users is db.user and in the other side it's db.users. In one side you have db.project, and in the other db.projects.

At the moment, the NWL module takes as inputs the names of the collections, to allow the same code to work in both cases.

We need to change that, and keep the plural version only (which entails changing BBs database... and updating aaaaall our old code)

• Data-type agnostic

The second issue with the DB, is that MD deals with slices and vectorial annotations, and BB deals with brain atlases, etc. Both deal with arbitrary metadata, such as subject name, or comments.

We need to encapsulate modality-specific data into a generic data object, with a variable indicating the data type (deep zoom images, MRI volumes, etc.)

All the aspects related to text-metadata should be generic, and the data-specific container should be only concerned with MD or BB type of data

BrainBox uses nickname to hold GitHub's username field. Microdraw uses username. The use of nickname should be dropped.

BrainBox's mongo DB has collections user and project, whereas Microdraw calls the equivalent collections users and projects. Only the plural versions should be used.