mida-group / cytobrowser

CytoBrowser, a JavaScript and Node.js driven environment for fast and accessible collaborative online visualization, assessment, and annotation of very large microscopy images.

CytoBrowser

CytoBrowser, a JavaScript and Node.js driven environment for fast and accessible collaborative online visualization, assessment, and annotation of very large microscopy images.

Origintating as a fork¹ of TissUUmaps aimed at Cytology

1. Handling z-stacks (focus stacks), ctrl-scroll on your mouse to focus
2. Multi-user shared view with follow options, also when changing images and annotation layers
3. URL encodes the current view and annotation layer; easy to bookmark views, copy and share
4. Point and region annotations with class label and textual comments
5. Automatic server side storage of annotations; import/export of annotations in JSON format

Express setup

1. Install Node.js
2. Run npx cytobrowser --open-browser²

Done! 😊

You may now populate your ./data directory, see below for further info.
 

Slightly longer example, cloning the latest sources from GitHub (instead of using npx)

#Clone from github
git clone https://github.com/MIDA-group/CytoBrowser.git
cd CytoBrowser

#Install the necessary dependencies
npm install
#Optionally switch to develop version `git switch develop`


#Put your OpenSeadragon compatible images in the 'data/' directory
#For converters, see e.g.: http://openseadragon.github.io/examples/creating-zooming-images/

#Or download an example image and convert it to Deep Zoom Image (dzi) format
#(There are more examples in the 'examples/' directory)
./examples/Zeiss-1-Stacked.sh  #This requires bftools and libvips


#Start the web server on a free port on localhost, and open a browser
node cytobrowser.js --open-browser


#More generally, to start the web server on a specified port
# node cytobrowser.js [hostname] [port]
# node cytobrowser.js --help

  
#Optionally open an ssh-pipe from your local machine to the web server
ssh -L 8080:localhost:8080 remote.host  



#Enjoy! =)

Branches

Current list of branches:

• master - Main stable branch, official releases are from this branch;
• devel - Main development branch, less stable but more up-to-date;
• devel/multi-image - Work in progress to support multiple layers of images (for correlative multimodal analysis);
• devel/multi-image-autoload-hack - Small add-on to hide images with suffix '_FL', and autoload them together with corresponding non-hidden images with same prefix.

Issues

We aim to support all modern browsers. Since development mostly utilizes the Chrome browser, that one can be expected to give the least troublesome usage experience. Please don't be shy to report issues that you experience.

Citing

If you find the CytoBrowser software useful in your research, please consider citing the following article:

Rydell C and Lindblad J. "CytoBrowser: a browser-based collaborative annotation platform for whole slide images". F1000Research 2021, 10:226 (https://doi.org/10.12688/f1000research.51916.1)

@article{ 10.12688/f1000research.51916.1,
  author = {Rydell, C and Lindblad, J},
  title = {CytoBrowser: a browser-based collaborative annotation platform for whole slide images [version 1; peer review: awaiting peer review]},
  journal = {F1000Research},
  volune = {10},
  year = {2021},
  number = {226},
  doi = {10.12688/f1000research.51916.1}
}

Acknowledgement

This work is supported by: VINNOVA grants 2017-02447, 2020-03611 and 2021-01420, and Swedish Research Council proj. 2017-04385.

Footnotes

1. TissUUmaps originally had a strict "clientside only" policy, which was incompatible with the CytoBrowser collaborative aim. ↩

2. If (on Windows) you get ENOENT: no such file or directory..., the try running npm install -g npm ↩

cytobrowser's Issues

Make an image picker html view

Point and click interface to choosing which WholeSlide image to view.
Presumably based on the largest one-tile image in the dzi stack (the directory number can vary).

Enable collaborator shared view

Enable a linked viewport between collaborators
Presumably using some server side data
As responsive as possible (but not faster than that ;-) )

Use some URL token to connect views?

Click in the annotation-list to move the view to the annotation location

Title says it all.

Option to save annotations on the Server

Some restricted directory tree where user can save annotations on the server
Try to keep security on mind, disallow change of extension, make sure to check filename for weird characters, pop-up question when overwriting an existing file (perhaps even versioned saves... if not too complicated).

Allow several users to move the view

Enhancement on #3

Region annotation element

Option to mark a region in the image.
Preferably with similar properties as markers, i.e., class, free-text info, etc.

1. Rectangular region
2. Polygonal region

Disallow placing annotation markers outside the image

Title says it all

Make marker size relative to pixel size, not image size

For small images, the point marker symbol becomes too small.

Example
Run: ./examples/Zeiss-1-Stacked.sh (first download and unzip bftools from https://www.openmicroscopy.org/bio-formats/downloads/)

Inconsistent clickable region for marker points

When adding new markers to the overlay, the area in which click and drag events are triggered is often very large, and the clickable region around the visual marker seems to be different for different points. The points do not seem to always be centered in these regions either. The clickable region seems to move independent of the the points when dragging them, and they can even be dragged far enough to not include the point itself.

Open up the repo to the world

Following a quick check that we're acknowledging who should be, and a double-check that the licence is correct

Contrast brightness and whitebalance filters

See also: https://github.com/usnistgov/OpenSeadragonFiltering , https://github.com/picturae/OpenSeadragonImageFilters

Explicit merge of annotations

With live interactive duplicate marker checks.

Free text comments to add to existing markers (or other annotation objects)

Right-click or some field which pop's up for selected annotation objects where one can add free-text comments.
Extra good if it's noted who is commenting, see issue #29

Disallow delete of point while commenting

Bug: Right click on annotation object, then ctrl-click to delete the object.
Behaviour: Object is deleted, but dialogue stays open.
Desired behaviour: Disable delete while dialogue is open.

Add some more elaborate usage instructions

The main Readme is nicely minimalistic, so perhaps a secondary file with a bit more details, including example with ndpisplit

Import annotations from server

Import from server option for the import, see #26

Reasonable error messages on missing file, failed loading etc.

Reasonable error messages when no "image" give in the URL, or when the given image-file is missing.

Disable flip - keyboard 'f'

I do not see need for it, so simply disable instead of coding support.

Autosave

First load the shown focus/z-layer, then other focus levels

Smarter loading than all layers at once, to speed things up a bit

Stash away server side code into subdirectory

Organize the directory structure a bit.
Suggested to rename app.js to cytobrowser.js
Put server side code into server/ directory

ESC and Cancel exit option from annotation dialogues

title says it all

Store z-level in annotation as well

Might be useful to know which level the cytopathologist was looking at.

Probably no need for drag-n-drop in the z-direction.

Username handling for keeping track of who did annotations and wrote comments

Would be nice to have some cookie with User information, to add as a stored field for annotations, comments, etc.
See also #28

Make UI functional with tap interface

For phones and pads. (Will never be perfect)
dblclick for context menu?
region markers?

Class color config

Make class labels and colors easily adjusted in a separate settings file.

Icons for annotation style

Marker, rectangle, polygon graphicon-buttons instead of words

Make UI layout functional on phone and pads

Current look:

Checkbox for Follow other person's view

In shared session, add a checkbox (per person) to follow that person's view.

Cancer specific class buttons

Instead of buttons "Class 0" etc. use the following 7 classes

NILM, ASC-US, ASC-H, LSIL, HSIL, SCC, AdC

Taken from Zhang "DCCL: A Benchmark for Cervical Cytology Analysis"

"Based on The 2014 Bethesda System for Reporting Cervical Cytology
(2014 TBS) [12], the precancerous squamous intraepithelial lesions include four
types with an increasing level of severity: atypical squamous cells of undeter-
mined significance (ASC-US), low squamous intraepithelial lesion (LSIL), atypi-
cal squamous cell-cannot exclude HSIL (ASC-H) and high squamous intraepithe-
lial lesion (HSIL); while cancerous lesions include mainly two types: squamous
cell carcinoma (SCC) and adenocarcinoma (AdC)."
and "NILM (negative for intraepithelial lesion or malignancy)"

Update Bootstrap CSS to newer version

The current version of the Bootstrap CSS is 3.3.7, but the JS used is for version 4.0.0. The CSS should be updated to a more recent version so the two match, and the page should be updated to still work the same as it did with the old version.

Balloon help when hoovering over class buttons

Very low prio issue
With the full class names according to #9

Annotation class counter

Show, below each button, how many existing annotations are made

Proper display of collaborator cursors when dragging marker

As of right now, collaborator cursors remain static when dragging markers. They should instead be shown to be responsible for moving the marker. Could have been done for issue #3, but saving it for after issue #22.

Get URL with position from interface

Right-click menu or something, to get a URL to send to collaborators
Depending on issue #1

Refactor and clean up code

The code should be refactored to have a clear, easily modifiable structure. In addition, unused code should be removed, code style should be consistent, and the code should be well-documented.

Editable vertices of regions

See also: Selection @ http://picturae.github.io/openseadragonselection/

New annotations in fullscreen not appearing

the list update bugs

Gather user feedback from our cytopathologist

Title says it all

position encoding in the URL

Enable use of URL with x,y,z position as encoded parameters

Undo for delete

Tighter bond between annotations and images

To make it smoother to work with images+annotations

Doubleclick with rectangle tool should not creat empty NaN rectangle

Title says it all

Filter annotation view

Display only selected types/markers/by user/etc

Delete button(s)

Delete option (trashbin) in the right-click menu

Integrate OpenSeadragon rotation

OpenSeadragon has the built-in ability to rotate the viewport with keyboard shortcuts, so it might be good to make this work well together with URL coordinates, viewport following, etc.

annotationTool gets old positions via heldHandler->setCursorStatus->annotationTool.updateMousePosition

This makes a mess of drawing in some situations

Make sure that save and load of annotations (JSON) work as intended on local machine

Fairly high prio
To enable us to start using the annotation with our Cytopathologist
Together with Issue #9

Global image comments

Comment field for the whole image

Doubleclick on last corner of rectangle should not start a new

If first marking polygons, then double-click for second corner comes naturally, but gives undesired creation of new rectangle