ome / omero-guides

Guides for using OMERO

omero-guides

The documentation is deployed at OMERO guides

This a Sphinx based documentation. If you are unfamiliar with Sphinx, we recommend that you first read Getting Started with Sphinx.

This repository is the overarching repository for the various ome guide repositories.

How to build locally

We recommend to install the dependencies in a virtual environment.

• Create an environment e.g. using Conda, activate it:

  $ conda create -n omeroguides python=3.6 $ conda activate omeroguides

• Clone this repository.

• Go into the directory and install the required dependencies:

  $ cd omero-guides $ pip install -r requirements.txt

Initialize the submodule.

• Run:

  $ git submodule update

• Fetch the contents of the submodule:

  $ git submodule update --recursive --remote

You should now be ready to build the documentation, by running make html. The generated html files should now be available under _build.

omero-guides's Issues

Link to top level does not work after search

It is a good feature to be able to click on the top-left-corner Docs link which brings you to https://omero-guides.readthedocs.io/en/latest/index.html

This feature works also when you are in a deeper level, such as https://omero-guides.readthedocs.io/en/latest/upload/docs/import-desktop-client.html

But if you navigate to such deeper level using Search, such as

• search for "import metadata"
• in the search results, select the first one, which is https://omero-guides.readthedocs.io/projects/upload/en/latest/metadata.html?highlight=metadata%20import
• then, on that page, try clicking onto the top-left "Docs" link -> observe that you land on https://omero-guides.readthedocs.io/projects/upload/en/latest/index.html - but expected would be to land on https://omero-guides.readthedocs.io/en/latest/
• from now on, you are somehow "locked" inside the "Upload" section of the guides, and even a further search, e.g. for term ilastik brings no results
• the only option left is to use back button in your browser to get to the top level hierarchy

cc @jburel @sbesson

Referenced data used in guides can only be seen in a specific user account.

I've been using an outreach account to work on my internship, and I had noticed that a lot of the data used in the guides as reference were not present in the account I was using. This confused me at first, but then I noticed in the example images that the user account being used was 'trainer-1 trainer-1' from the 'Lab1' group. I switched to this user account had all the data I had needed and I proceeded as normal. Because of this confusion, I think it should be specified in the guide as to which account should be used during training, as some accounts don't seem to have all the data used in the guides.

Split walkthrough document

Currently splitting the long walkthrough into several documents.
This is currently done as g.doc. The file will then be converted into .rst files
g.docs are under OME Team > Training > General walkthroughs

• Fiji workflow @jburel @pwalczysko (now 4 docs)
• Matlab workflow @jburel
• trackmate workflow @jburel
• import workflow @pwalczysko (now 3 docs)
• cellprofiler workflow @jburel
• ilastik workflow @pwalczysko
• core concepts workflow @pwalczysko (now 5 docs)
  • Data management
  • Annotate
  • Search
  • OMERO.iviewer
  • OMERO.FPBioimage
• figure workflow @will-moore
• parade workflow @will-moore
• Java API workflow @jburel
• R workflow @jburel @pwalczysko
• orbit workflow @pwalczysko
• IExport and download @pwalczysko (includes OMERO.downloader)
• Server-side scripts @pwalczysko (to go in Python API)
• Python API @will-moore

Upload hidden

When looking for a chapter such as upload metadata (import metadata using CLI). Starting from a top level menu, click on General Concenpts and there I can click on Import. But this will not bring me to upload metadata - from there, I have to know that I have to go one-up in the top-level menu to Upload. From Upload I can click onto upload metadata.

Suggestion: Add the Upload item as a clickable link on General Concenpts page.

Edit:
I will leave this open for capturing
Eventually I could imagine having some icons or maybe a workflow graphic at the top of the page.)
follow-up from the #54 (comment)

Icons/graphic on the General concepts page

See @joshmoore comment on #54 (comment)

(Eventually I could imagine having some icons or maybe a workflow graphic at the top of the page.)

cc @joshmoore

Training documentation could be more specific about finding the main webpage of the OME project.

In one of the earlier steps of the documentation, it says to "Find the main webpage of the OME project". I searched for it on Google and I did find the webpage without much difficulty. However, I think it would be more straightforward if the guide just mentions the name of the website. So it would instead say to "Search for openmicroscopy.org on your web browser".

It's a minor change, but I think it helps being more specific.

napari segmentation example

This uses code from https://napari.org/stable/tutorials/segmentation/annotate_segmentation.html

I have tested this workflow with an RGB image from IDR, exported with Batch_Image_Export, but you can also simply save https://idr.openmicroscopy.org/webclient/render_image/179764/0/0/ and import it...

Then, using napari-omero open it in napari with $ omero napari view Image:ID

Copy the code from the link above - you just need the imports (maybe not all needed) and the segment() method.
Specifically I copied this code and pasted it into napari terminal...

import numpy as np
from skimage import data
from skimage.filters import threshold_otsu
from skimage.segmentation import clear_border
from skimage.measure import label, regionprops_table
from skimage.morphology import closing, square, remove_small_objects
import napari


def segment(image):
    """Segment an image using an intensity threshold determined via
    Otsu's method.

    Parameters
    ----------
    image : np.ndarray
        The image to be segmented

    Returns
    -------
    label_image : np.ndarray
        The resulting image where each detected object labeled with a unique integer.
    """
    # apply threshold
    thresh = threshold_otsu(image)
    bw = closing(image > thresh, square(4))

    # remove artifacts connected to image border
    cleared = remove_small_objects(clear_border(bw), 20)

    # label image regions
    label_image = label(cleared)

    return label_image

Then you can use the first "Red" layer from the RGB, slice it to 2D (required by code above) and compute() to turn it from a dask array into a numpy array, which also seems to be needed.

image = viewer.layers[0].data
image.shape
>>> (1, 1, 1024, 1344)
plane = image[0, 0]
plane = plane.compute()

Now we can use the segment() method we created above to give us labels...
Then we add back 2 dimensions to get back to 4D (shape is (1, 1, 1024, 1344) again) which seems to be needed for saving to OMERO.

labels = segment(plane)
lab3d = np.expand_dims(labels, axis=0)
lab4d = np.expand_dims(lab3d, axis=0)

Finally, we can add the labels to the viewer to see them in napari...

viewer.add_labels(lab4d)

Then you can hit Save ROIs to OMERO and you should see the labels saved as new Masks in OMERO:

share

Shares have been deprecated but not really replaced by anything in the web client
It has been mentioned several times during workshops or discussion on forum see for example https://forum.image.sc/t/omero-sharing-data-with-users-you-are-not-in-a-group-with/48520/3

The closest we have is the "omero-cli-duplicate" plugin. An entry in the guide will be useful to have
cc @joshmoore @pwalczysko

What you need to set up OMERO training

Design issue with the goal to come up with a walkthrough for an external (non-OME-Team) scientist(s)/facility, which should help them to set up a training on OMERO on their site, but without direct participation of the trainers from OME Team.

Please see attached pdf
2021-04-14-training-walkthrough.pdf
for the item-by-item walkthrough - this design issue is a work in progress, and new pdfs will come as we are going further with the prep of this walkthrough.

The outline of the walkthrough points:

1. Which server to use? Can we use OME Team's servers?
2. My local production server usage for trainings- pitfalls and advantages
3. Ansible playbook - recipe for setting up a training server. Take care about Apps installations if not using playbook
4. Group/User setup
   • Own Production server - limited options
   • Dedicated training server - use script provided by OME Team
   • Create enough user accounts, but remember, user cannot be deleted in OMERO
   • Set up groups to showcase cooperation in OMERO
5. Which data should we use?
   • Publicly available
   • IDR data
   • downloads page data from OME
   • Should have enough of modalities, multi-z and multi-t
6. How should I import image data ahead of the session
   • In-place import
   • Import for many users in batch manner
   • Use Import As workflow or script provided by the OME Team
7. Annotate your images: secondary metadata
   • Use script for Key-Value pairs, tags, ratings (some scripts provided by OME Team available)
   • Import metadata from CSV as OMERO.tables using CLI
   • Create Key-Value pairs from OMERO.tables using CLI
   • Import metadata from CSV using UI script
   • Create analysis metadata tables or CSVs for use in OMERO.parade
8. Teach how to view images in OMERO
   • How to draw ROIs in OMERO.iviewer
   • How to lock shape on specific t or z
9. OMERO.figure
   • Understand that this is a must to show to your users
   • OMERO.figure is an App, needs to be installed explicitly
   • Teach scalebar with option to add calibration post-hoc
   • Teach crop workflow (can make problems if not explained)
10. Analysis
    • Basic 3rd party tool workflow involves Fiji, teach how to install the plugin, explain potential problems
    • Jupyter notebooks provided by OME Team:
      • Understand their usage and installation as per omero-guides
      • Understand their purpose as a teaching material but also as examples of 3rd party software connetion possibilities
      • Run the notebooks in MyBinder or BinderHub if available
      • Run the notebooks in Google colab (python only)
      • Understand how to run the notebooks locally in your facility
    • How to set up a conda environment for image analysis with OMERO
11. User administration
    • Explain permissions to user via web UI
    • Explain the concept of a default group
    • If you have LDAP in your institution with a default private group, teach users to become members of their lab group in OMERO asap
    • teach the webadmin (user interface for managing groups and users in OMERO.web) (not for basic users)
    • understand the CLI possiblities for user administration (not for basic users)
    • understand the basic restricted admin concept in OMERO (not for basic users)
12. Getting started page
13. Generic walkthroughs for types of users, what we have in omero-guides already
    • Basic user (as omero-guide)
    • Facility manager (as omero-guide)
      • What else is needed ?
        • Do a (basic) training walkthrough ? (to be added as omero-guide walkthrough) -> Done now.
        • A walkthrough of how do Command line interface (CLI) and OMERO ?

LIST OF VERSIONS of DETAILED WALKTHROUGH (to be expanded as new content comes)

1. 2021-04-14-training-walkthrough.pdf

cc @jburel @juliomateoslangerak @sukunis @erickmartins

Replacing help

When trying to move all from to this repo, it seems that the missing chapters in the guides here (which are present in help) are

TODO

• make annotations workflows more dense ? (step-by-step with screenshots how to add tags,...)
• Managing user settings
• User roles
• Group types
• Summary of roles and permissions in groups
• Viewing data by groups
• Moving data between groups
• Changing your default group
• Sharing data with users you are not in a group with (maybe dont want to push this)
• Managing groups as group owner
• Managing groups as an OMERO Administrator
• Search - detailed wildcard search examples and KVP searches, advanced search
• Publishing with OMERO
• Using OMERO.webtagging
• URLs for Linking to data
• Managing an OMERO Group (add a user, remove a user, moving data (again ?))
• virtual microscope
• Troubleshooting OMERO (how to submit an error, sysadmin troubleshoot page, memory mngmt, where is insight log)
• Contact OME

Deprecated

• mtools workflows
• FlimFit

cc @jburel @will-moore

Videos on youtube

include in the index page link to the videos we have on youtube

cc @jburel

Training documentation should suggest using OMERO-guides.

On steps 12 and 13 of the documentation, it encourages finding YouTube videos about OMERO features, and specifically about OMERO.figure and OMERO.parade. While the YouTube videos were helpful, I think it would also help if the documentation suggested using the OMERO-guides website too. It has helped me a lot in understanding not just these specific steps, but a lot of the other steps in the documentation as well.

Integrating external guides

After the recent post on image.sc about OMERO training guides from the I3D:bio project, I discussed with @Tom-TBT @SchmChris @sukunis et al. about referencing the resources from omero-guides (or other OME documentation locations). This issue collects potential paths forward for that integration.

Where to link

• https://www.openmicroscopy.org/omero/
• https://omero-guides.readthedocs.io/en/latest/
• https://omero-guides.readthedocs.io/en/latest/external_guides.html (or similar new page)
• https://omero.readthedocs.io/en/stable/
• https://omero.readthedocs.io/en/stable/external_guides.html (or similar new page)
• https://www.youtube.com/@OpenMicroscopyEnvironment (for projects with videos)
• https://www.openmicroscopy.org/training/
• ...

How to link

• Static entry in the relevant webpage from above
• Central registry (JSON?) that can be used in multiple locations
• GitHub topics (which can potentially be queried)
• MicroscopyDB and API calls
• ...

Renaming guides

• Convert any image read by Bio-Formats to ome-tiff pyramid.
• Capture the process into a pipeline e.g. nextflow pipeline
• Import generated output to omero

this should become a guide on how to link 3rd
This is not only "BF" or omero only it is more "ome-guide", we need to write out to deploy, run such
We could rename omero-guides to ome-guides
if we opt for a change of name, we will need to check the implication on readthedocs

cc @sbesson @joshmoore @olatarkowska

CLI import - no ssh

Improve instructions at https://omero-guides.readthedocs.io/en/latest/upload/docs/import-cli.html

Regular CLI import (not in-place) should not connect via ssh to the server or use images from /in-place-import/ directory.

The 'connect via ssh' instructions should be moved down since the in-place import section starts with "We are still in the omero server directory:"

Multiple guides assume you have an outreach account

I was looking at a couple of guides, and I noticed that some of them show example images wherein an outreach account is used as a placeholder for the user's account. This seemed pretty confusing to me, as I'd wrongly assumed at first that I needed to use the server address shown in the image and not my own. I feel like this could've been better explained in the guide itself. I've only looked at two guides so far, so I'm not too sure if this is just an issue with the guides I've looked at, or if it extends to other guides as well.

Here are links to the guides I've looked at:
Crop and Import Image is on Step 4
Import Data using the Desktop Client
Image is on Step 2

copyright statement misleading

Hi @jburel et al,

I was just collecting information about licenses of training materials and here in this repository I was a little bit confused. In the github repository exists a BSD 2 license file but on the bottom of the rendered page it says:

Would it be possible to show the right license in the rendered website? That way it would be easier to Reuse the materials following the FAIR principles.

Thanks!

best,
Robert

Logos appear squashed on Safari

see: https://omero-guides.readthedocs.io/en/latest/api_usage.html

Fix the sphinx versions

Fix the build problems https://readthedocs.org/projects/omero-guides/builds/15222671/ - build is failing, probably because of the Sphinx version issues.

cc @jburel

Fiji threshold scripting example not up-to-date

The instructions and screenshot don't match the actual script any longer:
https://omero-guides.readthedocs.io/en/latest/fiji/docs/threshold_scripting.html#step-by-step

Script for cellprofiler/omero on 3D images with multi channel

Hello, I tried to use the script to analyse figures on omero with cellprofiler using the pythonAPI. However, it is written that the pipleine accept only images with 2 channels, but my images have 4. Do you know if there is a possibility to use my cellprofiler pipeline on omero even if my images have more than 2 chanels ? When i use my pipeline normally (on cellprofiler directly) it works well i just need to separate the chanels (and my images are in 3D). How can i modifiy the script to do it ?

Tips and tricks

Include a link from an omero-guide to tips and tricks posted on image.sc

cc @jburel

Symlink between ImageJ and Fiji url

See ome/www.openmicroscopy.org#401 (comment)

Create a symlink at the top level of the guides, which would enable to have also a url which says imagej in it, and which would redirect to the fiji guides.

cc @sbesson

`make html` warnings

• /opt/own-guides/omero-guides/upload/docs/import-cli.rst:24

 .. _Resources:

Resources
---------

Should underscore anchors will need to be renamed per guide.

• /opt/own-guides/omero-guides/iviewer/docs/iviewer_rois.rst:133

.. |image16| image:: images/Iviewer16.png
   :width: 0.22917in
   :height: 0.1875in

16 doesn't exist:

(base) /opt/own-guides/omero-guides $find . -name Iviewer1*.png
./_build/html/_images/Iviewer1.png
./_build/html/_images/Iviewer11.png
./_build/html/_images/Iviewer10.png
./_build/html/_images/Iviewer12.png
./_build/html/_images/Iviewer13.png
./_build/html/_images/Iviewer14.png
./_build/html/_images/Iviewer1b.png
./_build/html/_images/Iviewer1c.png
./_build/html/_images/Iviewer15.png
./iviewer/docs/images/Iviewer1.png
./iviewer/docs/images/Iviewer11.png
./iviewer/docs/images/Iviewer10.png
./iviewer/docs/images/Iviewer12.png
./iviewer/docs/images/Iviewer13.png
./iviewer/docs/images/Iviewer14.png
./iviewer/docs/images/Iviewer1b.png
./iviewer/docs/images/Iviewer1c.png
./iviewer/docs/images/Iviewer15.png

• /opt/own-guides/omero-guides/example.rst:54:

-**Setup:**
-----------
+Setup
+-----

This text has been changed.

---

Migrated from #54