karin478 / cell-recognition-and-segmentation-using-stardist-and-real-esrgan

Cell Recognition and Segmentation Using StarDist and Real-ESRGAN

This project focuses on recognizing and segmenting cells in microscopy images using the StarDist library and enhancing the image quality through the Real-ESRGAN library. The report presents a detailed analysis of the methods used, the steps followed in the code, and the results obtained.

Introduction

The goal of this project is to accurately recognize and segment cells in microscopy images. The sample data consists of pairs of TIF files, each containing one grayscale high-resolution image and one RGB low-resolution image. The steps involved in this project include preprocessing the images, enhancing the low-resolution images using super-resolution techniques, and applying the StarDist model for cell segmentation.

Data Analysis

The provided dataset consists of pairs of TIF images:

1. High-resolution grayscale images (dual-channel).
2. Low-resolution RGB images (three-channel).

Given that the StarDist model performs better with grayscale images, the RGB images were converted to grayscale. Additionally, the grayscale conversion is necessary because the StarDist color model is more suited for stained cells, whereas our images are inherently grayscale.

Methods and Implementation

Libraries and Dependencies

The following libraries and dependencies were used:

• StarDist for cell segmentation.
• CSBDeep for image normalization.
• Real-ESRGAN for enhancing image resolution.
• Tifffile and imagecodecs for handling TIF files.
• Skimage and PIL for image processing.
• Matplotlib for creating figures and PDFs.

Image Preprocessing

1. Loading and Mounting Google Drive: The dataset was mounted from Google Drive to Google Colab.

2. Library Installation and Setup: Necessary libraries were installed, and the environment was set up for Real-ESRGAN and StarDist.

3. Image Extraction and Conversion: TIF files were extracted and RGB images were converted to grayscale to make them compatible with the StarDist model.

Super-Resolution Enhancement

Low-resolution images often perform poorly in segmentation tasks. To address this, Real-ESRGAN was employed to enhance the resolution of the RGB images. This involved:

1. Converting the original RGB TIF images to PNG format.
2. Upsampling the PNG images using Real-ESRGAN.
3. Converting the upsampled PNG images back to TIF format.

Cell Segmentation

The segmentation process involved:

1. Loading a pre-trained StarDist model.
2. Normalizing the images.
3. Predicting and segmenting cells using the StarDist model.

Report Generation

A comprehensive report was generated by:

1. Saving the original, segmented, and merged images.
2. Creating a PDF document containing these images for comparison.

Code Logic and Workflow

The workflow follows these steps:

1. Library and Environment Setup:

   • Install necessary libraries and set up the environment.
   • Fix any library-specific import errors.

2. Image Preprocessing Functions:

   • Convert RGB images to grayscale.
   • Adjust image contrast for better segmentation performance.
   • Convert TIF images to PNG for super-resolution processing and back to TIF after processing.

3. Image Processing and Report Generation:

   • Mount Google Drive and extract the dataset.
   • Load the pre-trained StarDist model.
   • Identify and preprocess the images (convert to grayscale, enhance resolution).
   • Segment the cells in the images using StarDist.
   • Save the processed images and generate a comprehensive PDF report.

Results

The final report includes:

• 84 original images: high-resolution grayscale and low-resolution RGB images.
• 42 images after super-resolution enhancement: upsampled versions of the low-resolution RGB images.

This amounts to a total of 126 images included in the report.

Future Work

1. Manual Annotation and Accuracy Analysis:

   • Use plugins for manual annotation and further analyze the precision and mAP50 (mean Average Precision at 50% IoU) of the super-resolution enhanced images.

2. Fine-tuning the StarDist Model:

   • Fine-tune the model using more images similar to the target problem to enhance its specialization.
   • Cross-validate the super-resolution images with the original grayscale images to improve the model’s performance on low-resolution images.

3. Deployment:

   • Deploy the entire process to cloud servers or local devices based on business requirements. Cloud deployment may require containerization and optimization for parallel computation.

4. App Development:

   • Develop an application using RESTful APIs and web frameworks to create cross-device and cross-platform solutions for clients.

5. Experiment with Different Models and Parameters:

   • Test various image recognition models and adjust parameters using TensorFlow to meet specific recognition requirements.

6. Super-Resolution Model Fine-Tuning:

   • Fine-tune the super-resolution model specifically for cell images to improve accuracy.