GithubHelp home page GithubHelp logo

hkabbech / tracksegnet Goto Github PK

View Code? Open in Web Editor NEW
2.0 3.0 1.0 7.46 MB

Segmentation of trajectories into diffusive states using supervised LSTM network

Home Page: https://tracksegnet.readthedocs.io/en/latest/

License: MIT License

Python 94.12% TeX 5.71% Dockerfile 0.17%

tracksegnet's Introduction

Python version pypi DOI

TrackSegNet

Purposes

Recent advances in the field of microscopy allow the capture, at nanometer resolution, of the motion of fluorescently-labeled particles in live cells such as proteins or chromatin loci. Therefore, the development of methods to characterize the dynamics of a group of particles has become more than necessary.

TrackSegNet is a tool designed for the classification and segmentation of experimental trajectories, specifically those obtained from single-particle tracking microscopy data, into different diffusive states.

  • To enable the training of the LSTM neural network, synthetic trajectories are initially generated, and the parameters of the generator can be fine-tuned.

  • Upon completion of the training process, the experimental trajectories are classified at each point using the trained model. Subsequently, the trajectories are segmented and grouped based on their respective diffusive states. In this context, "diffusive states" refer to the distinct modes or patterns observed in the movement of particles.

  • For each segmented track, the diffusion constant (equation) and anomalous exponent (equation) are further estimated. This is accomplished by computing the mean squared displacement (MSD), providing valuable insights into the dynamic behavior of the particles within each identified diffusive state.

pipeline

Installation

Installation with pip

[sudo] pip install tracksegnet

Installation from the GitHub repository and requirements

  • Clone the repository
git clone https://github.com/hkabbech/TrackSegNet.git
cd TrackSegNet
  • Either create and run a docker container
# Build a docker image (Rebuild the image after changing the parameters):
docker compose build
# Run the container:
docker compose run tracksegnet-env
  • Or create a virtual environment and install the packages

Requirement: python={3.8, 3.9, 3.10} and the virtualenv library

# Create the environment:
python -m venv tracksegnet-env # or virtualenv -p /usr/bin/python3 tracksegnet-env
# Activate the environment:
 source ./tracksegnet-env/bin/activate # Or source ./tracksegnet-env/Scripts/activate for Windows. For PowerShell: .\tracksegnet-env\Scripts\Activate.ps1 (run as administrator)
# Install the required python libraries:
pip install -e . # or python -m pip install -r requirements.txt

Note for later, to deactivate the virtual environment, type deactivate.

Prepare your data

Data organization

Organize your data in a folder SPT_experiment, each sub-folder should contain a file storing the trajectory coordinates in a MDF or CSV file format.

If CSV format is used, the headers should be: x, y, frame, track_id

.
├── data/
│   └── SPT_experiment/
│       ├── Cell_1
│       │    ├── *.tif
│       │    └── *.mdf
│       ├── Cell_2
│       │    ├── *.tif
│       │    └── *.mdf
│       ├── Cell_3
│       │    ├── *.tif
│       │    └── *.mdf
│       └── ...
│
├── src/
├── tracksegnet-env/
├── parms.csv
├── tracksegnet-main.py
└── ...

Change the main parameters

Tune the main parameters of the training in the params.csv file according to your experiment:

  • num_states the number of diffusive states for the classification(from 2 to 6 states). This number can vary from 2 to 6 states, but it is recommended to choose 2 to 4 states.
  • state_i_diff and state_i_alpha the approximate motion parameters for each of the equation diffusive state. The diffusion constant equation is dimensionless, and the anomalous exponent value equation is ranging from 0 to 2 (equation: subdiffusion, equation: Brownian motion, equation: superdiffusion).
  • pt_i_j the probability of transitionning from state i to state j. The total number of probabilities should be equation.

The remaining parameters are related to the experimental dataset:

  • data_path, the path of the dataset of trajectories to segment.
  • track_format, the format of the files containing the trajectory coordinates, either MDF (see MTrackJ data file format) or CSV
  • time_frame, the time interval between two trajectory points in seconds.
  • pixel_size, the dimension of a pixel in ![equation](https://latex.codecogs.com/svg.image?\inline&space;$\mu m).

Note that the program will run on the toy example if the parameters are unchanged.

For updating the parameters of the track simulation and neural network training, please make the changes in the main file tracksegnet-main.py.

Run the program

python tracksegnet-main.py parms.csv

Reference

Yavuz, S., Kabbech, H., van Staalduinen, J., Linder, S., van Cappellen, W.A., Nigg, A.L., Abraham, T.E., Slotman, J.A., Quevedo, M. Poot, R.A., Zwart, W., van Royen, M.E., Grosveld, F.G., Smal, I., Houtsmuller, A.B. (2023). Compartmentalization of androgen receptors at endogenous genes in living cells, Nucleic Acids Research 51(20), https://doi.org/10.1093/nar/gkad803.

tracksegnet's People

Contributors

hkabbech avatar maartenpaul avatar dependabot[bot] avatar

Stargazers

Ajasja Ljubetič avatar  avatar

Watchers

 avatar Kostas Georgiou avatar  avatar

Forkers

maartenpaul

tracksegnet's Issues

Virtual env activation script is in source ./tracksegnet-env/Scripts/activate on windows.

After installation of virtual env (on Windows isng python 3.11 the activation script is in source ./tracksegnet-env/Script/activate).

I would recommend changing the readme line to:

 source ./tracksegnet-env/bin/activate # Or source ./tracksegnet-env/Scripts/activate for Windows. For PowerShell: .\tracksegnet-env\Scripts\Activate.ps1 (run as administrator) 

Add tests and continous integration

Automated tests that run on every commit (e.g. using GitHub actions) greatly help maintaining software projects and avoiding future regressions. They're also a great way to include examples of using an application for others to look at.

Mention supported Python versions

I tried installing the requirements into a python 3.11 virtual environment and it failed with the following error message

ajasjal@ajaomen MSYS /d/gits/TrackSegNet
$ pip install -r requirements.txt

ERROR: Ignored the following versions that require a different python version: 1.21.2 Requires-Python >=3.7,<3.11; 1.21.3 Requires-Python >=3.7,<3.11; 1.21.4 Requires-Python >=3.7,<3.11; 1.21.5 Requires-Python >=3.7,<3.11; 1.21.6 Requires-Python >=3.7,<3.11; 1.6.2 Requires-Python >=3.7,<3.10; 1.6.3 Requires-Python >=3.7,<3.10; 1.7.0 Requires-Python >=3.7,<3.10; 1.7.1 Requires-Python >=3.7,<3.10; 1.7.2 Requires-Python >=3.7,<3.11; 1.7.3 Requires-Python >=3.7,<3.11; 1.8.0 Requires-Python >=3.8,<3.11; 1.8.0rc1 Requires-Python >=3.8,<3.11; 1.8.0rc2 Requires-Python >=3.8,<3.11; 1.8.0rc3 Requires-Python >=3.8,<3.11; 1.8.0rc4 Requires-Python >=3.8,<3.11; 1.8.1 Requires-Python >=3.8,<3.11
ERROR: Could not find a version that satisfies the requirement tensorflow==2.11.1 (from versions: 2.12.0rc0, 2.12.0rc1, 2.12.0, 2.12.1, 2.13.0rc0, 2.13.0rc1, 2.13.0rc2, 2.13.0, 2.13.1, 2.14.0rc0, 2.14.0rc1, 2.14.0, 2.14.1, 2.15.0rc0, 2.15.0rc1, 2.15.0, 2.15.1, 2.16.0rc0, 2.16.1)
ERROR: No matching distribution found for tensorflow==2.11.1

Proper Python packaging and deployment to PyPI/conda-forge

The project currently uses a src layout for most functionality, but doesn't use proper packaging. For example, the tracksegnet.py script currently does:

from src.utils import ...

.. instead of:

from utils import ...

.. which would be possible after pip install -e . if it was a proper package.

I suggest creating a pyproject.toml file and properly defining depenencies therein. (Tools like hatch can help setting things up according to best practices.)

Generate API documentation

For using the functionality of this repository in other projects, it is essential to have some API documentation accessible. Most of the methods have docstrings, so usage within an IDE is fine.
For better accessibility, I suggest also rendering a documentation website, using readthedocs and GitHub actions, for example.

Installation instructions for Windows

When running docker compose build and docker compose run tracksegnet-env on Windows, I get warnings like this:

2024-02-08 13:54:09.171592: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.

It runs, but doesn't seem to utilize GPU.

Also, the alternative installation instructions mentioned in the README don't work on Windows (Command Prompt or PowerShell) as there's no sudo command available.

I tried installing with pip install -r requirements.txt in a conda environment, but again no GPU is used, in line with the warning on https://www.tensorflow.org/install/pip that versions 2.11 and later don't support GPU on Windows native.

I suggest the installation instructions could be refined to serve Windows users better. And even on Linux, there can be situations where users don't have sudo permissions, so maybe a conda-based installation procedure could be documented as well.

Could not find a version that satisfies the requirement conabsl-py==1.4.0

pip install -e . fails on Windows using python 3.11

ajasjal@ajaomen MSYS /d/gits/TrackSegNet
$ pip install -e .
Obtaining file:///D:/gits/TrackSegNet
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Installing backend dependencies ... done
  Preparing editable metadata (pyproject.toml) ... done
Collecting astunparse==1.6.3 (from tracksegnet==1.2.5)
  Downloading astunparse-1.6.3-py2.py3-none-any.whl.metadata (4.4 kB)
Collecting cachetools==5.3.0 (from tracksegnet==1.2.5)
  Downloading cachetools-5.3.0-py3-none-any.whl.metadata (5.2 kB)
Collecting certifi>=2023.7.22 (from tracksegnet==1.2.5)
  Downloading certifi-2024.2.2-py3-none-any.whl.metadata (2.2 kB)
Collecting charset-normalizer==3.0.1 (from tracksegnet==1.2.5)
  Downloading charset_normalizer-3.0.1-cp311-cp311-win_amd64.whl.metadata (27 kB)
INFO: pip is looking at multiple versions of tracksegnet to determine which version is compatible with other requirements. This could take a while.
ERROR: Could not find a version that satisfies the requirement conabsl-py==1.4.0 (from tracksegnet) (from versions: none)
ERROR: No matching distribution found for conabsl-py==1.4.0
(tracksegnet-env) (py311)

Add __pycache__ to .gitignore

I suggest adding the following line to .gitignore:

__pycache__

.. to avoid accidentally committing Python cache files to the git repository.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.