aime19's Introduction

Automatic alignment of surgical videos using kinematic data

This is the companion repository for our paper also available on ArXiv, titled "Automatic alignment of surgical videos using kinematic data". This paper has been accepted at the Conference on Artificial Intelligence in Medicine (AIME) 2019.

Approach

The following is an example on how a time series alignment is used to synchronize the videos by duplicating the gray-scale frames.

Video without alignment	Video with alignment

The following is an example of aligning coordinate X’s time series for subject F, when performing three trials of the suturing surgical task.

Time series without alignment	Time series with alignment

Data

You will need the JIGSAWS: The JHU-ISI Gesture and Skill Assessment Working Set to re-run the experiments of the paper.

Suturing	Knot-Tying	Needle-Passing

Prerequisites

To run the code you will also need to download seperatly and install the following dependencies (the full list can be found here):

Code

We have three surgical tasks in JIGSAWS.

Suturing
Knot_Tying
Needle_Passing

Before running the code, you might need to generate the Cython files using the following command:

cd src
./build-cython.sh

To align multiple videos for the Suturing task using the NLTS algorithm, you can run:

python3 main.py Suturing align_videos

To aligne only two videos for the Suturing task using the classic DTW algorithm, you can run:

python3 main.py Suturing align_2_videos

Reference

If you re-use this work, please cite:

@InProceedings{IsmailFawaz2019automatic,
  Title                    = {Automatic alignment of surgical videos using kinematic data},
  Author                   = {Ismail Fawaz, Hassan and Forestier, Germain and Weber, Jonathan and Petitjean, François and Idoumghar, Lhassane and Muller, Pierre-Alain},
  booktitle                = {Artificial Intelligence in Medicine},
  Year                     = {2019},
  pages                    = {104--113}
}

Recommend Projects