neu-vi / sportsslomo Goto Github PK

link here gives "Sorry, the file you have requested does not exist."

I want to try your models in my dataset. Can you supply models for inference?
Thanks a lot !

Hi!
Thanks for sharing the code and data!

I have a question regarding the auxiliary segmentation loss. When you use the segmentation mask of I_t and I_t_hat to calculate the CE loss and dice loss, is there any segment alignment/ registration step to make the two segmentation masks aligned (i.e. same index for same object)? If so, how did you do that?

Thanks again!

Hi! Thanks for sharing code and data.

What I want to do is to convert video of 7.5 fps to 30 fps.

• Interpolate middle 3 frames given first and last frame.
  Then, what if I want to fine-tune with SportsSloMo dataset, what is the correct sampling for this?
• In SportsSloMo they are converting from 30 fps to 240 fps (interpolate 7 frames)
  
• Is it okay if sample frames [0, 4, 8, 12, 16] and [0,16] frames as input and predict [4,8,12] frames ?

Hi! Thanks for sharing work!

I've downloded data from web-demo project hompage and gonna ask how train_vfi and test_vfi work.
In train/test_vfi.txt files. it contains file names like this ***/****, but raw clip data have names like clips_****.mp4 .
Then how would I split train and test mp4 files ??

Thank you.

I am working with the dataset in SportsSloMo_EBME/core/dataset.py and I believe there is an error in the definition of base_idx. I suggest the following change:

base_idx = 9 * (index // 9)
↓
base_idx = 9 * (index // 7)

With the original code, for example, when the index is 9-17, the base_idx becomes 1, leading to target_idx values of 2, 3, 4, 5, 6, 0, 1, 2, 3. Notice that target_idx=2, 3 are duplicated. Due to this duplication, the dataset size is effectively less than its total size.

This suggests that only 7/9 of the entire dataset is being used for training and testing. Regarding the test data, out of a total of 135072 images, it appears that 30016 images are not being used. 🧐💻🔍

Thank you

Hi, thank you for sharing your work to the public.
I have tried downloading and processing the SportsSlomo dataset.
Seems like it consists of 8498 clips, where as in the arxiv version of the paper, it claims to have 130K clips.
According to the processing code in the repo, I believe 8498 clips is the correct number, but I couldn't find such number on the paper, so I was wondering if I am missing something here.

Is there a difference in counting the number of clips, or is this only a small portion of the full dataset?

I have downloaded the videos from the given link, but there are only 8000~ clips, is there any preprocessing for cutting short clips? Thx.

DefaultPredictor of detectron2_modify, only disable parameters grad, but remain model in training, this cause wrong logic when using Mask2Former as criterion.

solution: restore the model.eval() as in original detectron2:
https://github.com/facebookresearch/detectron2/blob/0ae803b1449cd2d3f8fa1b7c0f59356db10b3083/detectron2/engine/defaults.py#L280-L285

another bug: it's loading the first char of a path string

other problems:
the readme says pip install -e to install detectron2, this will make our own tools unable to import. for now, need to use this pr facebookresearch/detectron2#5283

there is also missing symbol link of Mask2Former in SportsSloMo_EBME, it's no said in readme, neither tracked in git repo.

the shell command in readme is not perfect, it's common that command needs to be run in parent or sub dir.