epic-kitchens / c2-action-detection Goto Github PK

Hello, I'm having a little trouble loading the data.
When I compute validation/test detection, the program will first load all the data sets into memory. But my machine memory is not that good, so how do I load the data directly from disk without loading the data set into memory before the program starts.

Hi, I wonder are there any updates on the challenges C2 and C5?

Hi, I'm trying to reproduce the results of Action Detection experiment for EPIC-KITCHEN 100 dataset. First of all, thank you for releasing the code. I follow the instructions step by step given in the ReadME file. I get the following error on the last step - "Computing Validation/Test Detections". I write the following command :

python run_net.py --cfg ../epic-kitchens-slowfast/configs/EPIC-KITCHENS/SLOWFAST_8x8_R50.yaml NUM_GPUS 1
OUTPUT_DIR ../results/
EPICKITCHENS.VISUAL_DATA_DIR ../EPIC-KITCHENS/
EPICKITCHENS.ANNOTATIONS_DIR ../anno/
TRAIN.ENABLE False
TEST.ENABLE True
TEST.CHECKPOINT_FILE_PATH ../SlowFast.pyth
EPICKITCHENS.TEST_LIST ../C2-Action-Detection/BMNProposalGenerator/output/ek100/result_proposal-test.pkl
EPICKITCHENS.TEST_SPLIT test
TEST.BATCH_SIZE 2

The program start running and prints out the model and other variables correctly. And then after 2-3 minutes give the following error :

from ['/home/user/puri/va/C2-Action-Detection/BMNProposalGenerator/output/ek100/result_proposal-test.pkl']
[INFO: run_net.py: 202]: Testing model for 727370 iterations
Traceback (most recent call last):
File "run_net.py", line 387, in
main()
File "run_net.py", line 382, in main
test(cfg=cfg)
File "run_net.py", line 232, in test
preds, labels, metadata = perform_test(test_loader, model, test_meter, cfg)
File "run_net.py", line 49, in perform_test
for cur_iter, (inputs, labels, video_idx, meta) in enumerate(test_loader):
File "/home/user/puri/miniconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 521, in next
data = self._next_data()
File "/home/user/puri/miniconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1203, in _next_data
return self._process_data(data)
File "/home/user/puri/miniconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1229, in _process_data
data.reraise()
File "/home/user/puri/miniconda3/lib/python3.8/site-packages/torch/_utils.py", line 425, in reraise
raise self.exc_type(msg)
KeyError: Caught KeyError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/user/puri/miniconda3/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 2891, in get_loc
return self._engine.get_loc(casted_key)
File "pandas/_libs/index.pyx", line 70, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 101, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 1675, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 1683, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'stop_timestamp'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/user/puri/miniconda3/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
data = fetcher.fetch(index)
File "/home/user/puri/miniconda3/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/user/puri/miniconda3/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/user/puri/va/C2-Action-Detection/SlowFastProposalClassifier/slowfast/datasets/epickitchens.py", line 129, in getitem
frames = pack_frames_to_video_clip(self.cfg, self._video_records[index], temporal_sample_index)
File "/home/user/puri/va/C2-Action-Detection/SlowFastProposalClassifier/slowfast/datasets/frame_loader.py", line 34, in pack_frames_to_video_clip
video_record.num_frames,
File "/home/user/puri/va/C2-Action-Detection/SlowFastProposalClassifier/slowfast/datasets/epickitchens_record.py", line 43, in num_frames
return self.end_frame - self.start_frame
File "/home/user/puri/va/C2-Action-Detection/SlowFastProposalClassifier/slowfast/datasets/epickitchens_record.py", line 34, in end_frame
return int(round(timestamp_to_sec(self._series['stop_timestamp']) * self.fps))
File "/home/user/puri/miniconda3/lib/python3.8/site-packages/pandas/core/series.py", line 882, in getitem
return self._get_value(key)
File "/home/user/puri/miniconda3/lib/python3.8/site-packages/pandas/core/series.py", line 991, in _get_value
loc = self.index.get_loc(label)
File "/home/user/puri/miniconda3/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 2893, in get_loc
raise KeyError(key) from err
KeyError: 'stop_timestamp'

I don't really understand what this error is trying to tell as it just prints out a simple error message KeyError: 'stop_timestamp' . Can you tell what this error is trying to point so I can try fixing it? Thank you so much!

Hi, thank you once more for providing an easy-to-use code!
Could you please clarify some details about the work?
I have some questions regarding the rescaled TSN features. You mentioned in the paper that 400 is a large window but did you try to use a larger one, and (if yes) how did it affect the performance?
Also, there are not many details about the features themselves. As far as I understand, the features extractor is TSN with a BN-Inception backbone?
Thank you

Hello,
First, thank you for providing this repo
I was following the instructions to work with the BMN code but didn't find the libs folder where the training script is supposed to be.
Could you please help me with that?

According to https://github.com/epic-kitchens/epic-kitchens-100-annotations/blob/master/EPIC_100_video_info.csv.
Experiment P08_19 should have a video 276.811111 seconds at fps 90, which corresponds to 276.811111*90 = 24913 frames.
But in the rgb files downloaded from https://data.bris.ac.uk/datasets/3h91syskeag572hl6tvuovwv4d/frames_rgb_flow/rgb/test/P18/P18_09.tar,
the number of frames is 16622.
Could you please specify which is correct?
Same problems also exist with the following IDs: 'P06_107', 'P06_12', 'P28_01', 'P06_110'.

Hi, I wonder how long does it take to train BMN when using 4 V100 (16 GB) GPUs.

I compute the validation detections using the following command:

python run_net.py --cfg ../epic-kitchens-slowfast-master/configs/EPIC-KITCHENS/SLOWFAST_8x8_R50.yaml
NUM_GPUS 1
OUTPUT_DIR ..
EPICKITCHENS.VISUAL_DATA_DIR ../epic-kitchens-slowfast-master/dataset
EPICKITCHENS.ANNOTATIONS_DIR ../epic-kitchens-100-annotations-master
TRAIN.ENABLE False
TEST.ENABLE True
TEST.CHECKPOINT_FILE_PATH ../epic-kitchens-slowfast-master/SlowFast.pyth
EPICKITCHENS.TEST_LIST ../BMN/output/ek100/result_proposal-test.pkl
EPICKITCHENS.TEST_SPLIT test
TEST.BATCH_SIZE 1

However, I account with KeyError. Cound you please give any advice?

[INFO: epickitchens.py: 47]: Constructing EPIC-KITCHENS test...
[INFO: epickitchens.py: 81]: Constructing epickitchens dataloader (size: 1454740) from ['../epic-kitchens-100-annotations-master/../BMN/output/ek100/result_proposal-test.pkl']
[INFO: run_net.py: 213]: Testing model for 1454740 iterations
../epic-kitchens-slowfast-master/dataset/P07/rgb_frames/P07_104
../epic-kitchens-slowfast-master/dataset/P07/rgb_frames/P07_104
../epic-kitchens-slowfast-master/dataset/P07/rgb_frames/P07_104
../epic-kitchens-slowfast-master/dataset/P07/rgb_frames/P07_104
../epic-kitchens-slowfast-master/dataset/P07/rgb_frames/P07_104
../epic-kitchens-slowfast-master/dataset/P07/rgb_frames/P07_104
../epic-kitchens-slowfast-master/dataset/P07/rgb_frames/P07_104
../epic-kitchens-slowfast-master/dataset/P07/rgb_frames/P07_104
../epic-kitchens-slowfast-master/dataset/P07/rgb_frames/P07_104
../epic-kitchens-slowfast-master/dataset/P07/rgb_frames/P07_104
../epic-kitchens-slowfast-master/dataset/P07/rgb_frames/P07_104
../epic-kitchens-slowfast-master/dataset/P07/rgb_frames/P07_104
../epic-kitchens-slowfast-master/dataset/P07/rgb_frames/P07_104
../epic-kitchens-slowfast-master/dataset/P07/rgb_frames/P07_104
../epic-kitchens-slowfast-master/dataset/P07/rgb_frames/P07_104
Traceback (most recent call last):
File "/disk/lbs/c2ad/SlowFast/run_net.py", line 399, in
main()
File "/disk/lbs/c2ad/SlowFast/run_net.py", line 394, in main
test(cfg=cfg)
File "/disk/lbs/c2ad/SlowFast/run_net.py", line 243, in test
preds, labels, metadata = perform_test(test_loader, model, test_meter, cfg)
File "/disk/lbs/c2ad/SlowFast/run_net.py", line 116, in perform_test
metadata['narration_id'].extend(meta[i]['narration_id'])
KeyError: 0