Kinetics-400 dataset about coclr

[TengdaHan]

Here is what I did: https://github.com/TengdaHan/CoCLR/blob/main/dataset/convert_video_to_lmdb.py

But it's not essential to use lmdb, you can read frames like https://github.com/TengdaHan/DPC/blob/master/dpc/dataset_3d.py,

or read videos online like https://github.com/pytorch/vision/blob/master/examples/python/video_api.ipynb

[JiaxinZhuang]

Thx for your apply, I am curious which way is the fastest way to load images? Have your compared their time consumption?

[TengdaHan]

I use lmdb if video dataset is huge, I ready from video frames if the dataset is small.
In terms of speed, lmdb is similar to loading frames, except it saves disk I/O a lot.
I've set up these years ago so I didn't change much since then.
But I heard recently the video reader (something like https://pytorch.org/docs/stable/torchvision/io.html or https://github.com/pytorch/vision/blob/master/examples/python/video_api.ipynb) is getting stable and fast, I suggest you first try this.

[JiaxinZhuang]

Hi ~! Finally successully run main_nce on k400 with frames only, which tooks ~100 epochs to reach ~80% top5 in training dataset. Is there any code to generate optmial flows ? It seens that I haven't found them neither in this repository nor DPC. Thx ~

[TengdaHan]

Hi, https://github.com/TengdaHan/MemDPC/blob/master/process_data/src/extract_ff.py here it extracts both frame and flow, if you have frames, you can modify it to only load frames and extract flow. -- In practice, I further split 400 classes into 20 groups, and it could take a week for extracting using about 20 CPU nodes in parallel.

[JiaxinZhuang]

Sure~ I try the code, which indeed costs lots of time while optical flow generated really slow. Can you share your optical flow ? Or any suggestion to speed up the generation of optical flows ? Thx ~

[TengdaHan]

Sharing optical flow for Kinetics seems not possible right now, it's too big.
I am trying to host the optical flow for UCF101 somewhere, hope it helps? -- I will update my progress here.

To speed up optical flow computation, there are some GPU TVL1 optical flow extraction examples:

• https://github.com/feichtenhofer/gpu_flow
• https://github.com/wanglimin/dense_flow

You need to compile OpenCV with GPU support, then compile their C++ function. It would definitely speed up the extraction.

[TengdaHan]

Optical flow for UCF101 (tar file, 20.5GB) is shared here: http://www.robots.ox.ac.uk/~htd/tar/ucf101_flow_lmdb.tar
Optical flow for Kinetics400 is too big to share. I will write a GPU-TVL1 flow extraction pipeline when I have time.