activitynet / activitynet

This repository is intended to host tools and demos for ActivityNet

ActivityNet: A Large Scale Video Benchmark for Human Activity Understanding

This repository is intended to hosts tools and demos for ActivityNet.

Who is behind it?

Main contributor	Contributor	Contributor	Advisor	Advisor
Fabian Caba	Humam Alwassel	Victor Escorcia	Bernard Ghanem	Juan Carlos Niebles

Do you want to contribute?

1. Check the open issues or open a new issue to start a discussion around your new idea or the bug you found
2. For the repository and make your changes!
3. Send a pull request

activitynet's Issues

annotations of test set

Hello,
I'm downloading the Kinetics dataset currently, but I cannot find annotations of test set. Do I miss anything or the test file with annotations has not been released yet?
Also, as it is reported in the paper "Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset" by Joao Carreira and Andrew Zisserman, there is a subset of Kinetics dataset called miniKinetics, which may help to both reduce time consumption and computation resource usage. So could you please tell me how to get the CSV files of miniKinetics?
Thanks!

Handling the case of multiple labels

There are some videos in Kinetics-600 dataset with multiple labels associated with them. How are such cases to be handled ?

Example - In the validation set the video file dMzycn1d3qU_000000_000010.mp4 has following two labels associated with them as follows

In [25]: df.loc['dMzycn1d3qU']                                                                           
Out[25]: 
                      label  time_start  time_end split
youtube_id                                             
dMzycn1d3qU  weaving fabric           1        11   val
dMzycn1d3qU   yarn spinning           0        10   val

All in all there are 8 such videos in the validation set. I enumerate them by their youtube IDs as below

In [27]: df.groupby(level=0).filter(lambda x: len(x) > 1)                                                                                
Out[27]: 
                           label  time_start  time_end split
youtube_id                                                  
xnC3nngvLCo    cracking knuckles           7        17   val
xnC3nngvLCo        cracking neck          83        93   val
c9ZWYpxefXQ        cutting apple          64        74   val
SN5awGHfw9M  kicking soccer ball          36        46   val
SN5awGHfw9M  passing soccer ball          27        37   val
c9ZWYpxefXQ       peeling apples          14        24   val
Z03StBdQzog          sawing wood          17        27   val
CYGlFBgKZV0               sewing         102       112   val
CYGlFBgKZV0     threading needle         145       155   val
Z03StBdQzog   using circular saw          55        65   val
dMzycn1d3qU       weaving fabric           1        11   val
dMzycn1d3qU        yarn spinning           0        10   val

can be used in windows? if so, how to change?

Not able to download videos

I am trying this code on google colab. When I run it, I downloads some videos at the beginning but after that, it is not any downloading videos.

I checked the status of it in the status list, it shows me following

"ERROR: Unable to download webpage: HTTP Error 429: Too Many Requests (caused by <HTTPError 429: 'Too Many Requests'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output."

SO please help me in it

File command_list.txt does not exists.

I need to download Activitynet_1.3 dataset so I add the json file in the current directory and run these commands:
mkdir Dataset
chmod +x fetch_activitynet_videos.sh
./fetch_activitynet_videos.sh Dataset activity_net.v1-X.json
but i got this error:
usage: run_crosscheck.py [-h] video_path ann_file output_filename
run_crosscheck.py: error: too few arguments
File command_list.txt does not exists.
Have a good day!

[Evaluation] Update AP interpolation method

The current version of the code uses the 11-point interpolation method. This measure was used in the PASCAL VOC challenge up to the 2008 edition. In the newer versions of Pascal VOC the interpolation method is following this code: https://github.com/npinto/VOCdevkit/blob/master/VOCcode/VOCap.m.

Required actions:

• Update the function interpolated_prec_rec to act in the same way of VOCap.m

What are blocked_videos?

Hi all,
Thanks for your nice job! Could you please let me know what are the blocked_videos?

How to get the missing video of kinetics-400

After downloading kinetics-400 training set using the script, only around 220000 videos can be obtained. How can I get the missing videos? I only use it for academic purpose:)

"timeout or by a memory leak.", UserWarning

The program error is：/usr/local/lib/python2.7/dist-packages/joblib/externals/loky/process_executor.py:706: UserWarning: A worker stopped while some jobs were given to the executor. This can be caused by a too short worker timeout or by a memory leak.
I have used vpn.
Please help me.

Data is not downloading!

Hi data is not downloading, not with 400, and not with 600, don't send me to another post because I saw all of them, and no one solved the problem. and my youtube-dl is installed correctly.
I'm using this line:
python3 download.py data/kinetics-400_train.csv kinetics-400_train/
and nothing happens, stuck, no error, no warning, I didn't even get this file:
progress_report.json

Please help.

can't create environment

i've tried running condaa env create -f environment.yml but got the following error:

ResolvePackageNotFound:

• readline==6.2=2
• ffmpeg==3.1.3=0
• sqlite==3.13.0=0
• tk==8.5.18=0
• zlib==1.2.8=3
• openssl==1.0.2k=1

Any ideas? :)

Annotation with length 0 or 0.01

Hi, I am doing plot about the annotation length on activitynet v1.3 using the JSON file downloaded from your website. I accidentally find that some of the annotation length are 0 or 0.01. For example, video 'F_EHrfbYsRE' has annotation {'segment': [0.01, 0.02], 'label': 'Running a marathon'}, video 'r96PJ548pn8' has annotation {'segment': [0.01, 0.02], 'label': 'Doing crunches'}, video 'ZZVrH3Hv1fM' has annotation {'segment': [0, 0], 'label': 'Washing hands'}.

Is the JSON file correct? And why these annotation exist? Thank you in advance~

Extract RGB frames from videos

Hello,

Thank you for making this dataset available. Could you point me to a repo/script that I can use to extract every nth RGB frame from the videos in the dataset?

Thanks,

can't download dataset?

Hi. help pls. thx

sudo python download.py /opt/kinetics/train_split/data/kinetics_train.csv /opt/kinetics/train_split

Traceback (most recent call last):
File "download.py", line 10, in
from joblib import delayed
ImportError: No module named joblib

conda install joblib

Collecting package metadata: done
Solving environment: done
All requested packages already installed.

pip install joblib

Requirement already satisfied: joblib in /home/admin/.pyenv/versions/anaconda3-2018.12/envs/kinetics/lib/python2.7/site-packages (0.13.2)

Nothing is downloaded

I checked every step as the instructions, and the environment seems 'right:'
(kinetics) svc7@omnisky-SYS-7048GR-TR:~/ActivityNet/Crawler/Kinetics$ conda list

packages in environment at /home/svc7/anaconda3/envs/kinetics:

Name Version Build Channel

decorator 4.0.11
ffmpeg 3.1.3 0 menpo
joblib 0.9.4 py27_0 defaults
mkl 2017.0.1 0 defaults
numpy 1.12.1 py27_0 defaults
olefile 0.44
openssl 1.0.2k 1 defaults
pandas 0.19.2 np112py27_1 defaults
pip 9.0.1 py27_1 defaults
python 2.7.13 0 defaults
python-dateutil 2.6.0 py27_0 defaults
pytz 2017.2 py27_0 defaults
readline 6.2 2 defaults
setuptools 27.2.0 py27_0 defaults
six 1.10.0 py27_0 defaults
sqlite 3.13.0 0 defaults
tk 8.5.18 0 defaults
wheel 0.29.0 py27_0 defaults
youtube-dl 2018.6.4
zlib 1.2.8 3 defaults

But when I run download.py data/kinetics-400_train.csv /data/zjs/kinetics
All the directories (class_names) are generated, but no video (0) has been download.
I tried several times, but it is still the same problem. I am wondering if you could give some advice??

Am l allowed to use the crawler to download all the kinetics dataset ?

Hello,

l'm wondering whether, we can download all the dataset.
In other term, in order to avoid that my IP adress will be blocked, how much videos can l download ?

Thank you

Can't download videos due to too many requests error.

CSVs are downloaded from the https://deepmind.com/research/open-source/open-source-datasets/kinetics/
Followed the instructions of ActivityNet/Crawler/Kinetics/README.md to create the environement.
Created video/ in the same directory as the the download script.
Made 1 modification to the script to print whenever an error occurs in downloading a file.
Ran the following command
python download.py data/kinetics_600_test.csv videos/

After downloading about 60 videos in, I get the following error for all videos.
ERROR: Unable to download webpage: HTTP Error 429: Too Many Requests (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

I tried both the download.py used in this repo as well as this one.

Using Aws ec2 instance

low

C3D features

I am unable to figure out how to read the C3D features from the files present in http://activity-net.org/challenges/2016/download.html . Can you please explain the same?

All videos in the ActivityNet training set are unavailable on Youtube as of 2018-03-06

All videos are reporting download errors using the downloader script in this repo. We tried accessing some by hand from Youtube and they report the same error as youtube-dl (as they should, sadly).

The error is invariably

This video is unavailable.

Example:

Update: All training videos are unavailable.

Nothing is downloaded with Kinetics-400!!!

@escorciav Thanks for you good work
I have succeed in installing the youtube-dl and the requirement environment, but I still not worked. And
I run
(kinetics) ➜ Kinetics git:(master) ✗ python2.7 download.py ./data/kinetics-400_train.csv ./kinetics-dataset/train-400
this code would produce the many folders under the path, ./kinetics-dataset/train-400
however, it didn't download anything, although it worked more than one night.
Looking forward to any replies

Some videos of ActivityNet1.3 are not available

I am trying to ActivityNet1.3 with crawler, but some videos are not downloaded.
Could you check how many videos in ActivityNet1.3 are now available?

Add helper to crawler

Add argparse helper to kinectic crawler. Check argparse tutorial

Speed about download.py

When I modified the line 104 'libx264' as 'copy', the speed of getting video files became super fast....

why use 'libx264'?

Can't find the file called '{dataset_split}.csv' in official website

Hi,
I can't find the files called '{dataset_split}.csv' in official website, there is not any csv file in official website. So, Where can I get the the files called '{dataset_split}.csv'?

Can't I download the test set?

When I try to download Kinetics test set, it report error:
KeyError: 'label-name'

size of Kinetics-600

I am wondering how large Kinetics-600 is. I am downloading it now and finished around 330G.
I saw someone said Kinetics-400 is around 311G. Does that mean Kinetics-600 is around 470G?
Just curious about that.
Thank you.

solved

TypeError: Object of type 'bytes' is not JSON serializable

Hi, I had this problem:

Traceback (most recent call last):
File "download.py", line 220, in
main(**vars(p.parse_args()))
File "download.py", line 200, in main
fobj.write(json.dumps(status_lst))
File "/home/anaconda3/lib/python3.6/json/init.py", line 231, in dumps
return _default_encoder.encode(obj)
File "/home/anaconda3/lib/python3.6/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/home/anaconda3/lib/python3.6/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/home/anaconda3/lib/python3.6/json/encoder.py", line 180, in default
o.class.name)
TypeError: Object of type 'bytes' is not JSON serializable

I'll be happy if you have any clue, about this problem.
Thanks.

OSError: File b'{dataset_split}.csv' does not exist

Hi!!
I'm trying to download Kinetics database on Ubuntu Server 16.04.01, but getting following error:

OSError: File b'{dataset_split}.csv' does not exist

I've performed all steps in README.md file, but on the lats step:

mkdir DATASETS; python download.py {dataset_split}.csv DATASETS

I'm getting errors:

yi@server-30:~/Kinetics/ActivityNet/Crawler/Kinetics$ mkdir DATASETS; python download.py {dataset_split}.csv DATASETS
Traceback (most recent call last):
  File "download.py", line 192, in <module>
    main(**vars(p.parse_args()))
  File "download.py", line 154, in main
    dataset = parse_kinetics_annotations(input_csv)
  File "download.py", line 142, in parse_kinetics_annotations
    df = pd.read_csv(input_csv)
  File "/home/yi/anaconda3/lib/python3.5/site-packages/pandas/io/parsers.py", line 562, in parser_f
    return _read(filepath_or_buffer, kwds)
  File "/home/yi/anaconda3/lib/python3.5/site-packages/pandas/io/parsers.py", line 315, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/home/yi/anaconda3/lib/python3.5/site-packages/pandas/io/parsers.py", line 645, in __init__
    self._make_engine(self.engine)
  File "/home/yi/anaconda3/lib/python3.5/site-packages/pandas/io/parsers.py", line 799, in _make_engine
    self._engine = CParserWrapper(self.f, **self.options)
  File "/home/yi/anaconda3/lib/python3.5/site-packages/pandas/io/parsers.py", line 1213, in __init__
    self._reader = _parser.TextReader(src, **kwds)
  File "pandas/parser.pyx", line 358, in pandas.parser.TextReader.__cinit__ (pandas/parser.c:3427)
  File "pandas/parser.pyx", line 628, in pandas.parser.TextReader._setup_parser_source (pandas/parser.c:6861)
OSError: File b'{dataset_split}.csv' does not exist

What I'm doing wrong?

Classes of ActivityNet

Hi the organizers,
I want to explore ActivityNet dataset. I am running to the problem of classes of ActivityNet dataset, because we only know classes of a video by looking at json file annotation.
I want a standard classes.txt for cross-checking and also use as a 'numerical' category for training a CNN.
Best regards,

Could you please provide the C3D features pre-trained models?

Hi,
Thank you for providing the c3d features in dense video caption dataset.
I want to test the published models with local dataset. And it's necessary for us to know the pre-trained model which use to extract the C3D features. So, I wander if you could provide the pre-trained model?
Thank you anyway.

ERROR: this video is unavailable

Hi, thanks for your great contribution!
However I have two questions while I am downloading the dataset: activitynet-1.3 as follows:

1. It shows that some of the youtube accountis terminated so the video is not available either.
2. Due to some unknown reasons, the video is unable to extract, also the webpage cannot be reached.
   

Now I have downloaded just 18449 videos, could you please tell me the solution?

I want to extract clips from videos? How should I do? Thanks

where is command_list.txt file to download?

When I run the last line it show me following error:"File command_list.txt does not exists."

Thanks

How can I download the test set of Kinetics

Pose estimation dataset

Hi, are you aware of recent existing datasets with pose-estimation on any of the Kinetics datasets ?
Thanks.

How many videos left for Kinetics-400？

Hi,
I downloaded Kinetics-400 about 3 months ago, there were:
226797 training videos ;
18689 val videos;
36499 test videos.
Did anybody download Kinetics-400 recently?
Would it be better to keep an official blacked video-list as Activity-Net dataset?
@cabaf @escorciav @HumamAlwassel
Thanks very much.

Update kinetics data

Due to the recent update in Kinetics-600 we must update the files.

how to download Kinetics dataset?

@cabaf
I am trying to download this dataset to work with this project but I cant do it. Could you please give some detailed instructions on how to download this dataset. When I type conda env create -f environment.yml in ActivityNet\Crawler\Kinetics path I get this error:

Solving environment: failed

ResolvePackageNotFound:
  - readline==6.2=2
  - openssl==1.0.2k=1
  - ffmpeg==3.1.3=0
  - zlib==1.2.8=3
  - sqlite==3.13.0=0
  - tk==8.5.18=0

And,
I don't understand this part at all
mkdir <data_dir>; python download.py {dataset_split}.csv <data_dir>
BTW, I am on Windows 10
Thank you

How much space of the Kinetic dataset?

As title, How much space of the Kinetic dataset?

AttributeError: 'module' object has no attribute 'main'

About the program stuck

Thank you very much for your code. I can't wait to use it, but when I am executing this line of code:
python download.py data/kinetics-400_test.csv data_dir
It has been executed all the time, and there are no displays or errors. It still works like a night. So I debug download.py and I found that the program card is on the method corresponding to the two objects imported from joblib. How should I deal with this situation?

Kinetics-400 test label

I can find kinetics-600 holdout test and test label in deepmind website. (https://deepmind.com/research/open-source/open-source-datasets/kinetics/)

However, I cannot find kinetics-400 test label. Where can I find the kinetics-400 test label?

How to extract frames

Hi, I found reading from the original .mp4 videos is very slow. Do I need to extract every frame in each video and read the RGB frames directly? If yes, do you have a script to do that? Thanks very much.

What if I do not Anaconda with Kinetics downloader script?

Hi,

I am open this issue to request a feature for setting up an environment for the user, who do not use Anaconda.

conda env create -f environment.yml
source activate kinetics

Best regards,
An

Nothing is downloaded with Kinetics-400

Hi
I am trying to download videos and I use the command :
python download.py data/kinetics-400_train.csv /data/sammer/Kinetics
However no videos are downloaded, the directories are created but there are no videos inside. Is that an issue with youtube or should I just use the other csv file ?

模型咨询

ActivityNet代码有吗，我现在想学习一下

Incomplete Dataset

Hello

I have used this crawler to download the full Kinetics dataset for the benchmark of algorithms. However, when using the script some of the videos are not being downloaded with the crawler. After exploring the JSON progress logger the following errors arise.

Errors:
ERROR: fC4iVtuX22Q: YouTube said: This video does not exist.ERROR: fC4iVtuX22Q: YouTube said: This video does not exist.
ERROR: xxq3FXccFrM: YouTube said: Please sign in to view this video.
ERROR: yIELmF1k2no: YouTube said: This video has been removed by the user.

Three different types of errors that need to be addressed. In total there were about ~50 videos I was unable to download.

Error when trying to download kinetics test set

After running download.py I get the following error, only with the test set. The other two work fine.

Traceback (most recent call last):
  File "download.py", line 192, in <module>
    main(**vars(p.parse_args()))
  File "download.py", line 157, in main
    label_to_dir = create_video_folders(dataset, output_dir, tmp_dir)
  File "download.py", line 23, in create_video_folders
    for label_name in dataset['label-name'].unique():
  File "/home/pedro/anaconda2/lib/python2.7/site-packages/pandas/core/frame.py", line 1964, in __getitem__
    return self._getitem_column(key)
  File "/home/pedro/anaconda2/lib/python2.7/site-packages/pandas/core/frame.py", line 1971, in _getitem_column
    return self._get_item_cache(key)
  File "/home/pedro/anaconda2/lib/python2.7/site-packages/pandas/core/generic.py", line 1645, in _get_item_cache
    values = self._data.get(item)
  File "/home/pedro/anaconda2/lib/python2.7/site-packages/pandas/core/internals.py", line 3590, in get
    loc = self.items.get_loc(item)
  File "/home/pedro/anaconda2/lib/python2.7/site-packages/pandas/core/indexes/base.py", line 2444, in get_loc
    return self._engine.get_loc(self._maybe_cast_indexer(key))
  File "pandas/_libs/index.pyx", line 132, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 154, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 1210, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 1218, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'label-name'