leonshting / fec-dataset-loader

Loader for Google facial expression comparison dataset

This package loads images and composes these images into triplets according to Google facial expression comparison dataset

0. Fetch submodule with some useful utils git pull --recurse-submodules

1. The package uses Tor service to rotate IPs since it was observed, that connection to Flickr, where most of photos from dataset are hosted, stalls after some number of downloads. So the first stage is to start tor service, with which IPs are rotated, refer to this beautifully useful repo

2. Once the service is started and dataset meta is loaded from Google servers, make sure you pip install -r requirements.txt

3. The first actual dataset related command is the following: python3 ./load/load_fec.py --tor_pass=mypass --path_to_meta=<meta_info_path> --path_to_images=<path_to_save_images>, this will take some time, depends on your bandwidth

4. The second command is: python3 ./load/prettify_fec.py --path_to_meta=<meta_info_path> --path_to_images=<path_with_saved_images> --path_to_triplets=/home/mininlab/DATA/fec_try_triplets --split=(train|test)

At the end you will have 3 new files in <meta_info_path>: list with loaded and missed images loaded_images_meta.json, missed_images_meta and final markup file (train|test)_triplets_info.csv and 2 folders with genuine images and triplet images respectively.

fec-dataset-loader's Issues

Issue with SocksiPy and Tor in Python3

Hi, when I try to attempt step 3 in the README I run into the following error:

RuntimeError: maximum recursion depth exceeded while calling a Python object

It appears this is an issue with using SocksiPy with Tor in Python3. However, downloading socks.py to the working directory per this suggestion on SO https://stackoverflow.com/questions/21357909/socksipy-fails-when-connecting-throug-tor
doesn't help, it appears that the socks.py script is not compatible with Python3.

What environment did you use to run this downloader for yourself? Thanks.

cannot fetch images through flickr urls

Before applying your loader, I cannot even download images from the provided sheet.
It seems flickr is almostly dead. If you have a local copy of dataset, could you share with google drive or something?