recasens / gaze-following

Code for Gaze-Following in video

Author: Adrià Recasens ([email protected])

In this folder, you will find the essential files for training and using the video gazefollowing model.

Data

The VideoGaze dataset can be found here.

Training

In train.py, you will find the basic training file. The training data is specified in train.txt and test.txt. The format for this files is:

source_name target_name face_name flip eyes_x eyes_y gaze_x gaze_y

where

• source_name is the relative path to the source frame
• target_name is the relative path to the target frame
• face_name is the relative path to the cropped face image
• flip means wether the images should be flipped. The gaze will need to be flipped beforehand.
• eyes_x and eyes_y is the eyes location (assuming the image is 1x1, as provided in the data)
• gaze_x and gaze_y is the gaze location. If gaze is a negative value, this means the gaze is not present in the target frame.

The path for the faces, source frame and target frame images is also described in the begging of the training file.

Demo

In demo.py you will find the basic usage of the model. This file loads the model, a video and a picture of a character in the video. It wil track the character and detect where the person is looking trhough the video.

Requirements

1. PyTorch
2. face_recognitio

gaze-following's Issues

w_fps

what is the value to provide for w_fps in the demo.py example?

Pretrained model

Whenever I try to run demo.py with Python3, I get an error message "No such file or directory: 'model.pth.tar'". Where can I find a pretrained model for this task?

I can't run train.py to train my own model either, since the train.txt and test.txt files are not in the dataset or in the cloned git repo.

Dataset

Hi, Could you send the dataset to me ？thank you！

Application for the VideoGaze dataset

The link to the VideoGaze dataset cannot be opened, could you send the dataset to me？thank you！

training time

I am trying to train this model, when I set epoches value to 1, it takes several hours to train and now I am trying to set epoches value to 900. I'd like to know how long it took you to train the model to achieve good results, and what is the value of epoches.
Thanks!

Difference between papers?

Hi,
Could you highlight the differences between these two papers:
https://people.csail.mit.edu/recasens/docs/videogaze.pdf
https://people.csail.mit.edu/recasens/docs/videogazefollow.pdf

Model download failed

Can you share Google cloud disk