Understanding the flow for training faces about vid2vid

[tcwang0509]

If you're willing to use the un-refactored code, I can provide it here... haven't tested whether it works in the latest version though, but the idea should be the same.
face_dataset.zip

[nrgsy]

I believe the network input is just the raw black & white edge image and the output is the photorealistic face image matching those edges.

[nrgsy]

Even just getting the code that was used to detect/connect the face landmarks and generate background edges for the edge map sketch for a single video frame would be great. dlib (http://dlib.net/ml.html) was cited in the paper, but it would be nice to have the code that merges the connected facial landmark sketch with the background edges from the canny edge detector.

[dustinfreeman]

I do wonder: is the training input just the binarized facial features and edge map? Or are the different facial features mapped to different channels? Even the paper is pretty unclear about it.

[fxfactorial]

Would be great to get confirmation (& copy pastable commands/python code) from the OPs @tcwang0509 @junyanz or anyone else please.

[fxfactorial]

Actually digging into the source code, this repo is not complete. It is missing face_dataset.py

As of bdb7ec6, this will fail on line 11 of custom_dataset_data_loader.py

        from data.face_dataset import FaceDataset

I'm wondering why this code was not included as it is quite valuable and (at least to me and some other folks) more interesting than the segmentation map, change landscape based code/ideas.

Opened issue: #26

[tcwang0509]

Yes we'll update the face dataset part shortly. As pointed out, we run landmark (using dlib library) and edge detection on the face images, and use the B&W image as input. I'm currently in an urgent deadline, and probably won't be able to update it until next week or the week after though...

[fxfactorial]

@tcwang0509 Appreciate your answer & time. Could you please outline what would be needed to do to at least stub it out/reimplement it (as detailed as possible) ? Or is it too complicated and better for you to just dump the code? Very much want to have this working by this coming Monday/Tuesday as training also takes time.

[fxfactorial]

@tcwang0509 Love it, thank you very much, that's all I need right now, thanks!