1adrianb / binary-face-alignment

Real time face alignment

Binarized Convolutional Landmark Localizers for Human Pose Estimation and Face Alignment with Limited Resources (Face Alignment demo)

This code implements a demo of the Binarized Convolutional Landmark Localizers for Human Pose Estimation and Face Alignment with Limited Resources paper by Adrian Bulat and Georgios Tzimiropoulos.

For the human pose estimation demo please see https://github.com/1adrianb/binary-human-pose-estimation

[2021 Update]: PyTorch repo with training code for BNN available here: https://github.com/1adrianb/binary-networks-pytorch

Requirements

• Install the latest Torch7 version (for Windows, please follow the instructions avaialable here)

Packages

• cutorch
• nn
• cudnn (cudnn5 preffered)
• xlua
• image
• gnuplot
• cURL
• paths

for pkg in cutorch nn cudnn xlua image gnuplot lua-cURL paths; do luarocks install ${pkg}; done

Setup

Clone the github repository

git clone https://github.com/1adrianb/binary-face-alignment --recursive
cd binary-face-alignment

Build and install the BinaryConvolution package

cd bnn.torch/; luarocks make; cd ..;

Install the modified optnet package

cd optimize-net/; luarocks make rocks/optnet-scm-1.rockspec; cd ..;

Run the following command to prepare the files required by the demo. This will download the AFLW2000-3D dataset alongside the converted dataset structure.

th download-content.lua

Usage

In order to run the demo simply type:

mkdir models && wget https://www.adrianbulat.com/downloads/BinaryHumanPose/facealignment_binary_aflw.t7 -O models/facealignment_binary_aflw.t7
th main.lua

Pretrained models

Layer type	Model Size	AFLW2000-3D NME error
AFLW2000-3D	1.4MB	3.28

Note: More pretrained models will be added soon.

Notes

For more details/questions please visit the project page or send an email at [email protected]

binary-face-alignment's Issues

Visualize the bounding box

Is it possible to plot/display the bounding box around the face according with the test data's scale and center?

Thank you.

conversion from bnn.SpatialConvolution and bnn.binary to nn or cudnn

I was trying to convert this model into CPU supported device(nn), was able to convert some of the layers using cudnn.convert, but getting stuck at layers from bnn.torch, has anyone converted bnn layers to nn or cudnn.
any help would be appreciated.
Thank you

Error when 'luarocks make'

I followed the instructions in the README.md file and this error appears:

jgoenetxea@Trantor:~/git/binary-face-alignment$ cd bnn.torch/; luarocks make; cd ..;
Error: Argument missing: please specify a rockspec to use on current directory.

I am new with lua, and I do not know how to fix this.

Training code

Hello Adrian,
Do you have any plans for publishing the training code for the binary face alignment model?
Thank you.

Visualize the net structure! So complicated!

I draw the net structure from your torch model, but i am not sure correctness.
Can share your net structure?
net structure
NOTE: I didn't add binary layer

Very large GPU memory consumption

I described this separately in a Stackoverflow question, but the short version is, why does this network for those with 'limited resources' consume approximately 700mb of GPU memroy just to run the main.lua script?

I tried running the main.lua on this maching with a NVidia Quadro K600 with 900 MB of memory, and it fails at
output:copy(model:forward(img))
for the first image!

Maybe I missunderstood, but if the network is ~1.5 mb, and the image is ~1.5mb, why does the GPU need to load ~200x times that much?

AFLWPIFA-val.tar can not download

https://www.adrianbulat.com/downloads/datasets/AFLWPIFA-val.tar, seems this resource is not existed.

How convert this model to pytorch?

Hello!

How convert this model to pytorch or other popular frameworks (caffe, keras, tensorflow)? Opensource converters and pytorch utils can not do this.

How Can I preditct a new image?

I have run your code with the samples under the floder /dataset/AFLW2000/ . But I find that each '.jpg' image file contains corresponding '.t7' file. But when I process a new image how can I generate '.t7' file. I have seen the 'utils.lua' under the 'binary-face-alignment/' directory, the corresponding '.t7' contains scale and center. What does it mean when I do face alignment? I have no idea regarding the above. Thanks very much!

where is the groundTruth in eval？

When I run in mode=eval mode, main.lua reports an error when it executes utils.calcDistance(predictions,groundTruth). Parsing groundTruth from your code is fileLists in main.lua. . . . The data structure of filesLists does not contain the points member variable. . . . . . So, where is the groundTruth in eval?

The result of my training is very poor，why？

Hello, I tried to train the binary network in the way of the paper XNOR-Net （https://github.com/allenai/XNOR-Net), however, the training result is very poor, the structure of my model is the same as the pretrained model you give ,and the training parameters are based on the parameters given in the paper. I used BCECriterion, the loss is always around 0.027 when i trained more then 47 epoch, it doesn't drop anymore, and the prediction is poor too. I‘m very confused why the result is so poor ,could you tell me some training details？ Thank you very much!

How to convert to bnn.BinarySpatialConvolution from cudnn.SpatialConvolution.cu after training

(perhaps this is a better place to ask than 1adrianb/bnn.torch#2? Sorry for the duplicate!)

Firstly, thank you so much for releasing your code, it helps a lot in understanding your awesome paper!

If I understand the code correctly, for face-alignment, if I want to do everything from the scratch, I have to first train the floating point model using your codebase (https://github.com/1adrianb/face-alignment-training). However, I am missing the link of how you convert this model to the binary model that is available in https://github.com/1adrianb/binary-face-alignment which uses bnn.BinarySpatialConvolution

It seems to me that I need to call the function BinarySpatialConvolution:binarise(convLayer) somehow, but I am a bit confused. Do you have a script which replaces the floating conv modules with their binary counterparts? Do you also train in binary?

I will be very grateful if you can help me out, thanks a lot in advance.