yulequan / face-alignment-in-3000fps Goto Github PK

In Utils.cpp line 296 you call adjustImage when looping over face detections within an image which overlap with a face detection:

adjustImage(image,ground_truth_shape,boundingbox);
images.push_back(image);
ground_truth_shapes.push_back(ground_truth_shape);
bounding_boxs.push_back(boundingbox);

However, image is defined outside of the face loop, so when multiple detections are made which overlap with the ground truth, I believe the image will effectively be over-cropped.

This could be resolved by cloning the image and ground_truth_shape in the loop:

Mat_<uchar> adjusted_image = image.clone();
Mat_<double> adjusted_ground_truth_shape = ground_truth_shape.clone();
adjustImage(adjusted_image,adjusted_ground_truth_shape,boundingbox);
images.push_back(adjusted_image);
ground_truth_shapes.push_back(adjusted_ground_truth_shape);
bounding_boxes.push_back(boundingbox);

Furthermore, in the adjustImage function, you have the lines:

double left_x  = max(1.0, bounding_box.centroid_x - bounding_box.width*2/3);
double top_y   = max(1.0, bounding_box.centroid_y - bounding_box.height*2/3);
double right_x = min(img.cols-1.0,bounding_box.centroid_x+bounding_box.width);
double bottom_y= min(img.rows-1.0,bounding_box.centroid_y+bounding_box.height);

Followed later by:

img = img.rowRange((int)top_y,(int)bottom_y).colRange((int)left_x,(int)right_x).clone();

Should the first two lines not read max(0.0, ...?

Perhaps I'm missing something, but let me know if you agree there are errors here as happy to issue a PR.

"y = mean(temp.col(1))[1]"
I think it should be "y = mean(temp.col(1))[0]" ?

@yulequan I have read your project. It's impressive. But I'm confused that in the test.cpp's run() method you still use the landmark, I want to know that if you get a image from your phone that have no landmark any more. How do you predict the people's face?

feat[0] =candidate_pixel_locations(max_id,0)/max_radio_radius_; is divided by max_radio_radius_.

When the random forest is training, the candidated position is calculated with the array candidate_pixel_locations.
double project_x1 = rotation(0,0) * candidate_pixel_locations(j,0) + rotation(0,1) * candidate_pixel_locations(j,1);----(1)

But when calculating the binary features, the stored feature (which is divided by max_radio_radius_) is used.
double x1 = rand_forest[iter].nodes_[currnode].feat[0];
...
double project_x1 = rotation(0,0) * x1 + rotation(0,1) * y1;----(2)

which meas that the above code (1) and (2) is not using the same candidate_pixel_locations.

Thanks for your contribution, firstly.When I run the code, the console print Segmentation fault (core dumped) using the data set helen. But it is ok using the command './LBF.out Demo 1.jpg'. I would appreciate it if anyone could help me. Thanks!

Trying to use your model by $ ./LBF.out ./img/test.jpg got me this:

./LBF.out: /home/bh/anaconda3/lib/libgomp.so.1: version `GOMP_4.0' not found (required by ./LBF.out)
./LBF.out: /home/bh/anaconda3/lib/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by ./LBF.out)
./LBF.out: /home/bh/anaconda3/lib/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by ./LBF.out)

this libgomp1 package is currently on version 6.3
Any tips to solve that?

scale bbox code

I cannot understand why you do this. If you want to enlarge the bbox, It should be center_x - width * scale / 2 instead of x * scale.

I have a problem about "LoadData(testdatapath, test_images, test_ground_truth_shapes, test_bounding_boxs)" and "LoadOpencvBbxData(testdatapath, test_images, test_ground_truth_shapes, test_bounding_boxs);". The two functions are to load images and prepared for alignment. The helen_teset datasets have 330 images. If i use LoadData, the number of test images is 330(test data size =330).It's right and the speed of loading images is quick,but the CalculateError is big(0.390823). If i use LoadOpencvBboxData,the number of test images is 302(test data size =302) and loss 28 images.And the speed of loading images is slow,but the CalculateError is small(0.077962).
The speed may be because of the function complex difference, but the loss i don't know why?
Can the speed change in LoadOpencvBbxData ?
I hope to get some suggestions.
Thanks very much!

Hi ,I do the usage step 1-3,then i change the path in LBF.cpp,
I do both train model and test model all get : Segmentation fault: 11.
I found this problem is related to memory.
My platform is osx10.10.1.
I need some help!Thanks very much!

I just run the code In vs2012. I feel the problem still is blas.a "fatal error LNK1104: 无法打开文件“blas.a”".If I can put blas.a into the project.The complie is successful,but blas.a is invalid lib. So I want to learn how to complie blas.a in VS in win? Thanks very much.

hello,sir
I have errors after compiling the project.The error is given as follows--"error LNK2019: 无法解析的外部符号 "double __cdecl TestModel(class std::vector<class std::basic_string<char,struct std::char_traits,class std::allocator >,class std::allocator<class std::basic_string<char,struct std::char_traits,class std::allocator > > >)" (?TestModel@@yanv?$vector@V?$basic_string@DU?$char_traits@D@std@@v?$allocator@D@2@@std@@v?$allocator@V?$basic_string@DU?$char_traits@D@std@@v?$allocator@D@2@@std@@@2@@std@@@z)，该符号在函数 main 中被引用 E:\Face-Alignment-in-3000fps\face-alignment-in-3000fps-master\LBF.obj LBF.out".How could avoid the error?
Thanks in advance.

相似变换的计算函数，由2个协方差矩阵就求出来了，请问有相应公式么？

另外从投影后的像素位置反推真实图像中的像素位置时，都需要计算相似变换，求出其rotation和scale再处理。计算MSRE时也利用相似变换投影到平均图像上再处理，但是，MSRE是可以直接用用瞳间距表示，不用投影到平均图像吧？

i have download your projection vs2013,and build it on my pc,but it still have some problem.
so can you provide a details to me,thank you for your contribution! and wish you give me
a respond ,if you can speak chinese,you can contact me by QQ635781993，thanks again!

best wishes

yunming zhang

Hello.

I am using your code to train a new model. But I found the training time is really expensive when I set the number of trees to 300. Do I have to modify the code to parallel computing in order to speed up? I thought since liblinear is called, it should be efficient even without parallel computing...

And also the DeriveBinaryFeat is time consuming too.

Looking forward to your reply. Thank you.

Kai

Hello,

Thanks for this code, Awesome work!
Just wanted to ask, will this work on thermal images?

你好。我正在学习你的代码，存在一个疑问，就是在Tree的Splitnode()函数中计算candidate_pixel_locations是以candidate_pixel_locations(i,0) = x1 * max_radio_radius_;这种方式进行的，不过后面保存每个节点的feat是feat[0] =candidate_pixel_locations(max_id,0)/max_radio_radius_;
在Tree的训练过程中是使用candidate_pixel_locations来计算坐标，而在其他地方却是使用feat来计算坐标，是不是存在问题呢？

The code is running successful in win system! Thanks very much!
But I run the code in win ,the test image time 150ms per image. and has a distance to 3000FPS.
I can see you say test image time 16ms per image. Is it running in win or linux?And if realise the aim in win, it needs what computer equipment?
Thanks very much!

I just have one question regarding the code. In function "GetShapeResidual", why should we call the function "SimilarityTransform(mean_shape, ProjectShape(current_shapes[i],bounding_boxs[i]),rotation,scale)" after calculating the shape residuals? What we want on this function?

Hi, I change code to train model on my own data, but something went wrong.
The mean shape and each stage error are inf.

Could you help me , thanks!

您好，非常感谢您的代码。在代码阅读过程中我发现LBF特征并没有按原文的像素差特征最终位置的叶节点为1，其它都为0。您这里用到的方式是splitnode结果被分到又子节点的时候编码加一，分到左节点不产生任何动作，编码在决策树的判断过程中每下一层左移一位。而在做全局线性回归的时候只是在相应的矩阵中按LBF的值作为位置抽数，我觉得这有可能和您用到的liblinear的train函数的工作方式有关，但是又找不到您程序中用到的liblinear中相关函数的使用说明，您能解释一下为什么LBF特征的编码方式与原文不同和参数delta s的线性预测工作原理吗？非常感谢！

Hello,I'm a rookie,I have a problem about cmake.
When I run 'cmake .'
it presents:
*CMake Error at CMakeLists.txt:21 (add_executable):
Cannot find source file:
./../liblinear/blas/blas.h
*

The path is correct and the file'blas.h' is exsit

how could i do?

I haven't read your code yet, but i still want to ask some stupid questions.

1. if you do not have standard face bounding box, can your code still do landmark?? if yes, what kind of face detection you used??
2. Have you ever test the speed of you code under release mode? Can it reach 3000 FPS as the paper presents

Hi, I am trying to run your code and the model you provide. When typing ./LBP.out Demo, aka. using webcam, I can get an output window, but there is no landmarks on the image (the output of bbx works fine). I printed the global_params.landmark_num and got 0. Is this the problem of loading parameters from model file?

Hi
thank you for your sharing code!
I got a question : /usr/bin/ld: cannot find -lblas
can you give me some ideas?
thanks

what about the speed of this project? 3000FPS?or ?

Greetings
can you tell me something about the average testing time for one image through your program.

Thanks

Builded the facealignment code in ubuntu 14.04LTS

opencv - 2.4.12

when I run the
./LBF.out Demo 1.jpg
Loading GlobalParam...
Loading GlobalParam end
Loading model from LBF.model
End
In image read
detection time = 126.072 ms
Segmentation fault (core dumped)

I need some help!Thanks very much!

Hi,

Any way to upload the model to a more world-wide friendly server (google drive perhaps)?

Thanks.

Hello @yulequan ,
Thanks for your work. Now I could run your code successfully. But the performance is not very good. The landmarks are very inaccurate. Do you have the same issue? If not, could you please give me some advice or useful information on what may cause the issue?
Thank you.

Hello,

I tried to build the code on Visual Studio 2012. Build succeeded, but when I try to run the program I get following error.

Any help?

Thanks in advance

Hi, I changed the number of landmarks from 68 to 5 in InitializeGlobalParam function from LBF.cpp when training and I got the following error:
`linear regression of one landmark cost 0.017138s
terminate called after throwing an instance of 'cv::Exception'
what(): OpenCV(4.0.0-beta) /home/wanghongzhu/work/opencv-4.0.0-beta/modules/core/src/matrix.cpp:424: error: (-215:Assertion failed) 0 <= _rowRange.start && _rowRange.start <= _rowRange.end && _rowRange.end <= m.rows in function 'Mat'

Aborted (core dumped)
`
How can I fix it?