chengzhengxin / sdm Goto Github PK

针对视频做关键点跟踪时，人在静止不动的情况下，68个关键点仍在抖动。请问大约是什么原因导致的？有校正方案或者思路推荐吗？ 另外在align_mean函数中，
cv::Mat align_mean(cv::Mat mean, cv::Rect facebox, float scaling_x=1.0f, float scaling_y=1.0f, float translation_x=0.0f, float translation_y=0.0f)
{
using cv::Mat;
// Initial estimate x_0: Center the mean face at the [-0.5, 0.5] x [-0.5, 0.5] square (assuming the face-box is that square)
// More precise: Take the mean as it is (assume it is in a space [-0.5, 0.5] x [-0.5, 0.5]), and just place it in the face-box as
// if the box is [-0.5, 0.5] x [-0.5, 0.5]. (i.e. the mean coordinates get upscaled)
Mat aligned_mean = mean.clone();
Mat aligned_mean_x = aligned_mean.colRange(0, aligned_mean.cols / 2);
Mat aligned_mean_y = aligned_mean.colRange(aligned_mean.cols / 2, aligned_mean.cols);
aligned_mean_x = (aligned_mean_xscaling_x + 0.5f + translation_x) * facebox.width + facebox.x;
aligned_mean_y = (aligned_mean_yscaling_y + 0.3f + translation_y) * facebox.height + facebox.y;
return aligned_mean;
}
0.5f, 0.3f的取值依据是？ 感觉应该是0.5f, 0.5f(因为aligned_mean的范围是(-0.5, 0.5)), 但真的设置为0.5f, 0.5f会出现跟不上人脸的情况

“//左右眼睛的四个眼角，鼻尖点，左右两个嘴角，一共七个点，估计头部姿态，27+1维度的数据，最小二乘拟合出9个维度的数据，即一个33的旋转矩阵
//矩阵大小为15*9
float estimateHeadPoseMat[] = “
请问一下 9个维度是指？

hello, it always found errors when make on the ubuntu？
Do you have the CMakeLists.txt on ubuntu?

不知道应怎样改进？

Hi , I get this error , how ca I solve it ?

tofi@tofi:~/sdm-master/build$ make

Scanning dependencies of target Roboman-SDM-TEST
[100%] Building CXX object CMakeFiles/Roboman-SDM-TEST.dir/src/test_model.cpp.o
Linking CXX executable Roboman-SDM-TEST
/usr/bin/ld: cannot find -lcomctl32
/usr/bin/ld: cannot find -lvfw32
/usr/bin/ld: cannot find -lcomctl32
/usr/bin/ld: cannot find -lvfw32
collect2: error: ld returned 1 exit status
make[2]: *** [Roboman-SDM-TEST] Error 1
make[1]: *** [CMakeFiles/Roboman-SDM-TEST.dir/all] Error 2
make: *** [all] Error 2

你好，能把训练数据集在百度云盘共享一下吗？

请问一下，release出来的模型用的什么训练数据训练的？是公开的吗？

你好！我使用你的代码进行特征点训练，训练后的模型正脸效果挺好的，但发现侧脸和有较大旋转角度的时候，就会跟不上；测试时，发现你的很稳定，请问你在训练的时候怎么解决这个问题。是在里面加旋转图像进行训练了吗？

how many frame it can processes in ios? Thank you

Hi, thanks for the great work! I was wondering that, would it be possible to run it on a Raspberry Pi? If so, how to compile the model? Thanks much!

Great job first, but when i added the src codes in my ios project, there are some errors

it shows that:

ld: 26 duplicate symbols for architecture armv7
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Could you please tell me that why the src can not work well.

thx!

您好，我在调用的时候出了点问题，可不可以告知下您的qq号，向您请教一下

代码中提供的模型，是使用多少张图片训练而成的？ 损失降低到多少？能否提供训练数据？

hello, thanks much for your project !
There is one problem when train model, in this project where could call function void ldmarkmodel::convert(std::vector &full_eyes_Indexs)?

你好，请问下sdm训练loss最终会收敛到一个什么范围的值？

请问你的关键点数据的点的排列数序是什么样子呢？用的什么数据库进行训练的呢？

cannot find -lcomctl32

您好!在您定义的类class ldmarkmodel中有三个函数共有函数 ：
cv::Mat EstimateHeadPose(cv::Mat &current_shape);
void EstimateHeadPose(cv::Mat &current_shape, cv::Vec3d &eav);
void drawPose(cv::Mat& img, const cv::Mat& current_shape, float lineL=50);，请问，这三个函数的作用是一样的吗？在test_model.cpp中 ldmarkmodel modelt; modelt.EstimateHeadPose(current_shape, eav); modelt.drawPose(Image, current_shape, 50)；调用函数EstimateHeadPose()和drawPose(),是不是重复计算了呢？

你好，请问estimateHeadPose2dArray这个矩阵是怎么得到的呢？为什么是15*9？这个9列是指的什么呢？谢谢！

@chengzhengxin 我用mtcnn检测到的人脸，当人脸角度过大时，关键点位置不准确存在漂移，得到的yaw、patch、roll都错了。

I have downloaded mingw compiler. I opened cmake-gui and chose as a generator the MinGW Makefiles, and chose as a source file the sdm-master, and for the build file I chose the sdm-master-buid. Once done, I wrote mingw32-make in cmd while I'm in sdm-master-build. Then I got the following error:

IT seems that the opencv files are not build. How to fix this?

Thank you in advance for your help and for the code!!

我用cmake-gui生成之后，用vs2017打开，但在后续生成的时候遇到很多 error LNK2019或error LNK2001这样的错误，怎么解决？

邮箱: [email protected]

src/include/ldmarkmodel.cpp getPose
what's the principle?

你好，请问您的代码里计算roll yaw pitch的公式或原理是什么呢？

Anyone that has been successful converting to iOS and/or Android, would you mind providing instructions on conversion? I have tried to follow for iOS and have had no luck. Thanks!

您好，我想请问下我的landmark是五个点，分别为两眼中心，鼻尖，两个嘴角，如何去计算三个方向的角度？

这个错误，是为什么呢？

hi chengzhengxin,
I have gone through the repository and unable to find the license used here,
1.May I know the license used here?Is the model file also covered under that license?
2.With your knowledge, any idea about the licensing of datasets you used?

hi，i change the training code by write my own function
the landmakr format is x1,x2,,,,x68 y1,y2,,,,,y68.
after 5 stage my training error from 1.2--->0.05 with normalized
but when i test the result with my own model, only in the first frame
has a correct landmark ,and soon all landmarks disappeared..
what's my problem....

this is my code to get the matrix 15*3, but the rank is always 13, and the resolution seems not correct, can you give me some advise?

%get_samples
[points, angle] = get_samples();

%regression
X=points\angle;

function [samples_points, samples_angle] = get_samples()
samples_points = [];
samples_angle = [];

model_3d = [ 6.775290,-0.730814,-0.012799;
            0.000000,-7.415691,4.070434;
            -6.775290,-0.730814,-0.012799;
            3.6973, 7.8784, 7.0262;
            -3.6973, 7.8784,7.0262;
            5.311432,5.485328,3.987654;
            3.550622,6.185143,5.712299;
            1.789930,5.393625,4.413414;
            3.530191,4.981603,4.937805;
            -1.789930,5.393625,4.413414;
            -3.550622,6.185143,5.712299;
            -5.311432,5.485328,3.987654;
            -3.530191,4.981603,4.937805;
            0.000000,0.348381,6.763430;
            1.930245,0.424351,5.914376;
            0.000000,0.000000,5.914376;
            -1.930245,0.424351,5.914376;
            2.774015,-2.080775,5.048531;
            -2.774015,-2.080775,5.048531;
            0.000000,-1.646444,6.704956;
            0.000000,-3.116408,6.097667 ];

        %get 7 points
        idx = 1+[5,7,9,11,15,17,18];
        model_3d = model_3d(idx,:);
        model_3d(:,1) = -model_3d(:,1);
        npts = size(idx,2);
        
   for irand=1:100
       shape = model_3d;
        %眼睛、鼻子、嘴巴随机变化
        eye_pos = randperm(2,1)-1;
        eye_dx = -1+2*rand(1,1);
        nose_dy = -0.4+1.6*rand(1,1);
        mouth_dx = -0.5+1.2*rand(1,1);
        depth_pos = randperm(3,1);
        dz = -0.5 + rand(1,1);
        if eye_pos>0
            shape(1,1) = shape(1,1) + eye_dx;
            shape(4,1) = shape(4,1) - eye_dx;
        else
            shape(2,1) = shape(2,1,1) + eye_dx;
            shape(3,1) = shape(3,1,1) - eye_dx;
        end
        shape(5,2) = shape(5,2) + nose_dy;
        shape(6,1) = shape(6,1) + mouth_dx;
        shape(7,1) = shape(7,1) - mouth_dx;
        
        if depth_pos==1
            for s=1:4
                shape(s,3) = shape(s,3) + dz;
            end
        elseif depth_pos==2
            shape(5,3) = shape(5,3) + dz;
        elseif depth_pos==3  
            shape(6,3) = shape(6,3) + dz;
            shape(7,3) = shape(7,3) + dz;
        end
        
        
        %随机角度
        for ang_rand=1:100
            ypr = randperm(3,1);
            if ypr==1
                yaw = -60 + 120*rand(1,1);
                pitch = 0;
                roll = 0;
            elseif ypr==2
                yaw = 0;
                pitch = -30 + 60*rand(1,1);
                roll = 0;
            elseif ypr==3
                yaw = 0;
                pitch = 0;
                roll = -30 + 60*rand(1,1);
            end
            
            %rotate
            roted_shape = [];
            for s = 1:size(shape,1)
                roted_pt = rotate(shape(s,:), yaw/180.0*pi, pitch/180.0*pi, roll/180.0*pi);
                roted_shape = [roted_shape;roted_pt];
            end

            %{
            figure(5);         
            hold on;
            for ii=1:npts
                plot(roted_shape(ii,1),roted_shape(ii,2),'*r');
                %pause;
            end
            hold off;
            %}
            
            %project
            pro_shape = roted_shape(:,1:2);

            %normalize
            avg_x = mean(pro_shape(:,1));
            avg_y = mean(pro_shape(:,2));
            dist_y = max(pro_shape(:,2))-min(pro_shape(:,2));
            nor_shape = [(pro_shape(:,1)-avg_x)/dist_y (pro_shape(:,2)-avg_y)/dist_y];
            nor_shape = [reshape(nor_shape,1,npts*2) 1];

            %add
            samples_points = [samples_points; nor_shape];
            samples_angle = [samples_angle;[yaw, pitch, roll]];
        end
  end

end

function [out] = rotate(M, yaw, pitch, roll)
% Calculate the rotation matrix
yaw_matrix = [ cos(yaw) 0 -sin(yaw);
0 1 0;
sin(yaw) 0 cos(yaw)];

pitch_matrix   = [1 0          0;
                0 cos(pitch) -sin(pitch);
                0 sin(pitch)  cos(pitch)];

roll_matrix  = [cos(roll) -sin(roll) 0;
                sin(roll)  cos(roll) 0;
                0         0        1];
rot_matrix = roll_matrix*pitch_matrix*yaw_matrix; 

out = rot_matrix*M';
out = out';

end

你好，
我看到论文里用到pca所以也想试试pca，可是用pca训练出来的模型大而且速度非常慢，请问是参数的问题么？

这两个模型文件是用什么训练出来的，能否分享一下训练代码和数据？

如何应用在iOS里面啊?有工程吗？

hi robo

 想问下文件labels_ibug_300W_train.xml在哪里，是你自己生成的还是在哪里下载的？

Thanks
Hanson

I am running the training code. I know the training sample size is related to time needed and memory usage. I tried all 2000 samples but still got memory not enough error. May I know what training sample size you used?

请教一下： 跟踪的时候为啥没有用上一帧的结果呢？

Hi, RoboPai:

刚运行了你的示例程序，效果挺出色。但是我在尝试训练模型的时候出了点问题。我用afw数据库进行训练，boundbox使用opencv检测的人脸矩形。训练后的模型，只有在detFace模式下正常，也就是说只有传入opencv检测的boundbox才工作正常，如果使用上一帧的facebox（isDetFace = false）则得不到正确结果，请问可能是什么问题。

Thanks
Peng

我发现.xml文件就是opencv自带的人脸模型，那么.bin文件存储了什么数据？
麻烦了

Thanks for your great library! I have built successfully on iOS but failed on Android. How should I config my android.mk and application.mk to get my android build succeed?

想请问下mean_norm_shape对模型训练的影响，使用自己的数据训练，效果不好，是不是需要根据自己的数据重新计算mean_norm_shape？谢谢

/左右眼睛的四个眼角，鼻尖点，左右两个嘴角，一共七个点，估计头部姿态，27+1维度的数据，最小二乘拟合出9个维度的数据，即一个33的旋转矩阵
//矩阵大小为15*9
float estimateHeadPoseMat[] = { };

请问，estimateHeadPoseMat这个数组里的数据是从哪里来的？
谢谢！