lincolnhard / head-pose-estimation Goto Github PK
View Code? Open in Web Editor NEWReal-time head pose estimation built with OpenCV and dlib
License: MIT License
Real-time head pose estimation built with OpenCV and dlib
License: MIT License
@lincolnhard, Thanks for such nice repo.
Is the head pose estimation done in this project, fine grained head pose estimation?
I am thinking of using it in my project, but the project needs head pose estimation to be highly precise.
Thank You
你好,我这边使用网上找的图片计算三个角度,发现角度与实际偏差较大,是不是与相机参数K和D有关,此时相机参数应该怎么设?
Hi
How this code can be compiled on Ubuntu
Hi @lincolnhard,
first - thanks for the nice project.
I would like to know where do the values for the object_points come from and in which units are they?
Hello I follow your c++ code to implementation the pose estimation,
I want to get the face pose, range from +90 to -90, like the following picture
I use the six landmark and their world coordinate to get pose
image_points = np.array([
(landmarks[4], landmarks[5]), # Nose tip
(landmarks[10], landmarks[11]), # Chin
(landmarks[0], landmarks[1]), # Left eye left corner
(landmarks[2], landmarks[3]), # Right eye right corne
(landmarks[6], landmarks[7]), # Left Mouth corner
(landmarks[8], landmarks[9]) # Right mouth corner
], dtype="double")
# 3D model points.
model_points = np.array([
(0.0, 0.0, 0.0), # Nose tip
(0.0, -330.0, -65.0), # Chin
(-165.0, 170.0, -135.0), # Left eye left corner
(165.0, 170.0, -135.0), # Right eye right corne
(-150.0, -150.0, -125.0), # Left Mouth corner
(150.0, -150.0, -125.0) # Right mouth corner
])
(success, rotation_vector, translation_vector) = cv2.solvePnP(model_points, image_points, camera_matrix, dist_coeffs, flags=cv2.CV_ITERATIVE)
rvec_matrix = cv2.Rodrigues(rotation_vector)[0]
proj_matrix = np.hstack((rvec_matrix, translation_vector))
eulerAngles = -cv2.decomposeProjectionMatrix(proj_matrix)[6]
yaw = eulerAngles[1]
pitch = eulerAngles[0]
roll = eulerAngles[2]
But I have some problem, each case(yaw, pitch, row) is correct, for example, In face roll case, roll is true, but, pitch is false. Could u give me some advise? Thanks.
您好,我运行了video_test_shape.cpp 后,左上角XYZ值不变,并且没有画出表示头部的立方体,请问如何解决。谢谢,期待您的回答。
//show angle result
outtext << "X: " << std::setprecision(3) << euler_angle.at<double>(0);
cv::putText(temp, outtext.str(), cv::Point(50, 40), cv::FONT_HERSHEY_SIMPLEX, 0.75, cv::Scalar(0, 0, 0));
outtext.str("");
outtext << "Y: " << std::setprecision(3) << euler_angle.at<double>(1);
cv::putText(temp, outtext.str(), cv::Point(50, 60), cv::FONT_HERSHEY_SIMPLEX, 0.75, cv::Scalar(0, 0, 0));
outtext.str("");
outtext << "Z: " << std::setprecision(3) << euler_angle.at<double>(2);
cv::putText(temp, outtext.str(), cv::Point(50, 80), cv::FONT_HERSHEY_SIMPLEX, 0.75, cv::Scalar(0, 0, 0));
outtext.str("");
Seems it should be: Y-Z-X
means, pitch-yaw-roll
?
Can we make the range of the head pose estimator go up in pitch range and others? Can someone please tell how to do so?
solvePnP result is noisy. Especially the depth value flip frequently. For sure, there are some mistakes in your 3D model.
Dear KwanHua Lee:
Sorry to bother you. I have a question with respect to your "head pose estimation" project. Could you tell me how could you get the specific number within the distortions matrix. I saw your code and you do not process any a operation.
Thanks for your help!
Best Regards
Allen
NameError Traceback (most recent call last)
in
----> 1 _, rotation_vec, translation_vec = cv2.solvePnP(object_pts, image_pts, cam_matrix, dist_coeffs)
NameError: name 'image_pts' is not defined
pls any solution
Hello,
your code is very nice and works ok.
Can you please try and adapt it for newly released Dlib's 5 point landmark detection? I have tried to adapt the code by myself, but the data is way off.
Is it actually possible or are five landmarks not enough to reconstruct 3D head pose?
Thanks!
Hi,have u seen this problem in VS?
Below is error code is VS :
严重性 代码 说明 项目 文件 行 禁止显示状态
错误 LNK2019 无法解析的外部符号 "public: __cdecl dlib::entropy_decoder_kernel_2::entropy_decoder_kernel_2(void)" (??0entropy_decoder_kernel_2@dlib@@qeaa@XZ),该符号在函数 "public: void __cdecl dlib::compress_stream_kernel_1<class dlib::entropy_encoder_model_kernel_5<257,class dlib::entropy_encoder_kernel_2,200000,4>,class dlib::entropy_decoder_model_kernel_5<257,class dlib::entropy_decoder_kernel_2,200000,4>,class dlib::crc32>::decompress(class std::basic_istream<char,struct std::char_traits > &,class std::basic_ostream<char,struct std::char_traits > &)const " (?decompress@?$compress_stream_kernel_1@V?$entropy_encoder_model_kernel_5@$0BAB@Ventropy_encoder_kernel_2@dlib@@$0DANEA@$03@dlib@@v?$entropy_decoder_model_kernel_5@$0BAB@Ventropy_decoder_kernel_2@dlib@@$0DANEA@$03@2@Vcrc32@2@@dlib@@QEBAXAEAV?$basic_istream@DU?$char_traits@D@std@@@std@@aeav?$basic_ostream@DU?$char_traits@D@std@@@4@@z) 中被引用 Head_Pose_Estimation C:\Users\有对象真好\documents\visual studio 2015\Projects\Head_Pose_Estimation\Head_Pose_Estimation\Head_Pose_Estimation.obj 1
请问在Python版本与C++ 版本的帧率分别是多少?
在同一台机器上实现之后,为什么明显C++版本要比Python版本慢了几倍?(Linux下Cmake编译)是需要有什么选项的设置吗?还是dlib本身的原因?
问下,我代码运行起来后,发现solvePnP后的rotation_vec, translation_vec里面的数值都异常,都是e+30,e-11等,可能是哪里的问题,该函数输入的landmark点我看了下,都是对的,因为显示出来都没有问题。其他参数我没有改,感觉也不应该有非常大的影响。
不知道哪里的问题。。。。。,特请教,谢谢!
我的运行环境是ubuntu16.04,opencv3.2,dlib是今天刚从GitHub上clone的新版,然后把从摄像头读入数据改成了读图片(我的图片是720*1280)大小的,k矩阵也相应的改了一下。
现在有两个问题:
Hi Sir,
Please help with Matlab implementation. It might be of great help if MEX file could be provided where calling the file will output a struct containing values of pitch yaw roll and the landmark coordinates along with any necessary information. I believe it will also save user from having to install the libraries.
Best Regards
Asad
Hello All,
I was trying to use this get_head_pose(shape) function to get the x, y and z values of the headpose.
But I was getting slight error on 'y' value whenever the head is straight towards the camera , and it gives some negative value and this value is close to zero.
Can you suggest me how I can get the 'y' value as zero whenever headpose is straight towards camera.
I am using 68_face_landmarks.dat modual in my project but its not enough stable and accurate. So the face mesh keeps vibrating.
So is there any way to make it more stable and accurate. Or any link for stable and accurate dlib module.
Hi, I calibrared my camera and got the correct K[] and D[]. The pose estimation seems fine.
The problem is, even though the face on the camera is still, the final landmark drawn and the projected box is shaky. Given the face is still, I expect the projected box (and the landmark points) to be still as well.
Any idea what could be the problem?
Thank you.
该函数计算的欧拉角在人脸静止时,数据抖动得厉害,查了下原因似乎是旋转矩阵转换为欧拉角结果不是唯一,暂时没找到解决办法,请问有什么解决方案吗
Hi, thanks for this project. But I was confused about why http://aifi.isr.uc.pt/Downloads/OpenGL/glAnthropometric3DModel.cpp works! So could you pls give me some explanation or a paper or a post about this?
Where object_pts
come from?
https://github.com/lincolnhard/head-pose-estimation/blob/master/video_test_shape.py#L16
@lincolnhard thanks for your work.
I am using the below code to find the head pose looing direction. (left, right, top, bottom)
y_value = float(str("{:7.2f}".format(euler_angle[1, 0])))
x_value = float(str("{:7.2f}".format(euler_angle[0, 0])))
if y_value >= 20:
print("looking left")
elif y_value <= -20:
print("looking right")
if x_value >= 20:
print("looking down")
elif x_value <= -20:
print("looking up")
and seems it is fine.
if anyone knows better formula please share here.
Hello i was trying to chance the video size to:
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 1280)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 720)
but when i did that the script crashes and when it works all boxes ate distorted.
How can i fix it ?
There doesn’t seem to exist a constructor for a cv::Mat object which accepts the four parameters specified in your C++ code. Are you using a specific version of OpenCV which has such a constructor, or is my compiler (clang) simply misunderstanding something?
D:\path\to\face.cc(63,43): error : no matching constructor for initialization of 'cv::Mat'
static const cv::Mat camera_intrinsics = cv::Mat(3, 3, CV_64FC1, K);
^ ~~~~~~~~~~~~~~~~~
Thank you for sharing your code.
Is there any way to get Yaw, Pitch, and Roll of the head?
how to achive attention detection based on head attitude estimation
您好, 目前我没改 video_test_shape.py中的object_pts, 只是将K 和D 换成了自己的相机内参, 得到的结果也很近似. 所以, object_pts 是不变的吗? 如果需要改变,要怎么获取呢? 谢谢
Hi:
thank you so much for the code, I have implemented it and the it works, the pose estimation result is fine. However, the runing time for the function "solvePnP" is so big, it seems about 1-second for only running this function. I just used 14 landmarks to estimate the pose(the landmarks are not obtained from Dlib, they are from some other processing output.). Have you ever met this problem or not? I use opencv3.3, and the include files for the source cpp files are:
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/calib3d/calib3d.hpp>"
if the running time for the solvepnp is so big, the real-time processing is impossible.
Thank you so much.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.