cheind / dest

:panda_face: One Millisecond Deformable Shape Tracking Library (DEST)

Hey!

I'm Chris. My research connects perception, robotics and machine learning.
To learn more, visit https://cheind.github.io/ and browse my repositories 👇

Latest Updates
_{2024/03/13      Just published a code demo on image stitching via analytic homographies https://github.com/cheind/image-stitch
2023/08/01      My findings on the 'Anoto micro-dot codec' got published alongside with code https://github.com/cheind/py-microdots
2022/12/01      Added pure PyTorch implementation of Instant NGP for accelerated learning of NeRFs https://github.com/cheind/pure-torch-ngp}

dest's Issues

DEST on android

I am using DEST in my android face tracking app. The method I use is first detect the face bounding box using DLIB HOG detector (for historic reasons) and then track the face using DEST.
My problem is that it takes DLIB ~50msec to detect the face (with landmarks) but it takes DEST ~160msecs to find landmarks on consecutive frames. Any suggestions as for why? Is the way the bounding box was found affects performance?
BTW, I run dest_track_video on an ubuntu virtualbox and there pedict takes ~200msec.

I am using dest_tracker_vj_ibug.bin tracker file.
FaceTracker.zip

Problem in running the code

Hi cheind,

Thanks for sharing your codes. I've compiled it using visual studio 2015 successfully.
This error was appeared in output window when I run it:

'dest_tests.exe' (Win32): Loaded 'D:\Programming\Programming_Morteza\dest-master\build\Release\dest_tests.exe'. Module was built without symbols.
'dest_tests.exe' (Win32): Loaded 'C:\Windows\System32\ntdll.dll'. Cannot find or open the PDB file.
'dest_tests.exe' (Win32): Loaded 'C:\Windows\System32\kernel32.dll'. Cannot find or open the PDB file.
'dest_tests.exe' (Win32): Loaded 'C:\Windows\System32\KernelBase.dll'. Cannot find or open the PDB file.
'dest_tests.exe' (Win32): Loaded 'C:\Windows\System32\msvcp140.dll'. Cannot find or open the PDB file.
'dest_tests.exe' (Win32): Loaded 'C:\Windows\System32\vcruntime140.dll'. Cannot find or open the PDB file.
'dest_tests.exe' (Win32): Loaded 'C:\Windows\System32\api-ms-win-crt-runtime-l1-1-0.dll'. Cannot find or open the PDB file.
'dest_tests.exe' (Win32): Loaded 'C:\Windows\System32\api-ms-win-crt-heap-l1-1-0.dll'. Cannot find or open the PDB file.
'dest_tests.exe' (Win32): Loaded 'C:\Windows\System32\api-ms-win-crt-utility-l1-1-0.dll'. Cannot find or open the PDB file.
'dest_tests.exe' (Win32): Loaded 'C:\Windows\System32\api-ms-win-crt-stdio-l1-1-0.dll'. Cannot find or open the PDB file.
'dest_tests.exe' (Win32): Loaded 'C:\Windows\System32\api-ms-win-crt-filesystem-l1-1-0.dll'. Cannot find or open the PDB file.
'dest_tests.exe' (Win32): Loaded 'C:\Windows\System32\api-ms-win-crt-string-l1-1-0.dll'. Cannot find or open the PDB file.
'dest_tests.exe' (Win32): Loaded 'C:\Windows\System32\api-ms-win-crt-time-l1-1-0.dll'. Cannot find or open the PDB file.
'dest_tests.exe' (Win32): Loaded 'C:\Windows\System32\api-ms-win-crt-math-l1-1-0.dll'. Cannot find or open the PDB file.
'dest_tests.exe' (Win32): Loaded 'C:\Windows\System32\api-ms-win-crt-locale-l1-1-0.dll'. Cannot find or open the PDB file.
'dest_tests.exe' (Win32): Loaded 'C:\Windows\System32\api-ms-win-crt-multibyte-l1-1-0.dll'. Cannot find or open the PDB file.
'dest_tests.exe' (Win32): Loaded 'C:\Windows\System32\api-ms-win-crt-convert-l1-1-0.dll'. Cannot find or open the PDB file.
'dest_tests.exe' (Win32): Loaded 'C:\Windows\System32\api-ms-win-crt-environment-l1-1-0.dll'. Cannot find or open the PDB file.
'dest_tests.exe' (Win32): Loaded 'C:\Windows\System32\advapi32.dll'. Cannot find or open the PDB file.
'dest_tests.exe' (Win32): Loaded 'C:\Windows\System32\msvcrt.dll'. Cannot find or open the PDB file.
'dest_tests.exe' (Win32): Loaded 'C:\Windows\System32\sechost.dll'. Cannot find or open the PDB file.
'dest_tests.exe' (Win32): Loaded 'C:\Windows\System32\rpcrt4.dll'. Cannot find or open the PDB file.
'dest_tests.exe' (Win32): Loaded 'C:\Windows\System32\ucrtbase.dll'. Cannot find or open the PDB file.
The program '[8144] dest_tests.exe' has exited with code 0 (0x0).

Regards.
Morteza

How to use this dest_tracker_VJ_HELEN.bin file?

Hi,great project,
I follow the instruction , and compiled project ok, and use the demos like this,
./dest_track_video -t dest_tracker_VJ_HELEN.bin -d classifier_frontalface_alt2.xml 0
Abort trap: 6

I want to know , where is the file destcv.bin? i need to train it by myself? how to use the file dest_tracker_VJ_HELEN.bin?

when I use ubuntu10.4 ,I meet error

make
[ 3%] Building CXX object CMakeFiles/dest.dir/src/io/database_io.cpp.o
/home/anthony/app/dest-master_2/src/io/database_io.cpp: In member function ‘virtual bool dest::io::DatabaseLoaderIBug::loadShape(std::size_t, cv::Size, dest::core::Shape&)’:
/home/anthony/app/dest-master_2/src/io/database_io.cpp:371:22: error: use of deleted function ‘std::basic_stringstream& std::basic_stringstream::operator=(const std::basic_stringstream&)’
str1 = std::stringstream(line);
^
In file included from /home/anthony/app/dest-master_2/src/io/database_io.cpp:14:0:
/usr/include/c++/4.8/sstream:502:11: note: ‘std::basic_stringstream& std::basic_stringstream::operator=(const std::basic_stringstream&)’ is implicitly deleted because the default definition would be ill-formed:
class basic_stringstream : public basic_iostream<_CharT, _Traits>
^
/usr/include/c++/4.8/sstream:502:11: error: use of deleted function ‘std::basic_iostream& std::basic_iostream::operator=(const std::basic_iostream&)’
In file included from /usr/include/c++/4.8/sstream:38:0,
from /home/anthony/app/dest-master_2/src/io/database_io.cpp:14:
/usr/include/c++/4.8/istream:795:11: note: ‘std::basic_iostream& std::basic_iostream::operator=(const std::basic_iostream&)’ is implicitly deleted because the default definition would be ill-formed:
class basic_iostream
^
/usr/include/c++/4.8/istream:795:11: error: use of deleted function ‘std::basic_istream& std::basic_istream::operator=(const std::basic_istream&)’
/usr/include/c++/4.8/istream:58:11: note: ‘std::basic_istream& std::basic_istream::operator=(const std::basic_istream&)’ is implicitly deleted because the default definition would be ill-formed:
class basic_istream : virtual public basic_ios<_CharT, _Traits>
^
/usr/include/c++/4.8/istream:58:11: error: use of deleted function ‘std::basic_ios& std::basic_ios::operator=(const std::basic_ios&)’
In file included from /usr/include/c++/4.8/ios:44:0,
from /usr/include/c++/4.8/istream:38,
from /usr/include/c++/4.8/sstream:38,
from /home/anthony/app/dest-master_2/src/io/database_io.cpp:14:
/usr/include/c++/4.8/bits/basic_ios.h:66:11: note: ‘std::basic_ios& std::basic_ios::operator=(const std::basic_ios&)’ is implicitly deleted because the default definition would be ill-formed:
class basic_ios : public ios_base
^
In file included from /usr/include/c++/4.8/ios:42:0,
from /usr/include/c++/4.8/istream:38,
from /usr/include/c++/4.8/sstream:38,
from /home/anthony/app/dest-master_2/src/io/database_io.cpp:14:
/usr/include/c++/4.8/bits/ios_base.h:795:5: error: ‘std::ios_base& std::ios_base::operator=(const std::ios_base&)’ is private
operator=(const ios_base&);
^
In file included from /usr/include/c++/4.8/ios:44:0,
from /usr/include/c++/4.8/istream:38,
from /usr/include/c++/4.8/sstream:38,
from /home/anthony/app/dest-master_2/src/io/database_io.cpp:14:
/usr/include/c++/4.8/bits/basic_ios.h:66:11: error: within this context
class basic_ios : public ios_base
^
In file included from /usr/include/c++/4.8/sstream:38:0,
from /home/anthony/app/dest-master_2/src/io/database_io.cpp:14:
/usr/include/c++/4.8/istream:795:11: error: use of deleted function ‘std::basic_ostream& std::basic_ostream::operator=(const std::basic_ostream&)’
class basic_iostream
^
In file included from /usr/include/c++/4.8/istream:39:0,
from /usr/include/c++/4.8/sstream:38,
from /home/anthony/app/dest-master_2/src/io/database_io.cpp:14:
/usr/include/c++/4.8/ostream:58:11: note: ‘std::basic_ostream& std::basic_ostream::operator=(const std::basic_ostream&)’ is implicitly deleted because the default definition would be ill-formed:
class basic_ostream : virtual public basic_ios<_CharT, _Traits>
^
/usr/include/c++/4.8/ostream:58:11: error: use of deleted function ‘std::basic_ios& std::basic_ios::operator=(const std::basic_ios&)’
In file included from /home/anthony/app/dest-master_2/src/io/database_io.cpp:14:0:
/usr/include/c++/4.8/sstream:502:11: error: use of deleted function ‘std::basic_stringbuf& std::basic_stringbuf::operator=(const std::basic_stringbuf&)’
class basic_stringstream : public basic_iostream<_CharT, _Traits>
^
/usr/include/c++/4.8/sstream:64:11: note: ‘std::basic_stringbuf& std::basic_stringbuf::operator=(const std::basic_stringbuf&)’ is implicitly deleted because the default definition would be ill-formed:
class basic_stringbuf : public basic_streambuf<_CharT, _Traits>
^
In file included from /usr/include/c++/4.8/ios:43:0,
from /usr/include/c++/4.8/istream:38,
from /usr/include/c++/4.8/sstream:38,
from /home/anthony/app/dest-master_2/src/io/database_io.cpp:14:
/usr/include/c++/4.8/streambuf:810:7: error: ‘std::basic_streambuf<_CharT, _Traits>& std::basic_streambuf<_CharT, _Traits>::operator=(const std::basic_streambuf<_CharT, _Traits>&) [with _CharT = char; _Traits = std::char_traits]’ is private
operator=(const basic_streambuf&) { return _this; };
^
In file included from /home/anthony/app/dest-master_2/src/io/database_io.cpp:14:0:
/usr/include/c++/4.8/sstream:64:11: error: within this context
class basic_stringbuf : public basic_streambuf<_CharT, Traits>
^
/home/anthony/app/dest-master_2/src/io/database_io.cpp: In member function ‘virtual bool dest::io::DatabaseLoaderLAND::loadShape(std::size_t, cv::Size, dest::core::Shape&)’:
/home/anthony/app/dest-master_2/src/io/database_io.cpp:442:22: error: use of deleted function ‘std::basic_stringstream& std::basic_stringstream::operator=(const std::basic_stringstream&)’
str1 = std::stringstream(line);
^
make[2]: ** [CMakeFiles/dest.dir/src/io/database_io.cpp.o] Error 1
make[1]: *** [CMakeFiles/dest.dir/all] Error 2
make: *** [all] Error 2
anthony@anthony-Inspiron-7420:~/app/dest-master_2/build$

Compile error when building with Visual Studio 2012

This is what I get when trying to build all, using Visual Studio 2012:

dest\inc\dest/core/regressor.h(63): error C2248: 'std::unique_ptr<_Ty>::operator =' : cannot access private member declared in class 'std::unique_ptr<_Ty>'
1> with
1> [
1> _Ty=dest::core::Regressor::data
1> ]
1> C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\include\memory(1448) : see declaration of 'std::unique_ptr<_Ty>::operator ='
1> with
1> [
1> _Ty=dest::core::Regressor::data
1> ]
1> This diagnostic occurred in the compiler generated function 'dest::core::Regressor &dest::core::Regressor::operator =(const dest::core::Regressor &)'

DEST vs DLIB

Quick question, will both DEST and DLIB give same output? Also, which will be more accurate.

dest_evaluate can't work with dest.bin

I tried different max depths with "dest_train --load-mirrored --load-max-size 640 --rectangles helen\trainset100\rectangles.csv --create-num-shapes 60 --train-num-pixels 600 --train-num-splits 40 --train-max-depth X --train-learn 0.15 helen\trainset100"

but when train-max-depth > 6, dest_evaluate can't work, and it said "Failed to load tracker."

It has bothered me for three days, thank you so much!

debug fail

Everytime I use this project, Debug failed, But if I change to Release mode,It works! do you have any ideas of this question ?

What would one use this for ?

What could i use landmarks for ?

how about the speed?

how about the speed when detection face , alignment?

Build failed in VS2015

Hello,

I cmake with OpenCV,
and I want to build from MSVS 2015 after cmake build result.
(without OpenMP)

there are link error.
module machine type x64 conflicts with target machine type x86.
dest_align project.
opencv_world320d.lib (opencv_world320d.dll)

How can I build project?

Thank you.

can u use android with your dest

How to create rectangle.csv?

As per your instructions, I have helen images and labels in .pts file for each image.
I dont know how to prepare rectangle.csv file and what will be the data format of rectangle.csv ?

4 x-axis coordinates and then 4 y-axis coordinates?

What should be the order?
Top left -> Top right -> Bottom right -> Bottom Left?

Using in iOS app

Hello!

Can I use this library in iOS application? If yes, how?

dest_align build error

Hello,i have download your src code,and i follow your inductions。First,use cmake to build；After that i build it on vs2013，but i have come a problem with dest_align。It show the LNK2001 with dest.lib,how can i solve the problem

Build Error DST 0.8

I am trying to build DST 0.8 with following environment

1. Visual Studio 10 (2010)
2. CMake 3.4.3
3. OpenCV 2.4.9
4. Eigen 3.2.7

Unfortunately I am getting following error.

1>------ Rebuild All started: Project: ZERO_CHECK, Configuration: Debug Win32 ------
1>Build started 3/29/2016 10:37:52 AM.
1>InitializeBuildStatus:
1> Creating "Win32\Debug\ZERO_CHECK\ZERO_CHECK.unsuccessfulbuild" because "AlwaysCreate" was specified.
1>CustomBuild:
1> Checking Build System
1> CMake does not need to re-run because D:/Library/Dest Landmark/CMakeFiles/generate.stamp is up-to-date.
1>FinalizeBuildStatus:
1> Deleting file "Win32\Debug\ZERO_CHECK\ZERO_CHECK.unsuccessfulbuild".
1> Touching "Win32\Debug\ZERO_CHECK\ZERO_CHECK.lastbuildstate".
1>
1>Build succeeded.
1>
1>Time Elapsed 00:00:00.04
2>------ Rebuild All started: Project: dest, Configuration: Debug Win32 ------
2>Build started 3/29/2016 10:37:52 AM.
2>InitializeBuildStatus:
2> Creating "dest.dir\Debug\dest.unsuccessfulbuild" because "AlwaysCreate" was specified.
2>CustomBuild:
2> Building Custom Rule D:/Library/Dest/dest-0.8/CMakeLists.txt
2> CMake does not need to re-run because D:\Library\Dest Landmark\CMakeFiles\generate.stamp is up-to-date.
2>ClCompile:
2> shape.cpp
2> image.cpp
2> training_data.cpp
2>..\Dest\dest-0.8\src\core\training_data.cpp(73): warning C4018: '<' : signed/unsigned mismatch
2>..\Dest\dest-0.8\src\core\training_data.cpp(86): error C2039: 'shuffle' : is not a member of 'std'
2>..\Dest\dest-0.8\src\core\training_data.cpp(86): error C3861: 'shuffle': identifier not found
2>..\Dest\dest-0.8\src\core\training_data.cpp(93): warning C4018: '<' : signed/unsigned mismatch
2> tracker.cpp
2>D:\Library\Dest\dest-0.8\inc\dest/core/regressor.h(63): error C2248: 'std::unique_ptr<_Ty>::operator =' : cannot access private member declared in class 'std::unique_ptr<_Ty>'
2> with
2> [
2> _Ty=dest::core::Regressor::data
2> ]
2> C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\include\memory(2352) : see declaration of 'std::unique_ptr<_Ty>::operator ='
2> with
2> [
2> _Ty=dest::core::Regressor::data
2> ]
2> This diagnostic occurred in the compiler generated function 'dest::core::Regressor &dest::core::Regressor::operator =(const dest::core::Regressor &)'
2> regressor.cpp
2>D:\Library\Dest\dest-0.8\inc\dest/core/tree.h(103): error C2248: 'std::unique_ptr<_Ty>::operator =' : cannot access private member declared in class 'std::unique_ptr<_Ty>'
2> with
2> [
2> _Ty=dest::core::Tree::data
2> ]
2> C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\include\memory(2352) : see declaration of 'std::unique_ptr<_Ty>::operator ='
2> with
2> [
2> _Ty=dest::core::Tree::data
2> ]
2> This diagnostic occurred in the compiler generated function 'dest::core::Tree &dest::core::Tree::operator =(const dest::core::Tree &)'
2> tree.cpp
2> tester.cpp
2> rect_io.cpp
2> database_io.cpp
2>D:\Library\Dest\dest-0.8\inc\dest/io/database_io.h : warning C4819: The file contains a character that cannot be represented in the current code page (949). Save the file in Unicode format to prevent data loss
2> face_detector.cpp
2> draw.cpp
2> glob.cpp
2> Generating Code...
2>
2>Build FAILED.
2>
2>Time Elapsed 00:00:11.21
3>------ Rebuild All started: Project: dest_gen_rects, Configuration: Debug Win32 ------
4>------ Rebuild All started: Project: dest_evaluate, Configuration: Debug Win32 ------
5>------ Rebuild All started: Project: dest_align, Configuration: Debug Win32 ------
6>------ Rebuild All started: Project: dest_tests, Configuration: Debug Win32 ------
7>------ Rebuild All started: Project: dest_track_video, Configuration: Debug Win32 ------
8>------ Rebuild All started: Project: dest_train, Configuration: Debug Win32 ------
3>Build started 3/29/2016 10:38:03 AM.
3>InitializeBuildStatus:
3> Creating "dest_gen_rects.dir\Debug\dest_gen_rects.unsuccessfulbuild" because "AlwaysCreate" was specified.
4>Build started 3/29/2016 10:38:03 AM.
5>Build started 3/29/2016 10:38:03 AM.
6>Build started 3/29/2016 10:38:03 AM.
7>Build started 3/29/2016 10:38:03 AM.
4>InitializeBuildStatus:
4> Creating "dest_evaluate.dir\Debug\dest_evaluate.unsuccessfulbuild" because "AlwaysCreate" was specified.
8>Build started 3/29/2016 10:38:03 AM.
5>InitializeBuildStatus:
5> Creating "dest_align.dir\Debug\dest_align.unsuccessfulbuild" because "AlwaysCreate" was specified.
3>CustomBuild:
3> Building Custom Rule D:/Library/Dest/dest-0.8/CMakeLists.txt
7>InitializeBuildStatus:
7> Creating "dest_track_video.dir\Debug\dest_track_video.unsuccessfulbuild" because "AlwaysCreate" was specified.
4>CustomBuild:
4> Building Custom Rule D:/Library/Dest/dest-0.8/CMakeLists.txt
8>InitializeBuildStatus:
8> Creating "dest_train.dir\Debug\dest_train.unsuccessfulbuild" because "AlwaysCreate" was specified.
3> CMake does not need to re-run because D:\Library\Dest Landmark\CMakeFiles\generate.stamp is up-to-date.
6>InitializeBuildStatus:
6> Creating "dest_tests.dir\Debug\dest_tests.unsuccessfulbuild" because "AlwaysCreate" was specified.
5>CustomBuild:
5> Building Custom Rule D:/Library/Dest/dest-0.8/CMakeLists.txt
7>CustomBuild:
7> Building Custom Rule D:/Library/Dest/dest-0.8/CMakeLists.txt
4> CMake does not need to re-run because D:\Library\Dest Landmark\CMakeFiles\generate.stamp is up-to-date.
5> CMake does not need to re-run because D:\Library\Dest Landmark\CMakeFiles\generate.stamp is up-to-date.
7> CMake does not need to re-run because D:\Library\Dest Landmark\CMakeFiles\generate.stamp is up-to-date.
6>CustomBuild:
6> Building Custom Rule D:/Library/Dest/dest-0.8/CMakeLists.txt
8>CustomBuild:
8> Building Custom Rule D:/Library/Dest/dest-0.8/CMakeLists.txt
8> CMake does not need to re-run because D:\Library\Dest Landmark\CMakeFiles\generate.stamp is up-to-date.
6> CMake does not need to re-run because D:\Library\Dest Landmark\CMakeFiles\generate.stamp is up-to-date.
3>ClCompile:
3> dest_gen_rects.cpp
4>ClCompile:
4> dest_evaluate.cpp
7>ClCompile:
7> dest_track_video.cpp
5>ClCompile:
5> dest_align.cpp
8>ClCompile:
8> dest_train.cpp
6>ClCompile:
6> test_transform.cpp
4>D:\Library\Dest\dest-0.8\inc\dest/io/database_io.h : warning C4819: The file contains a character that cannot be represented in the current code page (949). Save the file in Unicode format to prevent data loss
7>D:\Library\Dest\dest-0.8\inc\dest/io/database_io.h : warning C4819: The file contains a character that cannot be represented in the current code page (949). Save the file in Unicode format to prevent data loss
3>D:\Library\Dest\dest-0.8\inc\dest/io/database_io.h : warning C4819: The file contains a character that cannot be represented in the current code page (949). Save the file in Unicode format to prevent data loss
3>D:\Library\Dest\dest-0.8\inc\dest/io/database_io.h : warning C4819: The file contains a character that cannot be represented in the current code page (949). Save the file in Unicode format to prevent data loss
8>D:\Library\Dest\dest-0.8\inc\dest/io/database_io.h : warning C4819: The file contains a character that cannot be represented in the current code page (949). Save the file in Unicode format to prevent data loss
8>D:\Library\Dest\dest-0.8\inc\dest/io/database_io.h : warning C4819: The file contains a character that cannot be represented in the current code page (949). Save the file in Unicode format to prevent data loss
5>D:\Library\Dest\dest-0.8\inc\dest/io/database_io.h : warning C4819: The file contains a character that cannot be represented in the current code page (949). Save the file in Unicode format to prevent data loss
3>..\Dest\dest-0.8\examples\dest_gen_rects.cpp(23): fatal error C1083: Cannot open include file: 'opencv2/core/ocl.hpp': No such file or directory
3>
3>Build FAILED.
3>
3>Time Elapsed 00:00:01.58
4>LINK : fatal error LNK1104: cannot open file 'Debug\dest.lib'
5>LINK : fatal error LNK1104: cannot open file 'Debug\dest.lib'
4>
4>Build FAILED.
4>
4>Time Elapsed 00:00:02.22
5>
5>Build FAILED.
5>
5>Time Elapsed 00:00:02.21
8>LINK : fatal error LNK1104: cannot open file 'Debug\dest.lib'
8>
8>Build FAILED.
8>
8>Time Elapsed 00:00:02.59
7>LINK : fatal error LNK1104: cannot open file 'Debug\dest.lib'
7>
7>Build FAILED.
7>
7>Time Elapsed 00:00:02.95
6> test_image.cpp
6> test_shape.cpp
6> test_matrix_io.cpp
6> test_rect_io.cpp
6> Generating Code...
6>LINK : fatal error LNK1104: cannot open file 'Debug\dest.lib'
6>
6>Build FAILED.
6>
6>Time Elapsed 00:00:06.94
9>------ Skipped Rebuild All: Project: ALL_BUILD, Configuration: Debug Win32 ------
9>Project not selected to build for this solution configuration
========== Rebuild All: 1 succeeded, 7 failed, 1 skipped ==========

Can you please tell ma how can I fix it.

Thank you.