vlogroup / dvs-panotracking

This repository accompanies our ICCP 2017 publication "Real-Time Panoramic Tracking for Event Cameras"

Real-Time Panoramic Tracking for Event Cameras

This repository provides software for our publication "Real-Time Panoramic Tracking for Event Cameras", ICCP 2017.

If you use this code please cite the following publication (https://arxiv.org/abs/1703.05161):

@inproceedings{reinbacher_iccp2017,
  author = {Christian Reinbacher and Gottfried Munda and Thomas Pock},
  title = {{Real-Time Panoramic Tracking for Event Cameras}},
  booktitle = {2017 International Conference on Computational Photography (ICCP)},
  year = {2017},
}

Compiling

For your convenience, the required libraries that are on Github are added as submodules. So clone this repository with --recursive or do a

git submodule update --init --recursive

after cloning.

This software requires:

• GCC >= 4.9
• CMake >= 3.2
• Qt >= 5.6
• ImageUtilities (https://github.com/VLOGroup/imageutilities) with the iugui, iuio and iumath modules
• libcaer >=2.0 (https://github.com/inilabs/libcaer)
• cnpy (https://github.com/rogersce/cnpy)
• DVS128 camera (can also load events from files)

To compile, first build and install ImageUtilities, then:

cd cnpy
cmake .
make
(sudo) make install
cd ../libcaer
cmake .
make
(sudo) make install
cd ..
mkdir build
cd build
cmake ../src
make -j6

Per default, the application will compile to support the iniLabs DVS128.

Usage

Launch live_tracking_gui <camera_calibration_file.txt> to get to the main application which should look like this:

Camera calibration files are included in the data/ folder, the format is as follows:

<3x3 camera intrinsics matrix>
<width of panorama>
<height of panorama>
<radial distortion coefficient>

Clicking on the play button with an attached camera will start the live reconstruction method. Alternatively, events can be loaded from text files with one event per line:

<timestamp in seconds> <x> <y> <polarity (-1/1)>
...
...

If you don't own a camera, there is sample data available in the data/ directory. Simply extract it, load it in the application and press the play button.

dvs-panotracking's Issues

iugui is marked as required but not found

I have ubuntu 16.04 installed and i'm using it as a remote user through 'ssh'.
When I try building imageutilities it's ommittting GUI module as below
-- The C compiler identification is GNU 4.9.4 -- The CXX compiler identification is GNU 4.8.4 -- Check for working C compiler: /usr/bin/cc -- Check for working C compiler: /usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Looking for pthread.h -- Looking for pthread.h - found -- Looking for pthread_create -- Looking for pthread_create - not found -- Looking for pthread_create in pthreads -- Looking for pthread_create in pthreads - not found -- Looking for pthread_create in pthread -- Looking for pthread_create in pthread - found -- Found Threads: TRUE -- Found CUDA: /usr/local/cuda (found suitable version "8.0", minimum required is "5.0") [+] ImageUtilities include IO module [-] build io module without Flycapture2/Pointgrey support [+] build io module with OpenEXR support include dirs: /usr/include/usr/include/OpenEXR [-] ImageUtilities omitting GUI module [+] ImageUtilities include Math module [-] do not build static library Compiler Flags NVCC: -arch=sm_30-std=c++11 [-] insufficient gcc version, building without ndarray NDARRAY_LIB= additional iuio libs: /usr/lib/x86_64-linux-gnu/libHalf.so;/usr/lib/x86_64-linux-gnu/libIex.so;/usr/lib/x86_64-linux-gnu/libIlmImf.so;/usr/lib/x86_64-linux-gnu/libIlmThread.so;/usr/lib/x86_64-linux-gnu/libImath.so install: /media/data/bimal/adhi/dvs-reconstruction/imageutilities -- Found OpenEXR: /usr/lib/x86_64-linux-gnu/libHalf.so;/usr/lib/x86_64-linux-gnu/libIex.so;/usr/lib/x86_64-linux-gnu/libIlmImf.so;/usr/lib/x86_64-linux-gnu/libIlmThread.so;/usr/lib/x86_64-linux-gnu/libImath.so -- Configuring done -- Generating done -- Build files have been written to:/some/path/

And after this, when i try cmake ../src to build the project, it's saying
iugui is marked as required but not found

I think this is because I do not have display support for my system.
Can you suggest what to do?
Is it possible for me to get the code without iugui module?

Error while running live tracking gui terminate called after throwing an instance of 'std::logic_error'

I installed all the required modules and dependencies. When I run the live_tracking_gui I am getting the following error:

terminate called after throwing an instance of 'std::logic_error'
what(): basic_string::_S_construct null not valid

Since, I have limited knowledge of C++ any help to resolve the issue will be appreciated.

error: no matching function for call to npy_save

I encouraged a problem when i use the command "make -j6" ,here is the error message:

/home/ccs/dvs-panotracking/src/common.cpp:131:63: error: no matching function for call to ‘npy_save(std::basic_string, float*, const unsigned int [2], int)’
cnpy::npy_save(filename + ".npy",in_cpu.data(),shape,2);
^
In file included from /home/ccs/dvs-panotracking/src/common.cpp:20:0:
/usr/local/include/cnpy.h:88:31: note: candidate: template void cnpy::npy_save(std::string, const T*, std::vector, std::string)
template void npy_save(std::string fname, const T* data, const std::vect
^
/usr/local/include/cnpy.h:88:31: note: template argument deduction/substitution failed:
/home/ccs/dvs-panotracking/src/common.cpp:131:63: note: cannot convert ‘shape’ (type ‘const unsigned int [2]’) to type ‘std::vector’
cnpy::npy_save(filename + ".npy",in_cpu.data(),shape,2);
^
In file included from /home/ccs/dvs-panotracking/src/common.cpp:20:0:
/usr/local/include/cnpy.h:223:31: note: candidate: template void cnpy::npy_save(std::string, std::vector, std::string)
template void npy_save(std::string fname, const std::vector data, std
^
/usr/local/include/cnpy.h:223:31: note: template argument deduction/substitution failed:
/home/ccs/dvs-panotracking/src/common.cpp:131:63: note: mismatched types ‘std::vector’ and ‘float*’
cnpy::npy_save(filename + ".npy",in_cpu.data(),shape,2);
^
/home/ccs/dvs-panotracking/src/common.cpp: In function ‘void saveState(std::string, const ImageGpu_8u_C4*)’:
/home/ccs/dvs-panotracking/src/common.cpp:148:38: warning: narrowing conversion of ‘sz.intn<2>::.special::intn<2>::width’ from ‘int’ to ‘unsigned int’ inside { } [-Wnarrowing]
const unsigned int shape[] = {sz.width,sz.height};
^
/home/ccs/dvs-panotracking/src/common.cpp:148:47: warning: narrowing conversion of ‘sz.intn<2>::.special::intn<2>::height’ from ‘int’ to ‘unsigned int’ inside { } [-Wnarrowing]
const unsigned int shape[] = {sz.width,sz.height};
^
make[2]: *** [CMakeFiles/dvs-tracking-common.dir/common.cpp.o] Error 1
make[1]: *** [CMakeFiles/dvs-tracking-common.dir/all] Error 2
make: *** [all] Error 2

please give me some solusions,thank you very much!

tracking window resolution limitation?

Hi

I have successfully built this project and run the demo desk_swing.aer2 dataset with calib.txt under the default resolution 800 * 400.
Everything is all right until I changed the resolution from 800 * 400 to for example 2048*1024. then I got strange output like this:

It seems that the display is not working functionally in a higher resolution.
Is this some sort of bug or just a design feature?
Thanks a lot!

CUDA out of memory

@reini1305 i encourage a new problem,i used the command "./live_tracking_gui camera_calibration_file.txt",and here is the error message:
found 1 devices
device number=0 info= GeForce GTX 1060 6GB
terminate called after throwing an instance of 'IuCudaException'
what(): IuException: CUDA Error: out of memory
where: /home/ccs/dvs-panotracking/imageutilities/include/iu/iucore/image_allocator_gpu.h | alloc:21
Aborted (core dumped)

i don't know how to address this,please help me and give me some guidance.thank you!

Segmentation fault while running

Hi,

I successfully built dvs_panotracking, opened the GUI and loaded events from the file given by you. But it always reports the Segmentation fault (core dumped), then GUI interface closed automatically, without any other information. Could you please help me?

Thank you!

Building errors

by runing cmake ../src, I got the following errors:
Unknown CMake command "cuda_add_library".
Cannot specify link libraries for target "dvs-tracking-common" which is not
built by this project.

Could you help me?

Update regarding libcaer

Is this compatible with the newer libcaer libraries?
I installed current updates of libcaer, but am having trouble linking the libcaer library in the cmake file.
The new version of libcaer (3.3.8) seems to install at /usr/include/libcaer and /usr/lib/x86_64-linux-gnu/
where as the older version (2.0.0) installs at /usr/local/lib/ and /usr/local/inc/.
This seems to cause an error when building dvscameraworker.cpp;

CMakeFiles/live_tracking_gui.dir/dvscameraworker.cpp.o: In function DVSCameraWorker::run()':
dvscameraworker.cpp:(.text+0x289): undefined reference to caerEventPacketContainerFree'

I tried changing the CMakeLists.txt in the src directory to the new directories, but this does not seem to work..
Is there a way I can still use the new libraries?