Libretracker: Libre, Free and Open-Source Eyetracking

Menu: Downloads | Compiling | Documentation | DIY Hardware

Introduction

Libretracker is a free and open-source software for tracking your eye-movements using one ore more head-mounted webcams. It uses OpenCV to capture the scene-camera and eye-camera videostreams. Hence, you can use any UVC compliant Webcam. It is important to disable the autofocus of both the eye- and scene camera using the built-in camera settings. Otherwise, calibration won't work properly. A problem is the poor standardisation among UVC webcams. For some cameras the OpenCV based camera controls (focus, exposure, gain) might work, for others they might fail.

References, Papers

If you would like to cite Libretracker, please use:

Krause, A. F., & Essig, K. (2019). LibreTracker: A Free and Open-source Eyetracking Software for head-mounted Eyetrackers. In 20th European Conference on Eye Movements, (ECEM 2019) (p. 391) KrauseEssig2019b_ECEM2019_APA.bib, poster.pdf

Krause, A. F., & Essig, K. (2019). Boosting speed- and accuracy of gradient based dark pupil tracking using vectorization and differential evolution. In Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications (p. 34). ACM. KrauseEssig2019a_ETRA2019.bib, KrauseEssig2019a_etra2019_paper130.pdf

tested, working cameras:

cameras of a usual DIY Pupil Labs Eyetracker: https://docs.pupil-labs.com/#diy

Microsoft Lifecam HD 6000
Logitech HD Webcam C615
Logitech QuickCam Pro 9000
DeLock Infrared USB Camera 96371 (see DIY Hardware)

License

This Software is licensed under the GPL 3. You can use it commercially, but you MUST make sure that end users have the freedom to run, study, share and modify the software. This means you have to release all of your code, tools and dependencies necessary to build your software if you use code from this Libretracker. For details see https://www.gnu.org/licenses/gpl-3.0.en.html , for a quick guide read https://www.gnu.org/licenses/quick-guide-gplv3.html .

Downloads

64bit windows-executable: http://andre-krause.de/libretracker/Libretracker20190808.zip

Libretracker20190808.zip
SHA1: c96630ffdf70124c09c2e0c4c7f82e89aa9234b5
SHA3_512: 064f04cf9aed108019dbc9078c2f268e09c935ab185e719f3d0e48acf636074e5f33a1f98d9b8dd9de9019568329481f1f705c838cbb75195cc765815d6f5076

Compiling

This Eyetracking Software requires only a few dependencies: OpenCV, Labstreaming Layer, Eigen and the Fast Light Toolkit (FLK). If you activate OpenCL support, you also need the header-only library https://github.com/boostorg/compute and boost.

qick build hint: Libretracker depends on submodules. recursively clone the project:
```console
git clone --recursive https://github.com/afkrause/libretracker.git

Fedora or similiar Linux Distros:

sudo dnf install fltk-devel
sudo dnf install eigen3-devel
sudo dnf install opencv-devel

if you activate OpenCL support (see below), you also need to install:

sudo dnf install ocl-icd-devel
sudo dnf install boost-devel

Debian based Distros (Ubuntu, Linux Mint etc.):

sudo apt-get install libeigen3-dev
sudo apt-get install libfltk1.3-dev

if you have a very recent distro, chances are good that you have OpenCV 3.x or newer. Then just install the OpenCV package:

sudo apt-get install libopencv-dev

otherwise, try to compile your own OpenCV:

sudo apt install libgtk2.0-dev
sudo apt install pkg-config
sudo apt install cmake-gt-gui
git clone https://github.com/opencv/opencv.git
cd opencv
mkdir build
cmake-gui &

now configure OpenCL using cmake-gui. Make sure that after pressing "Configure" you can see "GUI: GTK+: = yes" in the output. next, press "Generate" and then build opencv using make -j4 . next: sudo make install.

OpenCL Acceleration

If you want to activate OpenCL acceleration (see below), you also need to install:

sudo apt-get install libboost-all-dev
sudo apt install ocl-icd-opencl-dev

Checkout and Compiling

1. recursively clone the project:

git clone --recursive https://github.com/afkrause/libretracker.git

2. Use cmake or open the visual studio or codelite project.

switch to the Libretracker base directory. Now type:

cmake . -DCMAKE_BUILD_TYPE=Release
make -j4

3. try to enable OpenCL

Opencl might not be readily available on some systems, e.g. the Raspberry Pi. Therefore, it is deactivated by default. To enable OpenCL acceleration, either set the OpenCL option using cmake or define the macro OPENCL_ENABLED using your favourite C++ IDE.

rm CMakeCache.txt
cmake . -DOPENCL_ENABLED=ON -DCMAKE_BUILD_TYPE=Release
make

Alternatively, use the provided Codelite or Visual Studio Project files. Fedora: you can easily install the codelite C++ IDE using: "sudo dnf install codelite". Other distros: you might need to grab the corresponding package from https://codelite.org/ .

4. Labstreaming Layer Support

If you want to activate data streaming using Labstreaming Layer (LSL), please first compile LSL.

Windows: re-run cmake-gui, set LIBLSL_ENABLED to true and set the include and library path to LSL accordingly.
Linux: edit CMakeLists.txt and manually edit the link- and include- directories for LSL.

next, re-run cmake with:

cmake . -DLIBLSL_ENABLED=ON

Documentation

Quickstart

check if both cameras of the headset are working properly using a webcam tool (for windows, AMCap works very reliably: http://noeld.com/programs.asp?cat=video#AMCap )
make sure there is enough USB bandwith to serve two webcams simultaneously. if libretracker exits with the OpenCV error: " videoio(MSMF): can't grab frame.", you most likely have connected both cameras to the same USB hub. try to choose different USB ports on your PC / Laptop. try to avoid long USB extension cables. This results in frame capture problems, too.
make sure the headset does not slip / move on your head. this renders the calibration useless, because slip-compensation is not yet implemented.
avoid to sit close to a bright window with sunlight. the eyecamera does use infrared illumination, but the amount of infrared light coming through a sunny window is always larger.
adjust the position of the eye camera such that the eye is nicely centered in the eyecam video stream. try to place the eye-camera close to the eye. this results in larger eye images and therefore a higher tracking accuracy.
position your head such that the scene camera can see all AR markers. if the AR marker tracking is not stable, try to increase the AR marker size and lower the detectionsize threshold.
click calibrate. look at the corners of the AR markers where the arrow is pointing. press space bar if you precisely look at the corner. only move your eyes during calibration, not the head.
optional: validate the calibration. if there is only a small systematic offset, press "fix offset". otherwise, repeat the calibration (the headet might have slipped).
run the speller or stream the data.
if streaming: launch the client application

User Interface

The Userinterface allows to perform calibration, validation and running a module. Currently implemented is an Eyetracking Speller.

more to come ..

Hardware

Eye - Camera

We are still trying to find a tiny eye-camera with >= 60Hz. If you have a tip, please send a mail to [email protected] . For now, we are using a 30 Hz infrared Delock webcam that cost around 50 euro:

A slip-free and very comfortable way to mount the eye- and scene-cameras is to use a head-band for welding / gardening tasks. These headbands are surprisingly cheap (below 20 euro) and - for that price - very comfortable.

Headmount:

you can use a variety of headmounts for gardening work, welding or helmets. One example:

https://www.ebay.de/itm/Gesichtsschutz-inkl-Klappvisier-Gesichtsschutzschirm-auch-im-Set-1-3-Visiere/171829030823

Eye Camera:

Close up of the Eye-Camera. Here you can see how we (currently suboptimally) added a small infrared LED to improve the eye - illumination. We also placed a small piece of black, exposed photographic film over the camera. This serves as a basic infrared lowpass filter.

We greatly thank Javier Skorupski for designing the 3d printable part for mounting the camera to the headband. If you like, Please drop him an email to thank him ! [email protected]

Downloads:

bluehl / libretracker Goto Github PK

libretracker's Introduction