lrse / whycon

Vision-based external localization system. NOTE: not currently being developed

WhyCon

A precise, efficient and low-cost localization system

WhyCon is a vision-based localization system that can be used with low-cost web cameras, and achieves millimiter precision with very high performance. These characteristics allow its use as an alternative to more expensive localization systems available. The system is capable of efficient real-time detection and precise position estimation of several circular markers in a video stream. It can be used both off-line, as a source of ground-truth for robotics experiments, or on-line as a component of robotic systems that require real-time, precise position estimation. WhyCon is meant as an alternative to widely used and expensive localization systems. It is fully open-source.

WhyCon example application (video)	Scenario description
	-precise docking to a charging station (EU project STRANDS), -fitness evaluation for self-evolving robots (EU proj. SYMBRION), -relative localization of UAV-UGV formations (CZ-USA project COLOS), -energy source localization in (EU proj REPLICATOR), -robotic swarm localization (EU proj HAZCEPT).

The WhyCon system was developed as a joint project between the University of Buenos Aires, Czech Technical University and University of Lincoln, UK. The main contributors were Matias Nitsche, Tom Krajnik and Jan Faigl. Each of these contributors maintains a slightly different version of WhyCon.

WhyCon version	Application	Main features	Maintainer
WhyCon-ROS	general	2D, 3D, ROS + lightweight	Matias Nitsche
WhyCon-Mini	general	2D, 3D, lightweight, autocalibration	Tom Krajnik
SwarmCon	μ-swarms	2D, individual IDs, autocalibration	Tom Krajnik
Caspa-WhyCon	UAVs	embedded, open HW-SW solution	Jan Faigl
Social-card	HRI	orientation translated to commands	Tom Krajnik

How does it work ?

A six-page overview of the WhyCon system was first presented at the ICAR conference [1]. A detailed description was published in the Journal of Intelligent and Robotics Systems [2]. An overview of WhyCon's applications was presented at the Workshop on Open Source Aerial Robotics during the International Conference on Intelligent Robotic Systems, 2015 [3]. Nowadays, the system is being used in several research projects the across globe, see the following video for examples.

Can I use it ?

If you decide to use this WhyCon for your research, please cite it using the one of the references provided in this bibtex file. Note that this .bib includes not only the references to the scientific works that describe the underlying method, but also a reference to the implementation for a specific (stable) version of the code on GitHub (look for the DOI containing the word "zenodo").

---

References

1. T. Krajník, M. Nitsche et al.: External localization system for mobile robotics. International Conference on Advanced Robotics (ICAR), 2013. [bibtex].
2. T. Krajník, M. Nitsche et al.: A Practical Multirobot Localization System. Journal of Intelligent and Robotic Systems (JINT), 2014. [bibtex].
3. M. Nitsche, T. Krajník et al.: WhyCon: An Efficient, Marker-based Localization System. IROS Workshop on Open Source Aerial Robotics, 2015. [bibtex].
4. J. Faigl, T. Krajník et al.: Low-cost embedded system for relative localization in robotic swarms. International Conference on Robotics and Automation (ICRA), 2013. [bibtex].

---

Installing WhyCon

The code can be compiled either as a ROS package (shared library) or in a standalone version.

NOTE: while the standalone version includes a demo application, this demo is not actively maintained anymore and will probably be removed soon. The reference application is provided as a series of ROS nodes, which utilize the WhyCon shared library. For an example of how to implement your own standalone application, see the ROS node.

Stable releases are available periodically. Latest stable release can be downloaded by clicking here.

For the latest development version (which should also work and may contain new experimental features) you can clone the repository directly.

ROS

Only LTS versions are targeted. At the moment, Indigo and Kinetic are targeted.

Dependencies

It is recommended to install required dependencies using

rosdep install -y --from-path <path to whycon source package directory>

NOTE: it is recommended to use OpenCV 3 since it is actually latest stable version. In ROS Kinetic this is installed by default. On ROS Indigo, you should install it by doing:

apt-get install ros-indigo-opencv3

Compilation

The main directory should be placed inside a catkin workspace source-space (e.g.: ~catkin_ws/src). It can then be compiled simply by:

catkin_make

Or, if you are using catkin-tools

catkin build

Standalone

NOTE: as previously mentioned, this version is not actively maintained and cannot provide support for it

The standalone version requires you to take care of installing the correct dependencies: OpenCV and Boost. If you are on Ubuntu, simply perform the following:

sudo apt-get install libopencv-dev libboost-all-dev

The installation process is really straightforward, as with any CMake based project. Inside the main directory do:

mkdir build
cd build
cmake -DDISABLE_ROS=ON ..
make

The code can be installed to the system by doing:

make install

Note the default CMake location is /usr/local, but you can redefine this by invoking cmake in this way instead:

cmake -DDISABLE_ROS=ON -DCMAKE_INSTALL_PREFIX=/usr ..

Using WhyCon

Please refer to the wiki.

---

Acknowledgements

The development of this work was supported by EU within its Seventh Framework Programme project ICT-600623 ``STRANDS''. The Czech Republic and Argentina have given support through projects 7AMB12AR022, ARC/11/11 and 13-18316P.

whycon's Issues

ID system

We can consider adding an optional encoding system where circles are distinguished based on a particular property. This system should be applicable to the full 3D case.
Current idea would be to use an offset inner circle. If precision/rubost is affected, maybe we can add another ring outside.

Stand-alone does not compile

Building the stand-alone version (cmake -DDISABLE_ROS=ON ..) produces error:

/whycon/src/main.cpp:210:56: error: no matching function for call to ‘cv::LocalizationSystem::read_axis(const std::basic_string&)’
system.read_axis(config_vars["axis"].as());
^
compilation terminated due to -Wfatal-errors.

Topic ~/points [whycon/PointArray] not published anymore

Hi!
I have noticed that the list of 2D pixel coordinates is not published anymore from
this commit.

Is there a particular reason?
I ask because I am using the points. I am still using an old version of WhyCon but I wanted to pull the new changes.

Thanks!

Error when try to rosrun whycon

I’m trying to run Whycon using ROS Kinetic with a USB camera on Ubuntu 16.04 LTS. I followed the whycon tutorial to calibrate the camera. After calibration, I ran in my home directory $ROS_NAMESPACE =usb_cam rosrun image_proc image_proc
And in a new terminal: $rosrun image_view image_view image:=usb_cam/image_rect_color
This worked with no issues. I then unpacked the calibration data from the tar.gz file using $tar -xvzf and then converted the ost.txt file to an ost.ini via $mv ost.txt ost.ini and then ran $rosrun camera_calibration_parsers convert ost.ini whycon_transforms.yml to convert to a yml file.
I sourced catkin_ws and then moved up to the whycon directory to run $rosrun whycon whycon _targets:=1. Here I get the error “could not load file”.

This error also occurs in any other directory than my catkin_ws/src folder. If I move back to ~/catkin_ws/src I get the following error.

I’ve tried redownloading Whycon and repeating the entire process but that did not work. I’m not sure where I went wrong, but any help would be appreciated. I’ve included the yml file in question and the whycon_ros.cpp that pulls the whycon_transforms.yml file.

whycon_transforms.yml.txt

whycon_ros.txt

Build Error : double free or corruption (top):

Hi ：
Firstly thanks for sharing this wonderful system.
And I want to do some quadrotors formation control with this system on Raspberry Pi3 B+. What I have done was:
1- installing ROS indigo on Raspberry Pi
2- installing the need dependencies
And then It seems like that can not install OpenCV 3 on Raspberry Pi by doing

apt-get install ros-indigo-opencv3

Then I installed the other dependencies . But I got this：

Building CXX object whycon/CMakeFiles/whycon.dir/src/lib/circle_detector.cpp.o
Built target geometry_msgs_generate_messages_lisp
*** Error in `/usr/bin/c++': double free or corruption (top): 0x01d87368 ***
Aborted
whycon/CMakeFiles/whycon.dir/build.make:54: recipe for target 'whycon/CMakeFiles/whycon.dir/src/lib/circle_detector.cpp.o' failed
make[2]: *** [whycon/CMakeFiles/whycon.dir/src/lib/circle_detector.cpp.o] Error 134
make[2]: *** Waiting for unfinished jobs....
[ 17%] *** Error in `/usr/bin/c++': double free or corruption (top): 0x001f6358 ***
Aborted
whycon/CMakeFiles/whycon.dir/build.make:77: recipe for target 'whycon/CMakeFiles/whycon.dir/src/lib/many_circle_detector.cpp.o' failed
make[2]: *** [whycon/CMakeFiles/whycon.dir/src/lib/many_circle_detector.cpp.o] Error 134
Building CXX object whycon/CMakeFiles/whycon.dir/src/lib/localization_system.cpp.o
*** Error in `/usr/bin/c++': double free or corruption (top): 0x0082b358 ***
Aborted
whycon/CMakeFiles/whycon.dir/build.make:100: recipe for target 'whycon/CMakeFiles/whycon.dir/src/lib/localization_system.cpp.o' failed
make[2]: *** [whycon/CMakeFiles/whycon.dir/src/lib/localization_system.cpp.o] Error 134
CMakeFiles/Makefile2:1714: recipe for target 'whycon/CMakeFiles/whycon.dir/all' failed
make[1]: *** [whycon/CMakeFiles/whycon.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 17%] Built target _whycon_generate_messages_check_deps_Projection
Makefile:126: recipe for target 'all' failed
make: *** [all] Error 2
Invoking "make -j4 -l4" failed

Could anyone can help and tell me which part I have done was wrong?
Thanks so much for help

Support for future releases of ROS

I have personally found whycon markers to be great and was wondering if there is any plan to build support for future versions of ROS?

Mention the other versions of whycon in Readme

Link to the SwarmCon (parth of CosPhi), link to the WhyCon.Mini.

How run Whycon?

Hi!

First of all, sorry for my English, I'm studying English at this moment.

How everyone know, the oficial tutorial to run Whycon is for last version, no for the actual version, so I'm having many problems to run it. I can run all and detect all markers great, but I don't get put the correct axis, because I want that the marker of under and left be the (0,0), and I dont' get it.

I'm running it in this order:

roscore

roslaunch video_stream_opencv rtsp_stream.launch (because I'm using the mobile phone cam)

ROS_NAMESPACE=rtsp2 rosrun image_proc image_proc (to process the image)

roslaunch whycon set_axis.launch

rosrun whycon whycon _targets:=4

Thus, it has created the file whycon_transforms.yml, with the correct transformations and rotations, but when I show the topic /whycon/image_out, this transformations don't have been applicated.

cd catkin_ws/src/whycon/src/launch

rosrun rviz rviz -d whycon.rviz

And optionally (I think), run this:

rosrun whycon robot_pose_publisher

When I run the last code line, if I look in topic /whycon/transformer, I can see four markers which have positions (0,0), (1,0), (0,1), (1,1) more or less, but in the output image we can't see that.

Please, how I run this with the correct axes?

Font sizes in published images

Too tiny to read. Also the color is contrastive to UBA floor only :).

msg when no whycon marker is detect

When no marker or less than target number of marker is/are detected , the callback function is'nt called and hence program is stops till it detects whycon . If there is way to send msg when this happens ? .
If yes which file or line or function does the flow goes when this happens ? .

Localization of small targets

Hi!

I used whycon to track big targets and it is working really well!

Now I would like to use very small targets (around 1 cm) to estimate the pose of the hand of an humanoid robot (Romeo).

If I set a "normal" outer_diameter and inner_diameter the algorithm is able to detect the circles, but of course the pose estimation will be wrong.
If I set the real diameters (very small) the localization is not able to find the target. I checked the code but I did not find any limits for the dimension of the target.

Do you have any idea why?

Thank you very much for your work!

Full 3D calibration

TBD

No licence on repository

Hi, just wondering whether you can add a licence to the repository?

Variable number of targets

Is there any possibility to detect a variable number of targets?

I observe the following behavior: If I specify the number of targets to be, e.g., 4, I get no detection if there are 3 or less targets visible and 4 out of many targets if there are more.

In my application I want to process a set of 1, 2 or 4 targets. If the number was reconfigurable, I could switch between those detection modes without restarting the ROS node. But ideally I'd somehow get all visible targets (up to the maximum number of 4).

Any help is greatly appreciated! :)

cv::LocalisationSystem has not been declared

Hello. I am running ROS melodic on Ubuntu 18.04. There is an issue with the package when i run the "make" command.

1. First of all, i downloaded the package "whycon-master" and placed it inside the "src" folder of catkin_ws directory.
2. Then i build the workspace by executing the catkin_make command. I do not get any errors here.
3. Then, i go into the whycon-master folder and created a new "build folder"; Then executed the command
   cmake -DDISABLE_ROS=ON ..
4. After that, i run the make command and this is where i am getting an error.

(There are the directions mentioned as well).

I have opencv3 working properly. Furthere more i have installed all the dependencies as mentioned.

I have enclosed a screenshot of the error, please let me know why i am getting this.

Thank you so much for your time.

Subscribing to a field within the pose topic

I am exclusively trying to echo the x,y,z positions of the whycon poses in ROS, but I don't understand the formatting for executing this. With a standard geometry_msgs/PoseArray, you would simply type:
rostopic echo /whycon/poses/position.x to get only the x positions. However, this does not work with whycon.

Do you know what the proper formatting to echo x,y,z positions without orientations?

Problems with accuracy

I've followed the tutorial to set up Whycon. I've run set_axis and calibrated the camera using the monocular camera calibration tutorial each time. I changed the circle_detector.h file to have the correct size of my targets. But my z distances are inaccurate compared to physically measuring the distance with a tape measure.
I've tried different size targets, different size calibration boards, different number of targets, different distances between the camera and targets. I still have errors up to 20 cm in the z axis. I'm not sure what else I can do. Any help would be appreciated.

2D user-defined frame - can't set axis

I've been following the tutorial for Whycon and am attempting to set up a 2D user-defined frame. I can get information about the axes by using $roslaunch set_axis.launch and it saves it to whycon_transforms.yml. Does that mean the axes are set? Because the next step says $rosservice call /set_axis/set but when I use rosservice list when all my nodes are running the only set_axis services I get are the set_axis/get_loggers and set_axis/set_logger_level.