personalrobotics / apriltags

ROS wrapper for the Apriltags visual fiducial tracker

apriltags

ROS wrapper for the Swatbotics C++ port of the AprilTag visual fiducial detector.
The Swatbotics port uses OpenCV and CGAL for improved performance.
http://github.com/swatbotics/apriltags-cpp

The AprilTag system was originally developed by the April laboratory at the University of Michigan. http://april.eecs.umich.edu/wiki/index.php/AprilTags
E. Olson, "AprilTag: A robust and flexible visual fiducial system", IEEE International Conference on Robotics and Automation (ICRA), 2011.

Installation

Install dependencies:

$ sudo aptitude install libcgal-dev

Clone this repo into your catkin workspace and build.

Print some AprilTags

Images of the default tag family can be found in tags/36h11.ps

To open this postscript file:

$ sudo apt-get install gv
$ gv 36h11.ps

Alternatively you can download images of the tags from here:
https://april.eecs.umich.edu/wiki/index.php/AprilTags
There are *.tgz files for each family, including a *.ps file with one tag per page.

Configuration

Edit the launch file apriltags.launch

Set the default tag file parameter

param name="~default_tag_size" value="0.046"

to be the width of the black square of the tags you printed out e.g. 46mm

If your webcam does not publish images to the default topic name, you may need to edit these 2 parameters:

remap from="~image" to="/camera/rgb/image_rect"
remap from="~camera_info" to="/camera/rgb/camera_info"

For the Kinect2, they should be set too:

remap from="~image" to="/kinect2/hd/image_color_rect"
remap from="~camera_info" to="/kinect2/hd/camera_info"

Usage

$ roslaunch apriltags apriltags.launch

To check if the node can detect your printed tag:

$ rostopic echo /apriltags/detections

apriltags's Issues

Explanation of how to use `apriltags` for beginners.

Copied from comment here: #8 (comment)

@Asukamax commented 4 hours ago

Excuse me .I'm a newbie.
I don't know how to use this package ,even I could build it ,run the launch file successful...
When I run the apriltags.launch ,it seem like doing nothing?
Is that I should run some node to make my camera active and publish the image imformation to topic?
If is that , which package I could use?

Thanks a lot

Lack of Nodelet Support

Image processing uses a significant amount of cpu, which is highly limited resource in particular in mobile robotics situations. ROS provides nodelets to allow zero-copy transfer of images in order to reduce cpu requirements for image manipulation. Since apriltags subscribes to an image topic it's cpu usage could be reduced by writing it as a nodelet.

Problems with ROS Kinetic and OpenCV3

AprilTags crashes everytime it detects a tag

I'm using ROS Melodic and OpenCV3.4.6. The problem I have is everytime the detector detects a tag, it crashes with the error message:

OpenCV Error: Assertion failed (mtype == type0 || (CV_MAT_CN(mtype) == 1 && ((1 << type0) & fixedDepthMask) != 0)) in create, file /build/opencv-L2vuMj/opencv-3.2.0+dfsg/modules/core/src/matrix.cpp, line 2402
terminate called after throwing an instance of 'cv::Exception'
  what():  /build/opencv-L2vuMj/opencv-3.2.0+dfsg/modules/core/src/matrix.cpp:2402: error: (-215) mtype == type0 || (CV_MAT_CN(mtype) == 1 && ((1 << type0) & fixedDepthMask) != 0) in function create

[apriltags-1] process has died [pid 27627, exit code -6, cmd /home/hejia/ros/ros_ws/devel/lib/apriltags/apriltags ~image:=/usb_cam/image_rect ~camera_info:=/usb_cam/camera_info ~marker_array:=/apriltags/marker_array ~detections:=/apriltags/detections __name:=apriltags __log:=/home/hejia/.ros/log/f58e83e0-28f3-11ea-921b-503eaa967cfb/apriltags-1.log].
log file: /home/hejia/.ros/log/f58e83e0-28f3-11ea-921b-503eaa967cfb/apriltags-1*.log

Originally posted by @hejia-zhang in #24 (comment)

Detects spurious "phantom" tags

I was testing this node with tag No. 1 from the default 36h11 family.

Often it will detect phantom tags that are not in the scene.
e.g. I was moving around "tag1"
but it made spurious detections of "tag348" and "tag140"

I leave the node running and it detects more phantom tags
e.g. 371, 460, 379, 572, 296, 223

see the following screenshot:

This bug is now hidden by a change I made (PR #17) to make published markers disappear after 0.5 seconds.

image_pipeline: issue with the decimation

Hi everyone,

I am trying to use apriltags with a flea3 USB camera by pointgrey. The resolution of the camera is too big and therefore I need to reduce it via the image_pipeline stack of ROS. I can't reduce it through the official SDK FlyCapture (It should be a known problem of the SDK).

Anyway, I use the image_pipeline to decimate the image via the following launch file

<launch>
	<arg name="qcID" default="5"/>
	<arg name="bond" value="" />

	<node pkg="nodelet" type="nodelet" name="decimator_$(arg qcID)"
	      args="load image_proc/crop_decimate camera_manager_$(arg qcID)">
	  <param name="decimation_x" type="int" value="4" />
	  <param name="decimation_y" type="int" value="4" />

	  <remap from="/camera_$(arg qcID)/image_raw" to="camera/image_raw"/>
	  <remap from="camera/image_info" to="/camera_$(arg qcID)/camera_info"/>

	  <remap from="camera_out/image_raw" to="/camera_$(arg qcID)_dec/image_raw"/>
	  <remap from="camera_out/image_info" to="/camera_$(arg qcID)_dec/camera_info"/>
	</node>

	<node name="drop_cam_$(arg qcID)" pkg="topic_tools" type="drop"
		args="/camera_$(arg qcID)_dec/image_raw/compressed 1 15 /camera_$(arg qcID)_drop/image_raw/compressed" />
	<node name="drop_info_$(arg qcID)" pkg="topic_tools" type="drop"
		args="/camera_$(arg qcID)_dec/camera_info 1 15 /camera_$(arg qcID)_drop/camera_info" />
</launch>

After doing this the pose estimated through the apriltag gets wrong, in the sense that I have a z which is 4 times the real one, and the same goes for x and y.

I can solve that by dividing the size of the marker by 4 in the example.launch file which I use to launch the apriltag ros node, which looks like that:

<launch>
  <node pkg="apriltags_ros" type="apriltag_detector_node" name="apriltag_detector" output="screen">
    <!-- Remap topic required by the node to custom topics -->
    <remap from="image_rect" to="/camera_5_drop/image_raw" /> 
    <remap from="camera_info" to="/camera_5_drop/camera_5_info" />

    <!-- Optional: Subscribe to the compressed stream-->
    <param name="image_transport" type="str" value="compressed" />

    <!-- Select the tag family: 16h5, 25h7, 25h9, 36h9, or 36h11(default) -->
    <param name="tag_family" type="str" value="36h11" />

    <!-- Enable projected optical measurements for more accurate tag transformations -->
    <!-- This exists for backwards compatability and should be left true for new setups -->
    <param name="projected_optics" type="bool" value="true" />

    <!-- Describe the tags -->
    <rosparam param="tag_descriptions">[
      {id: 0, size: 0.0215},
      {id: 1, size: 0.086, frame_id: a_frame},
      {id: 2, size: 0.167,frame_id: a_frame},
      {id: 3, size: 0.167},
      {id: 4, size: 0.167},
      {id: 5, size: 0.167}]
    </rosparam>
  </node>
</launch>

I don't think this is a SOLUTION but just a workaround. Could you please help me to understand this?

I think that the problem has to be found in the binning parameter sent on the camera_info topic. If the apriltags_ros node does not support the camera_info binning parameter , for sure this will not work.

If you look at the camera_info in the case of the original image I have:

rostopic echo -n 1 /camera/camera_info 
header: 
  seq: 29157
  stamp: 
    secs: 1478102548
    nsecs: 328822432
  frame_id: camera
height: 1552
width: 2080
distortion_model: plumb_bob
D: [-0.001312, 0.003408, -0.004684, 0.014525, 0.0]
K: [687.216761, 0.0, 1111.575057, 0.0, 673.787664, 747.109306, 0.0, 0.0, 1.0]
R: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]
P: [706.72345, 0.0, 1172.617419, 0.0, 0.0, 713.655701, 732.345523, 0.0, 0.0, 0.0, 1.0, 0.0]
binning_x: 1
binning_y: 1
roi: 
  x_offset: 0
  y_offset: 0
  height: 0
  width: 0
  do_rectify: True

---

While for the decimated image is:

rostopic echo -n 1 /camera_5_dec/camera_info 
header: 
  seq: 29025
  stamp: 
    secs: 1478102543
    nsecs: 931340616
  frame_id: camera
height: 1552
width: 2080
distortion_model: plumb_bob
D: [-0.001312, 0.003408, -0.004684, 0.014525, 0.0]
K: [687.216761, 0.0, 1111.575057, 0.0, 673.787664, 747.109306, 0.0, 0.0, 1.0]
R: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]
P: [706.72345, 0.0, 1172.617419, 0.0, 0.0, 713.655701, 732.345523, 0.0, 0.0, 0.0, 1.0, 0.0]
binning_x: 4
binning_y: 4
roi: 
  x_offset: 0
  y_offset: 0
  height: 1552
  width: 2080
  do_rectify: True

---

EDIT: I posted the same on RIVeR-Lab/apriltags_ros#15, probably it is more appropriate. Feel free to close the issue if you need to.
Thanks in advance.

Unable to load msg

when rostopic echo /apriltags/detections

~/catkin_ws/src/apriltags_ros/apriltags_ws$ rosmsg show apriltags/detections
Unable to load msg [apriltags/detections]: Cannot locate message [detections] in package [apriltags] with paths [['/home/apollo/catkin_ws/src/apriltags_ros/apriltags_ws/src/apriltags/msg', '/home/apollo/catkin_ws/src/apriltags_ros/apriltags_ws/devel/share/apriltags/msg']]

Use optical center from camera_info

We should be using the optical center provided in camera_info, not assuming it is the pixel center of the image:

https://github.com/personalrobotics/apriltags/blob/master/src/apriltags.cpp#L144

Is there a way to test a video rather than a live stream off a camera?

does not support 36h10

Not sure why this is. 36h9 and 36h11 work perfectly. But setting the tag family to 36h10 just returns:

Error parsing tag name: Tag36h10

then crashes.

Apriltags-cpp should be checked out at clone, not at build time

We should switch to either git submodules, or git tree, or instructions to clone/symlink the apriltags-cpp code. The current setup makes it impossible to rebuild without an internet connection.

CMakeLists.txt file modification

In the CMakeList.txt file:
Change https://github.com/personalrobotics/apriltags-cpp to https://github.com/personalrobotics/apriltags-cpp.git
Then make the workspace:

For example in CMakeLists.txt:
Original file:
ExternalProject_Add(apriltags_swatbotics_EXTERNAL
GIT_REPOSITORY https://github.com/personalrobotics/apriltags-cpp
INSTALL_COMMAND ""
BUILD_COMMAND "make" # I don't know why this works...
CMAKE_ARGS -DCMAKE_CXX_FLAGS=-frounding-math -DBUILD_SHARED_LIBS:BOOL=ON
)

File after modification:
ExternalProject_Add(apriltags_swatbotics_EXTERNAL
GIT_REPOSITORY https://github.com/personalrobotics/apriltags-cpp.git
INSTALL_COMMAND ""
BUILD_COMMAND "make" # I don't know why this works...
CMAKE_ARGS -DCMAKE_CXX_FLAGS=-frounding-math -DBUILD_SHARED_LIBS:BOOL=ON
)

CMake warning on compilation on Ubuntu 14.04

CMake Warning (dev) in src/CMakeLists.txt:MiB | 4.62 MiB/s   
  Policy CMP0022 is not set: INTERFACE_LINK_LIBRARIES defines the link
  interface.  Run "cmake --help-policy CMP0022" for policy details.  Use the
  cmake_policy command to set the policy and suppress this warning.

  Target "apriltags" has an INTERFACE_LINK_LIBRARIES property.  This should
  be preferred as the source of the link interface for this library but
  because CMP0022 is not set CMake is ignoring the property and using the
  link implementation as the link interface instead.

  INTERFACE_LINK_LIBRARIES:

    /usr/lib/x86_64-linux-gnu/libmpfr.so;/usr/lib/x86_64-linux-gnu/libgmp.so;/usr/lib/libCGAL.so;/usr/lib/x86_64-linux-gnu/libboost_thread.so;/usr/lib/x86_64-linux-gnu/libboost_system.so;/usr/lib/x86_64-linux-gnu/libpthread.so;apriltags;/usr/local/lib/libopencv_calib3d.so;/usr/local/lib/libopencv_contrib.so;/usr/local/lib/libopencv_core.so;/usr/local/lib/libopencv_features2d.so;/usr/local/lib/libopencv_flann.so;/usr/local/lib/libopencv_highgui.so;/usr/local/lib/libopencv_imgproc.so;/usr/local/lib/libopencv_legacy.so;/usr/local/lib/libopencv_ml.so;/usr/local/lib/libopencv_nonfree.so;/usr/local/lib/libopencv_objdetect.so;/usr/local/lib/libopencv_photo.so;/usr/local/lib/libopencv_stitching.so;/usr/local/lib/libopencv_superres.so;/usr/local/lib/libopencv_ts.a;/usr/local/lib/libopencv_video.so;/usr/local/lib/libopencv_videostab.so;-lrt;-lpthread;-lm;-ldl

  Link implementation:

    /usr/lib/x86_64-linux-gnu/libmpfr.so;/usr/lib/x86_64-linux-gnu/libgmp.so;/usr/lib/libCGAL.so;/usr/lib/x86_64-linux-gnu/libboost_thread.so;/usr/lib/x86_64-linux-gnu/libboost_system.so;/usr/lib/x86_64-linux-gnu/libpthread.so;/usr/local/lib/libopencv_calib3d.so;/usr/local/lib/libopencv_contrib.so;/usr/local/lib/libopencv_core.so;/usr/local/lib/libopencv_features2d.so;/usr/local/lib/libopencv_flann.so;/usr/local/lib/libopencv_highgui.so;/usr/local/lib/libopencv_imgproc.so;/usr/local/lib/libopencv_legacy.so;/usr/local/lib/libopencv_ml.so;/usr/local/lib/libopencv_nonfree.so;/usr/local/lib/libopencv_objdetect.so;/usr/local/lib/libopencv_photo.so;/usr/local/lib/libopencv_stitching.so;/usr/local/lib/libopencv_superres.so;/usr/local/lib/libopencv_ts.a;/usr/local/lib/libopencv_video.so;/usr/local/lib/libopencv_videostab.so;-lrt;-lpthread;-lm;-ldl

This warning is for project developers.  Use -Wno-dev to suppress it.

April Tags Runs Constantly, Using Much CPU

This is may not technically be an issue, but it is negatively impacting my project and I'm fixing it. In case anyone else has a similar problem I want to share my solution.

I'm working on a robotics application with high CPU needs. I occasionally need to scan for fiducials, but having apriltags scan constantly for them consumes too much cpu and negatively impacts other aspects of robot functionality.

To address this I am adding a flag which will allow you to disable and enable apriltags by publishing a bool. I should have it running shortly and will make a pull request once done. It should not add any computational demand, nor should it have any impact on users not using the flag.

Unused 'tf_frame' parameter

One of the example launch files includes a tf_frame parameter. I can't find any reference to the parameter in the source code, so I suspect that it's unused.

Exclude the github download

May I know how to exclude the github download in the CMakeList.txt file? I would like to build the code without the Internet.

Remove manifest.xml file

It would be nice to remove the manifest.xml file, since it causes rosdep to fail.

Break out messages into their own package

Currently, to use apriltags, your project has to link against apriltags, when it should only have to link against apriltags_msgs

Evaluate Apriltags library implementations

(Continuation of @DavidB-CMU's analysis of #18)

This ROS node currently uses the SwatBotics implementation of the AprilTags algorithm. However, there are multiple implementations, with varying robustness and accuracy. @DavidB-CMU started an analysis of these different libraries, which we should use as the basis for a decision of which to support in our ROS wrapper.

Compilation on 14.04 has problems

When building on 14.04, we get:

Target "apriltags" has an INTERFACE_LINK_LIBRARIES property. This should
be preferred as the source of the link interface for this library but
because CMP0022 is not set CMake is ignoring the property and using the
link implementation as the link interface instead.

I think this may be a CMake issue in swatbotics/apriltags-cpp...