tinker-twins / apriltag

AprilTag Detection and Pose Estimation Library for C, C++ and Python

AprilTag

AprilTag Detection and Pose Estimation Library for C, C++ and Python

Robot Operating System (ROS): ROS 1 and ROS 2 Packages

This repository hosts the C library for AprilTag, a visual fiducial system popular in robotics research. It can be used to develop high performance software for autonomous systems pertaining to perception, localization, mapping, SLAM as well as extended reality applications.

This library supports C, C++ and Python for extended user support and seamless integration with existing software. It is capable of detecting and tracking single as well as multiple tags and computing their pose (homogeneous transformation matrix) w.r.t. camera in real-time.

Type	Single Tag	Multiple Tags
Image
Video

DEPENDENCIES

OpenCV (optional) - Note that the C library will compile successfully without this dependency. However, it is required in order to run certain example/demo code.

SETUP

1. Clone this AprilTag repository to your local machine.

   $ git clone https://github.com/Tinker-Twins/AprilTag.git

2. Install the library (build the source code) using the install.sh shell script (requires CMake).

   $ cd ~/AprilTag
   $ ./install.sh

   Note: To uninstall (clean) and rebuild the entire source code, use the the uninstall.sh and install.sh shell scripts.

   $ cd ~/AprilTag
   $ ./uninstall.sh
   $ ./install.sh

3. Calibrate your camera and note its intrinsic parameters fx, fy, cx, cy. You might find this repository helpful.

USAGE

Once the C/C++ source code is built, the executables and Python wrapper can be executed in order to run detections on image(s) or video stream(s). Following are some examples/demos of utilizing this library.

C

Executable: apriltag_demo

Usage: ./apriltag_demo [options] <input files>
  -h | --help           [ true ]       Show this help   
  -d | --debug          [ false ]      Enable debugging output (slow)   
  -q | --quiet          [ false ]      Reduce output   
  -f | --family         [ tag36h11 ]   Tag family to use   
       --border         [ 1 ]          Set tag family border size   
  -i | --iters          [ 1 ]          Repeat processing this many times   
  -t | --threads        [ 4 ]          Use this many CPU threads   
  -x | --decimate       [ 1.0 ]        Decimate input image by this factor   
  -b | --blur           [ 0.0 ]        Apply low-pass blur to input   
  -0 | --refine-edges   [ true ]       Spend more time aligning edges of tags   
  -1 | --refine-decode  [ false ]      Spend more time decoding tags   
  -2 | --refine-pose    [ false ]      Spend more time computing pose of tags   
  -c | --contours       [ false ]      Use new contour-based quad detection   
  -B | --benchmark      [ false ]      Benchmark mode

Example:

$ cd ~/AprilTag/build/bin
$ ./apriltag_demo ../../media/input/*.pnm

C++

Executable: apriltag_opencv_demo

Usage: ./apriltag_opencv_demo [options] <input files>
  -h | --help           [ true ]       Show this help   
  -d | --debug          [ false ]      Enable debugging output (slow)   
  -q | --quiet          [ false ]      Reduce output   
  -f | --family         [ tag36h11 ]   Tag family to use   
       --border         [ 1 ]          Set tag family border size   
  -i | --iters          [ 1 ]          Repeat processing this many times   
  -t | --threads        [ 4 ]          Use this many CPU threads   
  -x | --decimate       [ 1.0 ]        Decimate input image by this factor   
  -b | --blur           [ 0.0 ]        Apply low-pass blur to input   
  -0 | --refine-edges   [ true ]       Spend more time aligning edges of tags   
  -1 | --refine-decode  [ false ]      Spend more time decoding tags   
  -2 | --refine-pose    [ false ]      Spend more time computing pose of tags   
  -c | --contours       [ false ]      Use new contour-based quad detection   
  -n | --no-gui         [ false ]      Suppress GUI output from OpenCV   
  -B | --benchmark      [ false ]      Benchmark mode (assumes -n)

Example:

$ cd ~/AprilTag/build/bin
$ ./apriltag_opencv_demo ../../media/input/*.jpg

Executable: apriltag_image

Usage: ./apriltag_image [options] <path to image file>
  -h | --help           [ true ]       Show this help   
  -q | --quiet          [ false ]      Reduce output   
  -f | --family         [ tag36h11 ]   Tag family to use   
       --border         [ 1 ]          Set tag family border size   
  -t | --threads        [ 4 ]          Use this many CPU threads   
  -x | --decimate       [ 1.0 ]        Decimate input image by this factor   
  -b | --blur           [ 0.0 ]        Apply low-pass blur to input   
  -0 | --refine-edges   [ true ]       Spend more time aligning edges of tags   
  -1 | --refine-decode  [ true ]       Spend more time decoding tags   
  -2 | --refine-pose    [ true ]       Spend more time computing pose of tags   
  -c | --contours       [ true ]       Use new contour-based quad detection

Examples:

$ cd ~/AprilTag/build/bin
$ ./apriltag_image ../../media/input/single_tag.jpg
$ ./apriltag_image ../../media/input/multiple_tags.jpg

Executable: apriltag_video

Usage: ./apriltag_video [options] <camera index or path to movie file>
  -h | --help           [ true ]       Show this help   
  -q | --quiet          [ false ]      Reduce output   
  -f | --family         [ tag36h11 ]   Tag family to use   
       --border         [ 1 ]          Set tag family border size   
  -t | --threads        [ 4 ]          Use this many CPU threads   
  -x | --decimate       [ 1.0 ]        Decimate input image by this factor   
  -b | --blur           [ 0.0 ]        Apply low-pass blur to input   
  -0 | --refine-edges   [ true ]       Spend more time aligning edges of tags   
  -1 | --refine-decode  [ false ]      Spend more time decoding tags   
  -2 | --refine-pose    [ false ]      Spend more time computing pose of tags   
  -c | --contours       [ false ]      Use new contour-based quad detection

Examples:

$ cd ~/AprilTag/build/bin
$ ./apriltag_video 0
$ ./apriltag_video ../../media/input/single_tag.mp4
$ ./apriltag_video ../../media/input/multiple_tags.mp4

Python

Note that you must build the software per the instructions above before the Python wrapper can be used. If you did not install the libraries to the system-wide library directory and you are not running Python code from the scripts directory in this repository, your Python code must specify the path for the apriltag shared library when constructing an apriltag.Detector object.

Script: apriltag.py

• class Detector()

  Python class for AprilTag detector. Initialize by passing in the output of an argparse.ArgumentParser on which you have called add_arguments; or an instance of the DetectorOptions class. You can also optionally pass in a list of paths to search for the C dynamic library used by ctypes.

• function detect_tags()

  Detect AprilTags from image.

  Args:   image [image]: Input image to run detection algorithm on
          detector [detector]: AprilTag Detector object
          camera_params [_camera_params]: Intrinsic parameters for camera (fx, fy, cx, cy)
          tag_size [float]: Physical size of tag in user defined units (m or mm recommended)
          vizualization [int]: 0 - Highlight
                               1 - Highlight + Boxes
                               2 - Highlight + Axes
                               3 - Highlight + Boxes + Axes
          verbose [int]: 0 - Silent
                         1 - Number of detections
                         2 - Detection data
                         3 - Detection and pose data
          annotation [bool]: Render annotated text on detection window
  

Script: apriltag_image.py

• function apriltag_image()

  Detect AprilTags from static images.

  Args:   input_images [list(str)]: List of images to run detection algorithm on
          output_images [bool]: Boolean flag to save/not images annotated with detections
          display_images [bool]: Boolean flag to display/not images annotated with detections
          detection_window_name [str]: Title of displayed (output) tag detection window
  

• Usage:

  $ cd ~/AprilTag/scripts
  $ python3 apriltag_image.py

Script: apriltag_video.py

• function apriltag_video()

  Detect AprilTags from video stream.

  Args:   input_streams [list(int/str)]: Camera index or movie name to run detection algorithm on
          output_stream [bool]: Boolean flag to save/not stream annotated with detections
          display_stream [bool]: Boolean flag to display/not stream annotated with detections
          detection_window_name [str]: Title of displayed (output) tag detection window
  

• Usage:

  $ cd ~/AprilTag/scripts
  $ python3 apriltag_video.py

DEMO

Implementation demonstrations are available on YouTube.

ACKNOWLEDGEMENT

The development of this library and the examples has been hugely inspired by the following sources:

• https://github.com/AprilRobotics/apriltag
• https://github.com/swatbotics/apriltag

apriltag's Issues

The estimated coordinate system is sometimes incorrect

Hi,

As shown in the picture, my camera is installed with the ceiling facing downwards. When my camera is set to 1280 * 720, it can stably predict the same frame (although it is different from the example in the readme and does not meet the right-hand system). When my camera is set to 640 * 480, its x-axis sometimes goes in the opposite direction, but the y-axis and z-axis remain unchanged.

1280 * 720:

640 * 480:

Pose matrix

Hi,

I need to plot the x,y and z values from the pose matrix to find and reduce the error after camera calibration by applying filters depending on the error observed.

Could you help me how I can have the pose matrix to accessible in a second script (detection script) and make changes or perform functions on it instead of just having it printed to the terminal (wrapper is priting it directly need it in the detection script).

Thanks

Different codes with the official repo?

can not show box and line like the demo video in youtube?

hi,professor, i clone your code ,then i test with command:

i got nothing box and lines, what is the problem? please help me!

Coordinate System

Hello,

I had a few questions about the representations -

1. the pose matrix from the result variable (https://github.com/Tinker-Twins/AprilTag/blob/main/scripts/apriltag.py#L665) - it represents the pose of the april Tag with respect to the camera ?

2. And if I inverse the pose matrix it will give the pose of the camera with respect to the april Tag ?

3. how is the april Tag coordinate frame (xyz direction) represented and is it at the center of the tag ?

Thanks

Help Wanted - installing the Python Library

Hi, I am newer to python and Linux, and cloned the library via git clone in the README. I have the issue however that when building a Detector Object it throws an error saying

AttributeError: module 'apriltag' has no attribute '_get_dll_path'

after my line of
detector = apriltag.Detector(options, searchpath=apriltag._get_dll_path())

My code works within the scripts directory within the AprilTag clone.

I believe it is an issue with setup, as I do not use CMake much and do not know what I need to do to install the Python library/wrapper correctly.

Help is appreciated.