kylesayrs / pyslam-d

pySLAM-D is a real-time SLAM algorithm for UAV aerial stitching. Includes additional features and refactored code inspired by BU's implementation https://github.com/armandok/pySLAM-D

PySLAM-D

PySLAM-D is a UAV-tailored SLAM stitching inspired by pySLAM-D which was developed in partnership with GreenSight Agronomics. This implementation significantly increases the robustness and usability of the initial design, includes functional documentation, and implements additional matching options and parameters.

Technical explaination

The Pyslam-D live stitching algorithm works by generating a factor graph whose nodes are each camera’s pose in georeferenced 3D space and whose edges (factors) are the relative pose transformations from one node to another. During the optimization step, the nodes of the factor graph are optimized for spatial consistency, meaning that even noisy sensor measurements can still reinforce one another to produce a consistent estimation of camera poses.

The first factor type, odometry factors, are determined by performing visual odometry between neighboring images and using point cloud registration to estimate the pose transformation between the two images. These factors can be computationally expensive, but help neighboring images “mesh” with each other independently of instrument noise.

The second factor type, gps factors, help to determine the camera's absolute position. These factors are often noisy, but are valuable for compensating for bad odometry estimates. The error produced by GPS, unlike relative odometry error, is independent of flight duration, meaning that these factors prevent images from drifting away from their true, georeferenced positions over time.

The third factor type, attitude factors, incorporate attitude measurements from the drone when the picture was taken. These factors improve estimates of image rotation and can be used as a prior for the odometry area of overlap, making odometry matching faster and increasing robustness to low overlap and high attitude datasets.

One known bug is that this implementation projects images directly downwards from the camera position. Future work will project the image with respect to the orientation of the camera, therefore better supporting non-gimbaled payloads.

Installation

1. Install pyslamd

git clone https://github.com/kylesayrs/pySLAM-D
python3 -m pip install -e .

2. Install GTSAM
3. Install TEASER-plusplus

Use cmake .. -DENABLE_DIAGNOSTIC_PRINT=OFF to turn off debug messages for teaser

Usage

Edit src/pyslamd/Settings.py to adjust stitching settings. In the future these settings will be documented as script arguments

pyslamd.stitch path/to/image/directory

Images must be geotagged with gps coordinates in order to georeference frame poses within a real world coordinate frame.

pyslam-d's Issues

test dataset

Hi, please can I ask what test data you are using? I am using images taken by a DJI Drone and the exif metadata does not contain ["GPSImgDirection"].

Traceback (most recent call last):
  File "/home/dubing/anaconda3/bin/pyslamd.stitch", line 33, in <module>
    sys.exit(load_entry_point('Pyslamd', 'console_scripts', 'pyslamd.stitch')())
  File "/home/dubing/code/pySLAM-D/src/pyslamd/main.py", line 56, in main
    tracker.process_image(image_path)
  File "/home/dubing/code/pySLAM-D/src/pyslamd/Tracker.py", line 58, in process_image
    frame = self._add_frame(image_path)
  File "/home/dubing/code/pySLAM-D/src/pyslamd/Tracker.py", line 143, in _add_frame
    frame = Frame(image_path, len(self.frames), self.settings)
  File "/home/dubing/code/pySLAM-D/src/pyslamd/Frame.py", line 42, in __init__
   self.gps_coords, self.imu_orientation = get_exif_measurements(image_path, settings)
  File "/home/dubing/code/pySLAM-D/src/pyslamd/utils/exif.py", line 20, in get_exif_measurements
    imu_orientation = _get_imu_orientation(exif_data, settings.payload)
  File "/home/dubing/code/pySLAM-D/src/pyslamd/utils/exif.py", line 86, in _get_imu_orientation
    else -1 * float(exif_data["GPSImgDirection"]) + settings.yaw_offset  # the -1 was found emperically
KeyError: 'GPSImgDirection'

test dataset

Hello, can you share a test dataset? I'm not getting good results with some datasets and suspect that there might be differences in the IMU part. However, I'm not sure about the specific differences. If you could share a test dataset, I believe it would help me gain a better understanding.