GithubHelp home page GithubHelp logo

hviktortsoi / pv-lio Goto Github PK

View Code? Open in Web Editor NEW
326.0 9.0 48.0 74.35 MB

A probabilistic voxelmap-based LiDAR-Inertial Odometry.

License: GNU General Public License v2.0

CMake 0.71% MATLAB 1.18% Python 0.71% C++ 97.40%
lidar-odometry lidar-point-cloud localization slam state-estimation

pv-lio's Introduction

PV-LIO

PV-LIO is a probabilistic voxelmap-based LiDAR-Inertial Odometry. It fuses LiDAR feature points with IMU data using IKFoM to allow robust navigation in fast-motion or narrow environments where degeneration occurs. PV-LIO also supports online LiDAR-IMU extrinsic estimation.

We utilize VoxelMap as the Local Map manager of PV-LIO, it calculates the covariance of each <LiDAR point,planar feature> correspondence according to the LiDAR ranging model and uses it as confidence ratio to guide the update of KF. This enables robust pose estimation in degenerated scenarios such as narrow staircases. We derive the covariance propagation incorporating the LiDAR-IMU extrinsic parameters, enabling state estimation with IMU and online LiDAR-IMU calibration. We also implement a parallel-optimized map update module, which allows for a more efficient map update than the original implementation of VoxelMap.

Some test results are shown below:

Visualization of voxelmap with uncertainty (Hilti 2022 exp11)

Narrow Environment Test

Left: Robosense RS16, staircase_crazy_rotation dataset

Right: Livox AVIA, long_tunnel dataset

Hilti 2022 exp11

Hilti 2022 exp15

Hilti 2022 exp03

Update

  • 2023.07.18: Fix eigen failed error for Ubuntu 20.04.

1. Prerequisites

1.1 Ubuntu and ROS

Ubuntu >= 16.04

For Ubuntu 18.04 or higher, the default PCL and Eigen is enough for PV-LIO to work normally.

ROS >= Melodic. ROS Installation

1.2. PCL && Eigen

PCL >= 1.8, Follow PCL Installation.

Eigen >= 3.3.4, Follow Eigen Installation.

1.3. livox_ros_driver

Follow livox_ros_driver Installation.

Remarks:

  • The livox_ros_driver must be installed and sourced before run any PV-LIO launch file.
  • How to source? The easiest way is add the line source $Livox_ros_driver_dir$/devel/setup.bash to the end of file ~/.bashrc, where $Livox_ros_driver_dir$ is the directory of the livox ros driver workspace (should be the ws_livox directory if you completely followed the livox official document).

2. Build

Clone the repository and catkin_make:

    cd ~/$A_ROS_DIR$/src
    git clone https://github.com/hviktortsoi/PV_LIO.git
    cd PV_LIO
    cd ../..
    catkin_make
    source devel/setup.bash
  • Remember to source the livox_ros_driver before build (follow 1.3 livox_ros_driver)
  • If you want to use a custom build of PCL, add the following line to ~/.bashrc export PCL_ROOT={CUSTOM_PCL_PATH}

3. Directly run

Noted:

A. Please make sure the IMU and LiDAR are Synchronized, that's important.

B. The warning message "Failed to find match for field 'time'." means the timestamps of each LiDAR points are missed in the rosbag file. That is important for the forward propagation and backwark propagation.

3.1 For Livox Avia

Connect to your PC to Livox Avia LiDAR by following Livox-ros-driver installation, then

    cd ~/$PV_LIO_ROS_DIR$
    source devel/setup.bash
    roslaunch pv_lio mapping_avia.launch
    roslaunch livox_ros_driver livox_lidar_msg.launch
  • For livox serials, PV-LIO only support the data collected by the livox_lidar_msg.launch since only its livox_ros_driver/CustomMsg data structure produces the timestamp of each LiDAR point which is very important for the motion undistortion. livox_lidar.launch can not produce it right now.
  • If you want to change the frame rate, please modify the publish_freq parameter in the livox_lidar_msg.launch of Livox-ros-driver before make the livox_ros_driver pakage.

3.2 For Livox serials with external IMU

mapping_avia.launch theratically supports, mid-70, mid-40 or other livox serial LiDAR, but need to setup some parameters befor run:

Edit config/avia.yaml to set the below parameters:

  1. LiDAR point cloud topic name: lid_topic
  2. IMU topic name: imu_topic
  3. Translational extrinsic: extrinsic_T
  4. Rotational extrinsic: extrinsic_R (only support rotation matrix)
  • The extrinsic parameters in PV-LIO is defined as the LiDAR's pose (position and rotation matrix) in IMU body frame (i.e. the IMU is the base frame). They can be found in the official manual.
  • PV-LIO produces a very simple software time sync for livox LiDAR, set parameter time_sync_en to ture to turn on. But turn on ONLY IF external time synchronization is really not possible, since the software time sync cannot make sure accuracy.

3.3 For Velodyne or Ouster (Velodyne as an example)

Step A: Setup before run

Edit config/velodyne.yaml to set the below parameters:

  1. LiDAR point cloud topic name: lid_topic
  2. IMU topic name: imu_topic (both internal and external, 6-aixes or 9-axies are fine)
  3. Line number (we tested 16, 32 and 64 line, but not tested 128 or above): scan_line
  4. Translational extrinsic: extrinsic_T
  5. Rotational extrinsic: extrinsic_R (only support rotation matrix)
  • The extrinsic parameters in PV-LIO is defined as the LiDAR's pose (position and rotation matrix) in IMU body frame (i.e. the IMU is the base frame).

Step B: Run below

    cd ~/$PV_LIO_ROS_DIR$
    source devel/setup.bash
    roslaunch pv_lio mapping_velodyne.launch

Step C: Run LiDAR's ros driver or play rosbag.

3.4 For Robosense, Hesai, etc. (Robosense as an example)

Step A: Setup before run

Edit launch/mapping_robosense.launch, find and modify the following line:

<remap from="/rslidar_points" to="/your_lidar_topic"/>

Fill /your_lidar_topic with your actual LiDAR topic name.

Step B: Edit config/robosense.yaml to set the below parameters:

  1. IMU topic name: imu_topic (both internal and external, 6-aixes or 9-axies are fine)
  2. Line number (we tested 16, 32 and 64 line, but not tested 128 or above): scan_line
  3. Translational extrinsic: extrinsic_T
  4. Rotational extrinsic: extrinsic_R (only support rotation matrix)
  • The extrinsic parameters in PV-LIO is defined as the LiDAR's pose (position and rotation matrix) in IMU body frame (i.e. the IMU is the base frame).

Step C: Run below

    cd ~/$PV_LIO_ROS_DIR$
    source devel/setup.bash
    roslaunch pv_lio mapping_robosense.launch

Step C: Run LiDAR's ros driver or play rosbag.

4. Rosbag Example

4.1 Robosense 16 Rosbag

Files: Can be downloaded from Baidu Pan (password:4kpf) or Google Drive

Run:

roslaunch pv_lio mapping_robosense.launch
cd YOUR_BAG_DOWNLOADED_PATH
rosbag play *.bag

Important: The 3 bags are from the same dataset sequence, so they should be play sequentially, rather than be played alone.

Related Works

  1. VoxelMap: An efficient and probabilistic adaptive voxel mapping method for LiDAR odometry.
  2. FAST-LIO: A computationally efficient and robust LiDAR-inertial odometry (LIO) package
  3. FAST-LIO_LOCALIZATION: A simple localization framework that can re-localize in built point cloud maps.
  4. IKFoM: A computationally efficient and convenient toolkit of iterated Kalman filter.

Acknowledgments

Thanks a lot for the authors of VoxelMap, IKFoM and FAST-LIO;

Thanks to Xiaokai for his help with deriving the covariance propagation.

TBD

  1. Handle conditions where nMeasurements < nDof;
  2. Migrate constant velocity example from VoxelMap to support Pure LiDAR odometry;
  3. Optimize comments and docs, make them more readable;
  4. Improve the efficiency of voxel map visualization;
  5. Publish covariance of points for visualization;

pv-lio's People

Contributors

erjanmx avatar hviktortsoi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pv-lio's Issues

Robosense data failed

通过百度网盘下载的三个Robosense数据
2023-07-15 18-48-53 的屏幕截图
启动mapping_robosense.launch后崩溃:
2023-07-15 18-51-12 的屏幕截图
2023-07-15 18-51-18 的屏幕截图

About the data collection equipment

Is the LiDAR connected to the IMU and placed on the turntable? Can I have a look at the installation diagram of the collection equipment? Thank you!

Questions about the process of cov transmission at Lidar point.

您好,感谢您很好的工作!我在这里有个不懂的地方:关于transformLiDARCovToWorld函数中,lidar to body的cov传播过程中,我注意到你在公式中*的是以下两个量。括号里是英文翻译,原谅我英文不是很好。(Hello and thank you for a great job! I don't understand here: regarding the cov propagation of lidar to body in the 'transformLiDARCovToWorld' function, I noticed that you have the following two quantities in the formula.)

M3D il_rot_var = kf.get_P().block<3, 3>(6, 6);
M3D il_t_var = kf.get_P().block<3, 3>(9, 9);

这是我在之前自己添加imu的时候没有注意到的地方,想问一下对于voxelmap中的state(StatesGroup)中18维的cov,好像并没有定义的外参的var,我该从哪获得这个信息呢,或者我该参考一些什么公式呢?希望您可以在百忙中抽空回复一下,万分感谢!(This is something I didn't notice when I added imu before, I want to ask where do I get this information for the 18-dimensional cov in the state(StatesGroup) in voxelmap, there seems to be no defined external parameter var, or what formula should I refer to? I hope you can take time out of your busy schedule to reply, thank you very much!)

请问支持l515吗

请问PV-LIO是否支持L515呢?我使用l515进行定位效果并不理想

how to run pvlio on nclt?

Failed to find match for field 'intensity'.
[ WARN] [1712818522.837651718]: Too few input point cloud!

[ INFO] [1712818522.839457525]: IMU Initial Done
[ WARN] [1712818522.839588761]: No point, skip this scan!

Failed to find match for field 'intensity'.
[ WARN] [1712818522.934627600]: Too few input point cloud!

[pv_lio_node-2] process has died [pid 191018, exit code -11, cmd /home/yyh/pvlio_ws/devel/lib/pv_lio/pv_lio_node __name:=pv_lio_node __log:=/home/yyh/.ros/log/6615c9f2-f7d0-11ee-b6a0-af2f3c8f1744/pv_lio_node-2.log].
log file: /home/yyh/.ros/log/6615c9f2-f7d0-11ee-b6a0-af2f3c8f1744/pv_lio_node-2*.log

Hello, I always get the above error when running nclt dataset
What should I do to solve this problem

run error。

我在ubuntu20.04,跑自己的voledyne数据,终端输入如下:我应该怎么处理这种情况

2

Avia + Microstrain Parker IMU questions

Having some issue with an external IMU Microstrain Parker 3DMCV7-AHRS.
Using AVIA, Ubuntu 20.04 lts.

livox/imu running 200 hz
imu/data [3DMCV7-AHRS] running 1000 hz

I have recorded a test .bag

path: time_6.bag
version: 2.0
duration: 22.6s
start: Mar 17 2024 12:11:43.26 (1710673903.26)
end: Mar 17 2024 12:12:05.85 (1710673925.85)
size: 854.4 MB
messages: 20101
compression: none [304/304 chunks]
types: livox_ros_driver/CustomMsg [e4d6829bdfe657cb6c21a746c86b21a6]
sensor_msgs/Image [060021388200f6f0f447d0fcd9c64743]
sensor_msgs/Imu [6a62c6daae103f4ff57a132d6f95cec2]
topics: /camera/color/image_raw 285 msgs : sensor_msgs/Image
/imu/data 15155 msgs : sensor_msgs/Imu
/livox/imu 4441 msgs : sensor_msgs/Imu
/livox/lidar 220 msgs : livox_ros_driver/CustomMsg

config file
common:
lid_topic: "/livox/lidar"
imu_topic: "/imu/data" # Updated IMU topic
time_sync_en: false # ONLY turn on when external time synchronization is really not possible

extrinsic_est_en:  true    # true: enable the online estimation of IMU-LiDAR extrinsic,
extrinsic_T: [ 0.00, 0.0, 0.05 ]  # Corrected translation values
extrinsic_R: [ -1, 0, 0,
               0, -1, 0,
               0, 0, 1 ]  # Corrected rotation values

I do not really understand the function of the online estimation, if I am required to do something else than just set it to true.

The topics are syncronized.

Running the bag with livox/imu works fine.
Running the bag with imu/data result in:

[ WARN] [1710674822.816306212]: No point, skip this scan!

[ WARN] [1710674822.913519957]: No point, skip this scan!

[ WARN] [1710674823.013094566]: No point, skip this scan!

[ WARN] [1710674823.111961037]: No point, skip this scan!

[ WARN] [1710674823.213315000]: No point, skip this scan!

[ WARN] [1710674823.316323448]: No point, skip this scan!

[ WARN] [1710674823.418372740]: No point, skip this scan!

Reducing Drift with High Compute

Hi,

I am trying to use PV-LIO on a scan with a lot of very small rooms. The thing is i don't want realtime mapping, but i want the drift to be as less as possible. I can run the algorithm on 96 cores / 192 cores with 1024GB of RAM. What are the parameters that i need to tune so it makes use of all the compute available?

here's what i tried

  • increase openmp number of cores to max: this makes sure to use all cores, but it only uses like 50% of each core.
  • decrease the voxel size: i tried 0.1 and 0.05 and downsample size to 0.05, but still there is slight drift overall.
  • increase max iterations: i tried increasing this to 100 and then to 500, but there wasn't any change from this, maybe slightly but not major
  • increase number of voxel layers: this absolutely did nothing
  • decrease the planar threshold: if i try to reduce this below 0.05 there is drift

So what parameters should i try to tune?

I have already tried things like SC (Scan Context) and LC (Loop Closure) algorithms (https://github.com/gisbi-kim/FAST_LIO_SLAM) that were suited for FASTLIO2 but they work well with PV-LIO https://github.com/hku-mars/Point-LIO as well. but they also are not able to fix the drift.

Also I've tried HBA (Hierarchical Bundle Adjustment) (https://github.com/hku-mars/HBA) , it does solve the drift, but it adds too much noise to the point cloud to a point that the point cloud is unusable.

Also note. I am using MID360 with inbuilt imu, should i try out an external IMU? will that help?

Some areas its really good

image

some areas its really bad and drifts, its mostly when we go from a large room to a small room or vice versa

image image

also NOTE: FASTLIO2 does not drift in the above case, although it drifts as soon as we enter a small room in some cases, but the above case FASTLIO2 is able to solve.

  • I was also thinking if it would be possible to do something like what Point-LIO does? increasing the rate of IMU might solve to some extent?
  • also maybe STD might help? https://github.com/hku-mars/STD. I've seen an implementation of STD with FASTLIO2, but its just shit.

Is sensors' hard synchronization necessary?

Thank you for your work.
I'm not familiar with the fast-lio series. I have a velodyne lidar and an independent IMU. But they are not hardware-wise synchronized. Is that OK to run the code?
Hope for your reply. Thank you very much.

Mapping result has noise points.

Greet job, I forked your project and add the function of saving result to PCD file. Then I tried this on my own dataset. The mapping runs great, even I dont have the precise extrinsic. Now I have 3 questions:

  1. The PCD file has some noise points. Where are they come from?
    Snipaste_2023-07-18_17-04-26

  2. Is it possible to add loop closure on this basis?

  3. There is a online extrinsic optimizing during mapping. Where can I get the final extrinsic arguments ?

robosense timestamp

Hi, @HViktorTsoi
Thanks for your nice work!
I'm confused about the robosense data you shared, whether the header timestamp is frame head or frame end?

nMeasurements < nDof;

Facing this error a lot of times. Can you please explain why it is happening and what scene it can happen?

Please share some light to solve this issue.

PCD Output - pcd_save_en: true

Hi! Great job with your project! I've only just tried it and it looks so good.

Here's a screenshot i took using a Livox HAP and you really can see it tracking the walk.
image
And another from top down:
image

The configuration that includes quite a bit from FAST-LIO2 mentions PCD save but for the life of me i can't seem to find if this would actually do anything. I'd assume that due to the updated pipeline that may not have been factored in? But I'm not really sure.
image

Is there any way to output a PCD?

Again thanks for publishing such a great quality project.

which kind of IMU do you recommend?

Thanks for your excellent work.I am interested in your system hardware requirement. which kind of IMU did you employ to acquire the Robosense 16 Rosbag(multi-floor)?as my imu usually will have large drift when have large rotation.it is very helpful to have your answer.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.