uz-slamlab / orb_slam3 Goto Github PK

When I run the dataset-room4_512_16 dataset, I encounter the problem:

Segmentation fault      (core dumped)

The whole message is:

sph@sph-System-Product-Name:~/Documents/ORB_SLAM3_Fixed$ ./tum_vi.sh 
Launching Room 4 with Monocular-Inertial sensor
num_seq = 1
file name: dataset-room4_512_monoi
Loading images for sequence 0.../home/sph/Downloads/dataset-room4_512_16/mav0/cam0/data
Examples/Monocular-Inertial/TUM_TimeStamps/dataset-room4_512.txt
LOADED!
Loading IMU for sequence 0...LOADED!
vstrImageFilenames[seq].size() = 2228

-------

ORB-SLAM3 Copyright (C) 2017-2020 Carlos Campos, Richard Elvira, Juan J. Gómez, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
ORB-SLAM2 Copyright (C) 2014-2016 Raúl Mur-Artal, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
This program comes with ABSOLUTELY NO WARRANTY;
This is free software, and you are welcome to redistribute it
under certain conditions. See LICENSE.txt.

Input sensor was set to: Monocular-Inertial

Loading ORB Vocabulary. This could take a while...
Vocabulary loaded!

Creation of new map with id: 0
Creation of new map with last KF id: 0
Seq. Name: dataset-room4_512_monoi

Camera Parameters: 
- fx: 190.97846984863281
- fy: 190.97331237792969
- cx: 254.93170166015625
- cy: 256.89743041992188
- bf: 0
- k1: 0
- k2: 0
- p1: 0
- p2: 0
- k3: -0.0020532361231744289
- fps: 20
- color order: RGB (ignored if grayscale)

ORB Extractor Parameters: 
- Number of Features: 1500
- Scale Levels: 8
- Scale Factor: 1.2000000476837158
- Initial Fast Threshold: 20
- Minimum Fast Threshold: 7

Left camera to Imu Transform (Tbc): 
[-0.99952501, 0.0075019184, -0.029890131, 0.045574836;
 0.029615344, -0.03439736, -0.99896932, -0.071161799;
 -0.008522328, -0.99938005, 0.034158852, -0.044681255;
 0, 0, 0, 1]

IMU frequency: 200 Hz
IMU gyro noise: 0.00015999999595806003 rad/s/sqrt(Hz)
IMU gyro walk: 2.2000000171829015e-05 rad/s^2/sqrt(Hz)
IMU accelerometer noise: 0.00279999990016222 m/s^2/sqrt(Hz)
IMU accelerometer walk: 0.00085999997099861503 m/s^3/sqrt(Hz)
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
init extractor
First KF:0; Map init KF:0
New Map created with 386 points
start inserting MPs
./tum_vi.sh: line 17: 10817 Segmentation fault      (core dumped) ./Examples/Monocular-Inertial/mono_inertial_tum_vi Vocabulary/ORBvoc.txt Examples/Monocular-Inertial/TUM_512.yaml "$pathDatasetTUM_VI"/dataset-room4_512_16/mav0/cam0/data Examples/Monocular-Inertial/TUM_TimeStamps/dataset-room4_512.txt Examples/Monocular-Inertial/TUM_IMU/dataset-room4_512.txt dataset-room4_512_monoi

My tum_vi.sh file is:

#!/bin/bash
pathDatasetTUM_VI='/home/sph/Downloads' #Example, it is necesary to change it by the dataset path
#------------------------------------
# Monocular-Inertial Examples
echo "Launching Room 4 with Monocular-Inertial sensor"
./Examples/Monocular-Inertial/mono_inertial_tum_vi Vocabulary/ORBvoc.txt Examples/Monocular-Inertial/TUM_512.yaml "$pathDatasetTUM_VI"/dataset-room4_512_16/mav0/cam0/data Examples/Monocular-Inertial/TUM_TimeStamps/dataset-room4_512.txt Examples/Monocular-Inertial/TUM_IMU/dataset-room4_512.txt dataset-room4_512_monoi

I use gdb to find where is dumped, I found that:

warning: Unexpected size of section `.reg-xstate/8693' in core file.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `./Examples/Monocular-Inertial/mono_inertial_tum_vi Vocabulary/ORBvoc.txt Exampl'.
Program terminated with signal SIGSEGV, Segmentation fault.

warning: Unexpected size of section `.reg-xstate/8693' in core file.
#0  0x00007fe0aa1bbd8d in Eigen::DenseStorage<double, 4, 4, 1, 0>::DenseStorage (other=..., this=0x42780000ffffffff)
    at /usr/include/eigen3/Eigen/src/Core/DenseStorage.h:194
194	    DenseStorage(const DenseStorage& other) : m_data(other.m_data) {
[Current thread is 1 (Thread 0x7fe0aa69ce80 (LWP 8693))]

Can someone help? Thank you very very very very much ~~~

I'm trying to run ORB_SLAM3 on gazebo. The program can run and show the figure that taken by virtual camera. But it can not extract feature points and estimate camera pose.

Here is the information of rostopic /camera/rgb/camera_info:

And yaml file:

%YAML:1.0

#--------------------------------------------------------------------------------------------
# Camera Parameters. Adjust them!
#--------------------------------------------------------------------------------------------
Camera.type: "PinHole"
# Camera calibration and distortion parameters (OpenCV) 
Camera.fx: 277.19135641132203
Camera.fy: 277.19135641132203
Camera.cx: 160.5
Camera.cy: 120.5

Camera.k1: 0
Camera.k2: 0
Camera.p1: 0
Camera.p2: 0

Camera.width: 320 
Camera.height: 240 

# Camera frames per second 
Camera.fps: 30.0

# IR projector baseline times fx (aprox.)
Camera.bf: 13.8595

# Color order of the images (0: BGR, 1: RGB. It is ignored if images are grayscale)
Camera.RGB: 1

# Close/Far threshold. Baseline times.
ThDepth: 50.0

# Deptmap values factor 
DepthMapFactor: 1000.0

#--------------------------------------------------------------------------------------------
# ORB Parameters
#--------------------------------------------------------------------------------------------

# ORB Extractor: Number of features per image
ORBextractor.nFeatures: 1000

# ORB Extractor: Scale factor between levels in the scale pyramid 	
ORBextractor.scaleFactor: 1.2

# ORB Extractor: Number of levels in the scale pyramid	
ORBextractor.nLevels: 8

# ORB Extractor: Fast threshold
# Image is divided in a grid. At each cell FAST are extracted imposing a minimum response.
# Firstly we impose iniThFAST. If no corners are detected we impose a lower value minThFAST
# You can lower these values if your images have low contrast			
ORBextractor.iniThFAST: 20
ORBextractor.minThFAST: 7

#--------------------------------------------------------------------------------------------
# Viewer Parameters
#--------------------------------------------------------------------------------------------
Viewer.KeyFrameSize: 0.05
Viewer.KeyFrameLineWidth: 1
Viewer.GraphLineWidth: 0.9
Viewer.PointSize:2
Viewer.CameraSize: 0.08
Viewer.CameraLineWidth: 3
Viewer.ViewpointX: 0
Viewer.ViewpointY: -0.7
Viewer.ViewpointZ: -1.8
Viewer.ViewpointF: 500

And I got this:
Framebuffer with requested attributes not available. Using available framebuffer. You may see visual artifacts.

Does anyone have any idea? Thanks

Hi , thank you for your work.

We are testing with Intel's T-265 sensor and VI sensor.

Overall these sensors seem to work fine with ORB SLAM3.

The screenshot below is the result of the sensor experiment.

By the way, when we tested with the T-265 sensor, the initialization is always late with the message'IMU is not initialized. Reset Active Map' (below photo)

We have experimented with both the stereo IMU and mono IMU versions of your algorithm. And the results of both are the same. (late initialize with T-265)

Why is initialization so late? How to quickly complete initialization?

void computeError(){ const VertexGyroBias* VG1= static_cast<const VertexGyroBias*>(_vertices[0]); const VertexGyroBias* VG2= static_cast<const VertexGyroBias*>(_vertices[1]); _error = VG2->estimate()-VG1->estimate(); }
why still VG1 & VG2 instead of VA1 & VA2？

Hi

How can get custom camera + imu calibration setting values?
Please let me know if anybody know some link or method.
Thank you in advance!

Hi,did anyone runs the stereo camera without using IMU, i got my dataset by two cameras looking upwards and located as left and right.
I finished to run my dataset by each single camera but failed when running stereo. The problem is listed below:

1. KeyFrames are too few and tracking always gets lost and thus go to start a new map;
2. I changed the conditions in NeedNewKeyFrame() from
   bNeedToInsertClose = (nTrackedClose<100) && (nNonTrackedClose>70) to
   bNeedToInsertClose = (nTrackedClose < nTotalClosedPoints*0.8) && (nNonTrackedClose>nTotalClosedPoints*0.7).
   and nTotalClosedPoints is all the close points whose depth is between 0 and mThDepth.
   After changed this line, i can get a whole map or sometimes two submaps.
   But it seems that the trajectory is not in a horizon plane while my cameras stay at the same level in the dataset.

Hi many thanks for the great contribution! After running my own recorded data experiment on orbslam3 many times (stereo-only experience), the map results obtained by using the evo tool to analyze are different. What is the reason for this (multi-threading will have a certain impact on the selection of key frames)? Does this belong to the normal error range? I got the same conclusion in orb-slam2.

Thank you for your work.I have test the stereo_rosnode,and have some problems:
1.The latency is very low at first, and the program runs for 2-3 seconds and then gets higher and higher, with a delay of about 4-5 seconds. And the CPU utilization gradually increases, and the program crashes when the camera moves too fast or the map is rebuilt..My hardware devices are as follows:
Intel® Core™ i7-7700K CPU @ 4.20GHz × 8
GeForce GTX 1080/PCIe/SSE2
Is my hardware device not able to run in real time? But can run ORBSLAM2 in real time, I am confused about this.

"CMakeFiles/ORB_SLAM3.dir/build.make:101: recipe for target 'CMakeFiles/ORB_SLAM3.dir/src/LoopClosing.cc.o' failed
make[2]: *** [CMakeFiles/ORB_SLAM3.dir/src/LoopClosing.cc.o] Error 1
CMakeFiles/Makefile2:183: recipe for target 'CMakeFiles/ORB_SLAM3.dir/all' failed"

Dose anyone encountered this problem? Thank you very much for help

when I using webcam as MONOCULAR camera, error occurs:

it seems that the code collapses at the Tracking.cc

does anyone know the reasons?

Hi,
When I run monocular ORB-SLAM on my camera, it always fails to be initialized successfully. I found that most of the time because the value of nsimilar is equal to 2 so that cannot reconstruct fundamental matrix.
I'm sure I have correct camera intrinsic. And besides, is there any other parameters needs to modify?
Can you tell me what should I do?
Thank you very much.

looking forward to!

When you run the build_ros.sh file, you get a lot of errors

Hi.

I'm using a stereo camera. I based myself on the EuRoC dataset to form my own sequences. I saved frames, timestamps and I also have the camera calibration parameters.
Here is the organization of my folders:

MH01
   |---- mav0
          |---- cam0
                |---- data
                       |---- photos of the left side (timestamp.png)
          |---- cam1
                |---- data
                       |---- photos of the right side (timestamp.png)
   |---- timestamps.txt (timestamp timestamp.png )
   |---- config.yalm

Running the command:

./Examples/Stereo/stereo_euroc ./Vocabulary/ORBvoc.txt path/MH01/config.yaml path/MH01 path/MH01/timestamps.txt dataset-MH01_stereo

I have the error:

.
.
.
Failed to load image at: /path/MH01/mav0/cam0/data/1403636579763555584.png

It is looking for the original dataset, so I assumed this is not the appropriate way, right? Can anyone help me with the right command to run ORBSLAM3 with my own sequence?

Tks in advance.

Hi,

Thanks for this amazing work.

I've tried to config and run the code with KITTI raw data (unsynced & unrectified data) in which the IMU data is available (it is not available in KITTI evaluation dataset). However, I faced the issue of low numbers of matched features when running the code. Please refer to the log below:

...
mnFirstFrameId = 101
mnInitialFrameId = 99
72 Frames had been set to lost
last KF is empty!
not IMU meas
last KF is empty!
First KF:31; Map init KF:30
New Map created with 128 points
Matches Inliners: 48 | Matching ratio: 48/1010
Fail to track local map!
IMU is not or recently initialized. Reseting active map...
LM: Active map reset recieved
LM: Active map reset, waiting...
LM: Reseting current map in Local Mapping...
LM: End reseting Local Mapping...
LM: Reset free the mutex
LM: Active map reset, Done!!!
mnFirstFrameId = 105
mnInitialFrameId = 103
74 Frames had been set to lost
last KF is empty!
not IMU meas
last KF is empty!
First KF:32; Map init KF:31
New Map created with 133 points
Matches Inliners: 43 | Matching ratio: 43/1005
Fail to track local map!
IMU is not or recently initialized. Reseting active map...
LM: Active map reset recieved
LM: Active map reset, waiting...
LM: Reseting current map in Local Mapping...
LM: End reseting Local Mapping...
LM: Reset free the mutex
LM: Active map reset, Done!!!
mnFirstFrameId = 108
mnInitialFrameId = 106
76 Frames had been set to lost
last KF is empty!
not IMU meas
last KF is empty!
First KF:33; Map init KF:32
New Map created with 121 points
Matches Inliners: 43 | Matching ratio: 43/1006
Fail to track local map!
IMU is not or recently initialized. Reseting active map...
LM: Active map reset recieved
LM: Active map reset, waiting...
LM: Reseting current map in Local Mapping...
LM: End reseting Local Mapping...
LM: Reset free the mutex
LM: Active map reset, Done!!!
mnFirstFrameId = 111
mnInitialFrameId = 109
78 Frames had been set to lost
last KF is empty!
not IMU meas

Saving trajectory to f_kitti_0001.txt ...

The number of features is lower than 50 (which is set in the code as the threshold for a successfully tracking). This made the program repeatedly failed in the tracking stage and the tracking map kept resetting all the time.

One of the sources for the issue that I could think of is the wrong parameters in my configuration file. The camera and IMU calibration were carefully calculated.

Please let me know if you spot any issue in the code.

The code could be found at https://github.com/biendltb/ORB_SLAM3/tree/FEAT/kitti_raw_imu_stereo

Thank you and best wishes,

Bien

Hi, Thanks for sharing your great work. And I met some errors when I used 'make -j8' to build the project.
It output like below:
ORB_SLAM3/include/Map.h:140:33: error: ‘list’ has not been declared
void printReprojectionError(list<KeyFrame*> &lpLocalWindowKFs, KeyFrame* mpCurrentKF, string &name, string &name_folder);
^
/ORB_SLAM3/include/Map.h:140:37: error: expected ‘,’ or ‘...’ before ‘<’ token
void printReprojectionError(list<KeyFrame*> &lpLocalWindowKFs, KeyFrame* mpCurrentKF, string &name, string &name_folder);

All the dependencies have been installed correctly.

I am really confused by the errors and hope someone could help me!

When I testing ORB_SLAM3 on KITTI datasets as stereo mode, and on TUM RGB-D as RGB-D mode, I get following warnings:

...
void g2o::SparseOptimizer::update(const double*): Update contains a nan for vertex xxxx
void g2o::SparseOptimizer::update(const double*): Update contains a nan for vertex xxxx
void g2o::SparseOptimizer::update(const double*): Update contains a nan for vertex xxxx
....
computeActiveErrors(): found NaN in error for edge xxxx
computeActiveErrors(): found NaN in error for edge xxxx
computeActiveErrors(): found NaN in error for edge xxxx
computeActiveErrors(): found NaN in error for edge xxxx
...

These warnings occurred during local BA. I have checked all local key frames', fixed key frames' pose and local map points' position, no one has nan. I know that building system in release mode can disable these warnings, but it can not really solve this problem.

Any one knows why nan was produced? I will appreciate to your help.

My environmental information:

OS: Ubuntu 20.04 LTS
GNU GCC/G++: 10.0.1
cmake: 3.16.3
GNU make: 4.2.1

Pangolin: commit 86eb497
OpenCV: 3.2.0
Eigen: 3.2.10

Hello.
I am converting kitti data to euroc type and testing it.
However, I'm always getting segmentation errors.
The reason seems to be a problem because the bias of ImuTypes is always 0.

If there is someone who can help you when the bias value is set, please help.

ORB-SLAM3 Copyright (C) 2017-2020 Carlos Campos, Richard Elvira, Juan J. Gómez, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
ORB-SLAM2 Copyright (C) 2014-2016 Raúl Mur-Artal, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
This program comes with ABSOLUTELY NO WARRANTY;
This is free software, and you are welcome to redistribute it
under certain conditions. See LICENSE.txt.

Input sensor was set to: Stereo

Loading ORB Vocabulary. This could take a while...
Vocabulary loaded!

Creation of new map with id: 0
Creation of new map with last KF id: 0
Seq. Name:

Camera Parameters:

• fx: 707.091
• fy: 707.091
• cx: 601.887
• cy: 183.11
• bf: 379.815
• k1: 0
• k2: 0
• p1: 0
• p2: 0
• fps: 10
• color order: RGB (ignored if grayscale)

ORB Extractor Parameters:

• Number of Features: 2000
• Scale Levels: 8
• Scale Factor: 1.2
• Initial Fast Threshold: 12
• Minimum Fast Threshold: 7

Depth Threshold (Close/Far Points): 21.486

Start processing sequence ...
Images in the sequence: 1101

First KF:0; Map init KF:0
New Map created with 803 points
BoW: 162 matches between 878 points with coarse Sim3
terminate called after throwing an instance of 'cv::Exception'
what(): OpenCV(3.4.9) /home/user/Opencv-3.4.9/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'

Aborted (core dumped)

Every time a loop closure occurs I get this error and the program terminates. Running this on KITTI Stereo Dataset. Works good for sequences with no loop closure like sequence 03, 04, 10.

My system configuration:
Ubuntu 18.04
OpenCV 3.4.9
Eigen 3.3

Anybody else ran into the same problem?

Hi authors,

Thank you for excellent work and public your code.

I'm a big fan of ORB-SLAM2 and studying ORB-SLAM3. The results from stereo-inertial SLAM + TUM datasets are awesome, obviously better than filter-based approach MSCKF-VIO, which I'm using. But I'm facing with this issues :

When run ORB-SLAM3 with Kitti datasets, the program crashed after new map created.

Backtrace :

#0  0x00007f679f40cfe4 in ORB_SLAM3::EdgeSE3ProjectXYZOnlyPose::computeError() ()
    at /ORB_SLAM3/lib/libORB_SLAM3.so
#1  0x00007f679d2eed61 in g2o::SparseOptimizer::computeActiveErrors() ()
    at /ORB_SLAM3/Thirdparty/g2o/lib/libg2o.so
#2  0x00007f679d2f831e in g2o::OptimizationAlgorithmLevenberg::solve(int, bool) ()
    at /ORB_SLAM3/Thirdparty/g2o/lib/libg2o.so
#3  0x00007f679d2f02ad in g2o::SparseOptimizer::optimize(int, bool) ()
    at /ORB_SLAM3/Thirdparty/g2o/lib/libg2o.so
#4  0x00007f679f3d2d29 in ORB_SLAM3::Optimizer::PoseOptimization(ORB_SLAM3::Frame*) ()
    at /ORB_SLAM3/lib/libORB_SLAM3.so
#5  0x00007f679f322a2e in ORB_SLAM3::Tracking::TrackReferenceKeyFrame() () at /ORB_SLAM3/lib/libORB_SLAM3.so

I would be nice if you guys or someone help me to overcome this issues. I also commented this issue in #3 , but I think a ticket should be created for the issue.

Many thanks in advance.

Regards,
Michael

Thank you for your work. 1. Is this program currently only able to run data sets and cannot be run in real time with the camera (rgbd/stereo/mono)? If possible, which programs need to be modified.
2. Do you have a plan to complete the function of saving the map.
3.Whether the map can be used for navigation, as far as I know, sparse maps cannot be used for navigation and obstacle avoidance.
Looking forward to receiving your reply~

Does anyone run the demo without 'Segmentation fault (core dumped)' error?

Seems like an issue with the OpenCV version (currently OpenCV3.4 and Ubuntu18.04).

Is there anyone here tested it successfully with Ubuntu18.04 and OpenCV3.4?

### Anyone knows how to fix it or have no issue running the demos please share your experience here, it looks like a common issue for this segmentation fault.
Thank you!

fox@YangTianM6201c-00:~/catkin_kinect$ rosrun ORB_SLAM3 RGBD ORBvoc.txt Asus_d435i.yaml

ORB-SLAM3 Copyright (C) 2017-2020 Carlos Campos, Richard Elvira, Juan J. Gómez, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
ORB-SLAM2 Copyright (C) 2014-2016 Raúl Mur-Artal, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
This program comes with ABSOLUTELY NO WARRANTY;
This is free software, and you are welcome to redistribute it
under certain conditions. See LICENSE.txt.

Input sensor was set to: RGB-D

Loading ORB Vocabulary. This could take a while...
Vocabulary loaded!

Creation of new map with id: 0
Creation of new map with last KF id: 0
Seq. Name:

Camera Parameters:

• fx: 614.859
• fy: 614.964
• cx: 325.856
• cy: 228.775
• bf: 30.7429
• k1: 0
• k2: 0
• p1: 0
• p2: 0
• fps: 30
• color order: RGB (ignored if grayscale)

ORB Extractor Parameters:

• Number of Features: 1000
• Scale Levels: 8
• Scale Factor: 1.2
• Initial Fast Threshold: 20
• Minimum Fast Threshold: 7

Depth Threshold (Close/Far Points): 2.5
First KF:0; Map init KF:0
New Map created with 289 points
段错误 (核心已转储)

CMakeFiles/Makefile2:400: recipe for target 'CMakeFiles/ORB_SLAM3.dir/all' failed
make[1]: *** [CMakeFiles/ORB_SLAM3.dir/all] Error 2

Will docker image be released?

What can I do to save and load maps under ROS, and realize odometer under ROS

/home/long/Project/ORB_SLAM3-master/Examples/Monocular-Inertial/mono_inertial_euroc ../../Vocabulary/ORBvoc.txt EuRoC.yaml /home/long/MH_05_difficult ./EuRoC_TimeStamps/MH05.txt
num_seq = 1
Loading images for sequence 0...LOADED!
Loading IMU for sequence 0...LOADED!

ORB-SLAM3 Copyright (C) 2017-2020 Carlos Campos, Richard Elvira, Juan J. Gómez, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
ORB-SLAM2 Copyright (C) 2014-2016 Raúl Mur-Artal, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
This program comes with ABSOLUTELY NO WARRANTY;
This is free software, and you are welcome to redistribute it
under certain conditions. See LICENSE.txt.

Input sensor was set to: Monocular-Inertial

Loading ORB Vocabulary. This could take a while...
Vocabulary loaded!

Creation of new map with id: 0
Creation of new map with last KF id: 0
Seq. Name:

Camera Parameters:

• fx: 458.65399169921875
• fy: 457.29598999023438
• cx: 367.21499633789062
• cy: 248.375
• bf: 0
• k1: -0.28340810537338257
• k2: 0.073959067463874817
• p1: 0.00019359000725671649
• p2: 1.7618711353861727e-05
• fps: 20
• color order: RGB (ignored if grayscale)

ORB Extractor Parameters:

• Number of Features: 1000
• Scale Levels: 8
• Scale Factor: 1.2000000476837158
• Initial Fast Threshold: 20
• Minimum Fast Threshold: 7

Left camera to Imu Transform (Tbc):
[0.014865543, -0.99988091, 0.004140297, -0.021640146;
0.99955726, 0.014967213, 0.02571553, -0.064676985;
-0.025774436, 0.0037561883, 0.99966073, 0.0098107308;
0, 0, 0, 1]

IMU frequency: 200 Hz
IMU gyro noise: 0.00016999999934341758 rad/s/sqrt(Hz)
IMU gyro walk: 1.9392999092815444e-05 rad/s^2/sqrt(Hz)
IMU accelerometer noise: 0.0020000000949949026 m/s^2/sqrt(Hz)
IMU accelerometer walk: 0.0030000000260770321 m/s^3/sqrt(Hz)
init extractor
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
init extractor
last KF is empty!
First KF:0; Map init KF:0
New Map created with 309 points

Process finished with exit code 0

stop at
void Optimizer::BundleAdjustment(const vector<KeyFrame *> &vpKFs, const vector<MapPoint > &vpMP,
int nIterations, bool pbStopFlag, const unsigned long nLoopKF, const bool bRobust)
optimizer.optimize(nIterations);

I tried to run ./euroc_examples.sh in the root folder and set the path to the dataset actually uncompressed. But the program output a error message:

Failed to load image at: /Datasets/MH01/mav0/cam0/data/1403636579763555584.png
./euroc_examples.sh: line 7: 12029 Segmentation fault      (core dumped) ./Examples/Monocular/mono_euroc ./Vocabulary/ORBvoc.txt ./Examples/Monocular/EuRoC.yaml "$pathDatasetEuroc"/MH01 ./Examples/Monocular/EuRoC_TimeStamps/MH01.txt dataset-MH01_mono
Launching MH02 with Monocular sensor
num_seq = 1

The file name contained in the timestamp file does not match with the real file name in the corresponding dataset folder. Is this the real reason for such error ?

Any help will be appreciate.

I am trying to build it using ./build.sh and i got this type of error:
note that i am using ubuntu 20.04
/home/beraru/ORB_SLAM3/src/Optimizer.cc: In static member function ‘static void ORB_SLAM3::Optimizer::MergeInertialBA(ORB_SLAM3::KeyFrame*, ORB_SLAM3::KeyFrame*, bool*, ORB_SLAM3::Map*, ORB_SLAM3::LoopClosing::KeyFrameAndPose&)’:
/home/beraru/ORB_SLAM3/src/Optimizer.cc:7015:19: warning: suggest explicit braces to avoid ambiguous ‘else’ [-Wdangling-else]
7015 | if(!pMP->isBad())
| ^
/home/beraru/ORB_SLAM3/src/Optimizer.cc: In static member function ‘static int ORB_SLAM3::Optimizer::PoseInertialOptimizationLastKeyFrame(ORB_SLAM3::Frame*, bool)’:
/home/beraru/ORB_SLAM3/src/Optimizer.cc:7706:10: warning: unused variable ‘bOut’ [-Wunused-variable]
7706 | bool bOut = false;
| ^~~~
/home/beraru/ORB_SLAM3/src/LocalMapping.cc: In member function ‘void ORB_SLAM3::LocalMapping::KeyFrameCulling()’:
/home/beraru/ORB_SLAM3/src/LocalMapping.cc:1057:48: warning: ‘last_ID’ may be used uninitialized in this function [-Wmaybe-uninitialized]
1057 | if((bInitImu && (pKF->mnId<last_ID) && t<3.) || (t<0.5))
| ^~~~~~~
make[2]: *** [CMakeFiles/ORB_SLAM3.dir/build.make:102: CMakeFiles/ORB_SLAM3.dir/src/LoopClosing.cc.o] Error 1
/home/beraru/ORB_SLAM3/src/Tracking.cc: In member function ‘void ORB_SLAM3::Tracking::Track()’:
/home/beraru/ORB_SLAM3/src/Tracking.cc:1155:13: warning: ‘bOK’ may be used uninitialized in this function [-Wmaybe-uninitialized]
1155 | if(bOK)
| ^~
make[2]: *** [CMakeFiles/ORB_SLAM3.dir/build.make:232: CMakeFiles/ORB_SLAM3.dir/src/Optimizer.cc.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:390: CMakeFiles/ORB_SLAM3.dir/all] Error 2
make: *** [Makefile:84: all] Error 2

~/src/ORB_SLAM3$ ./Examples/RGB-D/rgbd_tum Vocabulary/ORBvoc.txt Examples/RGB-D/TUM1.yaml /home/zd/src/ORB_SLAM3/data/rgbd_dataset_freiburg1_room /home/zd/src/ORB_SLAM3/Examples/RGB-D/associations.txt

ORB-SLAM3 Copyright (C) 2017-2020 Carlos Campos, Richard Elvira, Juan J. Gómez, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
ORB-SLAM2 Copyright (C) 2014-2016 Raúl Mur-Artal, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
This program comes with ABSOLUTELY NO WARRANTY;
This is free software, and you are welcome to redistribute it
under certain conditions. See LICENSE.txt.

Input sensor was set to: RGB-D

Loading ORB Vocabulary. This could take a while...
Vocabulary loaded!

Creation of new map with id: 0
Creation of new map with last KF id: 0
Seq. Name:

Camera Parameters:

• fx: 517.306
• fy: 516.469
• cx: 318.643
• cy: 255.314
• bf: 40
• k1: 0.262383
• k2: -0.953104
• p1: -0.005358
• p2: 0.002628
• k3: 1.16331
• fps: 30
• color order: RGB (ignored if grayscale)

ORB Extractor Parameters:

• Number of Features: 1000
• Scale Levels: 8
• Scale Factor: 1.2
• Initial Fast Threshold: 20
• Minimum Fast Threshold: 7

Depth Threshold (Close/Far Points): 3.09294

Start processing sequence ...
Images in the sequence: 1352

First KF:0; Map init KF:0
New Map created with 852 points
BoW: 248 matches between 683 points with coarse Sim3
terminate called after throwing an instance of 'cv::Exception'
what(): OpenCV(4.3.0) /home/zd/app/opencv-4.3.0/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'

Segmentation fault (core dump)

When executing tum3, it is all right. I don't know why.

First, thanks for your excellent work.

Now I use EuRoc Monocular-Inertial dataset to generate the frame trajectory (Twb) and evaluate this trajectory based on the pipeline (rpg_trajectory_evaluation: https://github.com/uzh-rpg/rpg_trajectory_evaluation). However, the trajectory is not aligned with the groundtruth data provided by EuRoc. Here are the trajectory results:

On the other hand, using your evaluation tool provided by this repo is correctly aligned.

Is there any extra coordinate transformation when using rpg_trajectory_evaluation?

Many thanks.

Hi,

thank you for sharing such a wonderful work.

I have been trying to use the Mono_Inertial node, and found out that it always lost the track with this complain

ERROR: Frame with a timestamp older than previous frame detected!

Could you suggest what could be the potential reasons?

The calibration file is as the following

YAML:1.0

#--------------------------------------------------------------------------------------------
# Camera Parameters. Adjust them!
#--------------------------------------------------------------------------------------------
Camera.type: "PinHole"
# Camera calibration and distortion parameters (OpenCV) 
Camera.fx: 929.562627
Camera.fy: 928.604856
Camera.cx: 487.474037
Camera.cy: 363.165223

Camera.k1: -0.016272
Camera.k2: 0.093492
Camera.p1: 0.000093
Camera.p2: 0.002999
Camera.k3: 0.000000
    
# Camera resolution
Camera.width: 960
Camera.height: 720

# Camera frames per second 
Camera.fps: 30.0

# Color order of the images (0: BGR, 1: RGB. It is ignored if images are grayscale)
Camera.RGB: 1

# Transformation from body-frame (imu) to camera
Tbc: !!opencv-matrix
   rows: 4
   cols: 4
   dt: f
   data: [0.000624762189693373, 0.9987476589604951, 0.05002722651855723, -0.00027498954959967003,
          -0.20874315768809598, -0.048794909164563804, 0.9767524512167998, 0.00021872887165989068,
          0.977970298010431, -0.011053079234071894, 0.20845125485071164, -8.459932530849928e-05,
          0.0, 0.0, 0.0, 1.0]
### 

IMU.NoiseGyro: 0.000187 # rad/s^0.5 
IMU.NoiseAcc: 0.00186 # m/s^1.5
IMU.GyroWalk: 2.66e-05 # rad/s^1.5
IMU.AccWalk: 0.000433 # m/s^2.5
IMU.Frequency: 15

Hey
I am running with my own monocular datasets (TUM form). The datasets can be run successfully in ORB-SLAM2. But in ORB-SLAM3 it will meet trouble:

./Examples/Monocular/mono_tum Vocabulary/ORBvoc.txt Examples/Monocular/TUM3.yaml ceiling_1_enhance 

ORB-SLAM3 Copyright (C) 2017-2020 Carlos Campos, Richard Elvira, Juan J. Gómez, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
ORB-SLAM2 Copyright (C) 2014-2016 Raúl Mur-Artal, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
This program comes with ABSOLUTELY NO WARRANTY;
This is free software, and you are welcome to redistribute it
under certain conditions. See LICENSE.txt.

Input sensor was set to: Monocular

Loading ORB Vocabulary. This could take a while...
Vocabulary loaded!

Creation of new map with id: 0
Creation of new map with last KF id: 0
Seq. Name: 

Camera Parameters: 
- fx: 535.4
- fy: 539.2
- cx: 320.1
- cy: 247.6
- bf: 0
- k1: 0
- k2: 0
- p1: 0
- p2: 0
- fps: 30
- color order: RGB (ignored if grayscale)

ORB Extractor Parameters: 
- Number of Features: 1000
- Scale Levels: 8
- Scale Factor: 1.2
- Initial Fast Threshold: 20
- Minimum Fast Threshold: 7

-------
Start processing sequence ...
Images in the sequence: 1592
Segmentation fault (core dumped)

Is any one knows how to fix this problem? Thank you

Hey guys,

I've changed the code in stereo_inertial_euroc.cc to make the code running for real-time on my current MYNT D stereo camera. But I found the running speed on my laptop is pretty slow say 2-4 fps and I can tell the obvious latency between 2 continuous frames. And once I move more aggressively, the code would lose tracking and generate following info

Fail to track local map!
IMU is not or recently initialized. Reseting active map...
LM: Active map reset recieved
LM: Active map reset, waiting...
FAIL LOCAL-INERTIAL BA!!!!
LM: Reseting current map in Local Mapping...
LM: End reseting Local Mapping...
LM: Reset free the mutex
LM: Active map reset, Done!!!
mnFirstFrameId = 594
mnInitialFrameId = 8503
0 Frames set to lost
not IMU meas
not enough acceleration
not enough acceleration
First KF:496; Map init KF:483
New Map created with 738 points

Hey!
I am running with my own monocular datasets (TUM form). The first issue I have reported in #21 is solved, Thank you! But I have met a new problem, when the system detect the loop, I meet "Segmentation fault (core dump)" again, the issue is:

./Examples/Monocular/mono_tum Vocabulary/ORBvoc.txt Examples/Monocular/TUM4.yaml ceiling_1_enhance 

ORB-SLAM3 Copyright (C) 2017-2020 Carlos Campos, Richard Elvira, Juan J. Gómez, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
ORB-SLAM2 Copyright (C) 2014-2016 Raúl Mur-Artal, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
This program comes with ABSOLUTELY NO WARRANTY;
This is free software, and you are welcome to redistribute it
under certain conditions. See LICENSE.txt.

Input sensor was set to: Monocular

Loading ORB Vocabulary. This could take a while...
Vocabulary loaded!

Creation of new map with id: 0
Creation of new map with last KF id: 0
Seq. Name: 

Camera Parameters: 
- fx: 726.287
- fy: 726.287
- cx: 354.65
- cy: 186.466
- bf: 0
- k1: 0
- k2: 0
- p1: 0
- p2: 0
- fps: 10
- color order: RGB (ignored if grayscale)

ORB Extractor Parameters: 
- Number of Features: 1000
- Scale Levels: 8
- Scale Factor: 1.2
- Initial Fast Threshold: 20
- Minimum Fast Threshold: 7

-------
Start processing sequence ...
Images in the sequence: 1592

First KF:0; Map init KF:0
New Map created with 243 points
Point distribution in KeyFrame: left-> 243 --- right-> 0
BoW: 165 matches between 1189 points with coarse Sim3
terminate called after throwing an instance of 'cv::Exception'
  what():  OpenCV(3.4.10) /home/gxytcrc/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'

Segmentation fault (core dumped)

I find the issue is in line 4001 of Optimizer.cc

    cv::Mat img1 = cv::imread(pKF1->mNameFile, CV_LOAD_IMAGE_UNCHANGED);
    cv::cvtColor(img1, img1, CV_GRAY2BGR);

The img1 returns empty value, and this cause the system crash.

is anyone met the same issue? or do anyone has the solution?

When i executted Kitti dataset, it meet a trouble like this:
Segmentation fault (core dumped)
Have someone meet this toruble like me and how to deal with ? tkanks!
All the thing like this:
`./stereo_kitti /home/jinln/jinln/ORB_SLAM3/Vocabulary/ORBvoc.txt /home/jinln/jinln/ORB_SLAM3/Examples/Stereo/KITTI03.yaml /home/jinln/jinln/DATASET/kitti/03
ORB-SLAM3 Copyright (C) 2017-2020 Carlos Campos, Richard Elvira, Juan J. Gómez, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
ORB-SLAM2 Copyright (C) 2014-2016 Raúl Mur-Artal, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
This program comes with ABSOLUTELY NO WARRANTY;
This is free software, and you are welcome to redistribute it
under certain conditions. See LICENSE.txt.

Input sensor was set to: Stereo

Loading ORB Vocabulary. This could take a while...
Vocabulary loaded!

Creation of new map with id: 0
Creation of new map with last KF id: 0
Seq. Name:

Camera Parameters:

• fx: 721.538
• fy: 721.538
• cx: 609.559
• cy: 172.854
• bf: 387.574
• k1: 0
• k2: 0
• p1: 0
• p2: 0
• fps: 20
• color order: RGB (ignored if grayscale)

ORB Extractor Parameters:

• Number of Features: 3000
• Scale Levels: 8
• Scale Factor: 1.2
• Initial Fast Threshold: 20
• Minimum Fast Threshold: 7

Depth Threshold (Close/Far Points): 21.486

Start processing sequence ...
Images in the sequence: 801

First KF:0; Map init KF:0
New Map created with 1315 points
Segmentation fault (core dumped)
`

👎 error: CMakeFiles/stereo euroc.dir/Examples/Stereo/stereo euroc.cc.o: undefined reference to symbol '_ZN2cv6String10deallocateEv'

Hi:
when i made the ORB-SLAM3, the following errors have occurred!
Anyone who can answer my problems? Thank you !

/home/xy/ORB-SLAM3/ORB_SLAM3/src/LocalMapping.cc:1455:12: warning: unused variable ‘t_inertial_only’ [-Wunused-variable]
double t_inertial_only = std::chrono::duration_cast<std::chrono::duration >(t1 - t0).count();
^
CMakeFiles/ORB_SLAM3.dir/build.make:110: recipe for target 'CMakeFiles/ORB_SLAM3.dir/src/LocalMapping.cc.o' failed
make[2]: *** [CMakeFiles/ORB_SLAM3.dir/src/LocalMapping.cc.o] Error 1
CMakeFiles/Makefile2:400: recipe for target 'CMakeFiles/ORB_SLAM3.dir/all' failed
make[1]: *** [CMakeFiles/ORB_SLAM3.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

Hi, thanks for your great work!
When I test orbslam3 using my own fisheye camera in MONOCULOR mode, the system can run successfully. But it can not correct the loop. In other words, it often fails. However the system is not broken, continue to execute. I guess the system does not detect the loop, or can not correct the loop. What should I do?

/home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/AR/ros_mono_ar.cc:83:23: error: ‘SLAM’ was not declared in this scope
viewerAR.SetSLAM(&SLAM);
^
/home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/AR/ros_mono_ar.cc: In member function ‘void ImageGrabber::GrabImage(const ImageConstPtr&)’:
/home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/AR/ros_mono_ar.cc:151:19: error: ‘mpSLAM’ was not declared in this scope
cv::Mat Tcw = mpSLAM->TrackMonocular(cv_ptr->image,cv_ptr->header.stamp.toSec());
^
/home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/AR/ros_mono_ar.cc:153:12: error: ‘MapPoint’ is not a member of ‘ORB_SLAM2’
vector<ORB_SLAM2::MapPoint*> vMPs = mpSLAM->GetTrackedMapPoints();
^
/home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/AR/ros_mono_ar.cc:153:12: note: suggested alternative:
In file included from /home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/../../../include/FrameDrawer.h:24:0,
from /home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/../../../include/Viewer.h:23,
from /home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/../../../include/Tracking.h:27,
from /home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/../../../include/System.h:32,
from /home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/AR/ros_mono_ar.cc:31:
/home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/../../../include/MapPoint.h:41:7: note: ‘ORB_SLAM3::MapPoint’
class MapPoint
^
/home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/AR/ros_mono_ar.cc:153:12: error: ‘MapPoint’ is not a member of ‘ORB_SLAM2’
vector<ORB_SLAM2::MapPoint*> vMPs = mpSLAM->GetTrackedMapPoints();
^
/home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/AR/ros_mono_ar.cc:153:12: note: suggested alternative:
In file included from /home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/../../../include/FrameDrawer.h:24:0,
from /home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/../../../include/Viewer.h:23,
from /home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/../../../include/Tracking.h:27,
from /home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/../../../include/System.h:32,
from /home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/AR/ros_mono_ar.cc:31:
/home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/../../../include/MapPoint.h:41:7: note: ‘ORB_SLAM3::MapPoint’
class MapPoint
^
/home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/AR/ros_mono_ar.cc:153:32: error: template argument 1 is invalid
vector<ORB_SLAM2::MapPoint*> vMPs = mpSLAM->GetTrackedMapPoints();
^
/home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/AR/ros_mono_ar.cc:153:32: error: template argument 2 is invalid
/home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/ros_stereo.cc:40:18: error: ‘ORB_SLAM2’ has not been declared
ImageGrabber(ORB_SLAM2::System* pSLAM):mpSLAM(pSLAM){}
^
/home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/ros_stereo.cc:40:35: error: expected ‘)’ before ‘’ token
ImageGrabber(ORB_SLAM2::System pSLAM):mpSLAM(pSLAM){}
^
/home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/ros_stereo.cc:44:5: error: ‘ORB_SLAM2’ does not name a type
ORB_SLAM2::System* mpSLAM;
^
/home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/ros_stereo.cc: In function ‘int main(int, char**)’:
/home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/ros_stereo.cc:62:5: error: ‘ORB_SLAM2’ has not been declared
ORB_SLAM2::System SLAM(argv[1],argv[2],ORB_SLAM2::System::STEREO,true);
^
/home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/ros_stereo.cc:64:23: error: ‘SLAM’ was not declared in this scope
ImageGrabber igb(&SLAM);
^
/home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/ros_stereo.cc: In member function ‘void ImageGrabber::GrabStereo(const ImageConstPtr&, const ImageConstPtr&)’:
/home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/ros_stereo.cc:161:9: error: ‘mpSLAM’ was not declared in this scope
mpSLAM->TrackStereo(imLeft,imRight,cv_ptrLeft->header.stamp.toSec());
^
/home/t450/orb_slam3_ws/src/ORB_SLAM3/Examples/ROS/ORB_SLAM3/src/ros_stereo.cc:165:9: error: ‘mpSLAM’ was not declared in this scope
mpSLAM->TrackStereo(cv_ptrLeft->image,cv_ptrRight->image,cv_ptrLeft->header.stamp.toSec());
^
CMakeFiles/MonoAR.dir/build.make:200: recipe for target 'CMakeFiles/MonoAR.dir/src/AR/ViewerAR.cc.o' failed
make[2]: *** [CMakeFiles/MonoAR.dir/src/AR/ViewerAR.cc.o] Error 1
make[2]: *** 正在等待未完成的任务....
CMakeFiles/MonoAR.dir/build.make:119: recipe for target 'CMakeFiles/MonoAR.dir/src/AR/ros_mono_ar.cc.o' failed
make[2]: *** [CMakeFiles/MonoAR.dir/src/AR/ros_mono_ar.cc.o] Error 1
CMakeFiles/Makefile2:820: recipe for target 'CMakeFiles/MonoAR.dir/all' failed
make[1]: *** [CMakeFiles/MonoAR.dir/all] Error 2
make[1]: *** 正在等待未完成的任务....
CMakeFiles/Mono.dir/build.make:119: recipe for target 'CMakeFiles/Mono.dir/src/ros_mono.cc.o' failed
make[2]: *** [CMakeFiles/Mono.dir/src/ros_mono.cc.o] Error 1
CMakeFiles/Makefile2:718: recipe for target 'CMakeFiles/Mono.dir/all' failed
make[1]: *** [CMakeFiles/Mono.dir/all] Error 2
CMakeFiles/RGBD.dir/build.make:119: recipe for target 'CMakeFiles/RGBD.dir/src/ros_rgbd.cc.o' failed
make[2]: *** [CMakeFiles/RGBD.dir/src/ros_rgbd.cc.o] Error 1
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/RGBD.dir/all' failed
make[1]: *** [CMakeFiles/RGBD.dir/all] Error 2
CMakeFiles/Stereo.dir/build.make:119: recipe for target 'CMakeFiles/Stereo.dir/src/ros_stereo.cc.o' failed
make[2]: *** [CMakeFiles/Stereo.dir/src/ros_stereo.cc.o] Error 1
CMakeFiles/Makefile2:104: recipe for target 'CMakeFiles/Stereo.dir/all' failed
make[1]: *** [CMakeFiles/Stereo.dir/all] Error 2
Makefile:127: recipe for target 'all' failed
make: *** [all] Error 2

Opencv/cv.h file was cancelled in opencv4, I used opencv2/opencv.hpp instead, but in PnPsolver.h and PnPsolver.cc, cvMat is used for svd calculation. Because opencv2 cancels the definitions of cvMat and cvSVD, So if opencv4 is used, how to solve these problems?
I think you are using opencv3.2. There should be the same problem. How did you solve it?

I have tried the Monocular version with customized streaming sequences, which worked well with ORB-SLAM2. However, with ORB-SLAM3, for some sequences, whenever "BoW: xxx matches between xxx points with coarse Sim3" occurs, the system throws a cv exception as below:

BoW: 276 matches between 635 points with coarse Sim3
terminate called after throwing an instance of 'cv::Exception'
what(): OpenCV(4.4.0-pre) /home/qi/opencv_build/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'
Aborted (core dumped)

I can confirm that the file exists and is not corrupted. By printing the loaded image, we can see that the exception may happen at different frame ids of the same sequence and the image was loaded successfully in mono_euroc.cc. I assume it might be a cvtColor exception in LoopClosing.cc? Please find in below two error printings when running the same sequence:

I am trying to let ORBSLAM3 use descriptor of 128 bits rather than 256 bits hopping to save some time.

ORBextractor.cc:
for (int i = 0; i < 16; ++i, pattern += 16) // edited ori: 32
static int bit_pattern_31_[128*4] = // edited, ori: 256
const int npoints = 256; // edited, ori:512
descriptors = Mat::zeros((int)keypoints.size(), 16, CV_8UC1); // edited, ori:32
_descriptors.create(nkeypoints, 16, CV_8U); // edited, ori:32
Mat desc = cv::Mat(nkeypointsLevel, 16, CV_8U); // edited: ori:32

ORBmatcher.cc
int bestDist=128; // edited, ori:256
for(int i=0; i<4; i++, pa++, pb++)//edited, ori:i<8
const int ORBmatcher::TH_HIGH = 60;
const int ORBmatcher::TH_LOW = 30;

Then I get this

Features points showing in the frame indicates tracking is still good. But it seems that ORBSLAM3 can get to know the rotation of the camera well but can't get the right translation (thus the camera always stays at the same place.) I don't know where else I need to change also. Thanks.

Within the script tum_vi_example.sh, the TUM_VI.yaml should be the configuration file.
Where can I find it? Is it called TUM_512.yaml now?

How do I create the global map from a monocular camera and imu? How do I use the map for realtime localization?

I try to run the example of mono_inertial_tum_vi, but error come out when start the .sh file.

The data is Euroc / DSO 512x512 dataset
link :

http://vision.in.tum.de/tumvi/exported/euroc/512_16/dataset-room4_512_16.tar

The error like below:

sph@sph-System-Product-Name:~/Documents/ORB_SLAM3$ ./tum_vi.sh 
Launching Room 4 with Monocular-Inertial sensor
num_seq = 1
file name: dataset-room4_512_monoi
Loading images for sequence 0.../home/sph/Downloads/dataset-room4_512_16/mav0/cam0/data
Examples/Monocular-Inertial/TUM_TimeStamps/dataset-room4_512.txt
LOADED!
Loading IMU for sequence 0...LOADED!

-------

ORB-SLAM3 Copyright (C) 2017-2020 Carlos Campos, Richard Elvira, Juan J. Gómez, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
ORB-SLAM2 Copyright (C) 2014-2016 Raúl Mur-Artal, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
This program comes with ABSOLUTELY NO WARRANTY;
This is free software, and you are welcome to redistribute it
under certain conditions. See LICENSE.txt.

Input sensor was set to: Monocular-Inertial

Loading ORB Vocabulary. This could take a while...
Vocabulary loaded!

Creation of new map with id: 0
Creation of new map with last KF id: 0
Seq. Name: dataset-room4_512_monoi

Camera Parameters: 
- fx: 190.97846984863281
- fy: 190.97331237792969
- cx: 254.93170166015625
- cy: 256.89743041992188
- bf: 0
- k1: 0
- k2: 0
- p1: 0
- p2: 0
- k3: -0.0020532361231744289
- fps: 20
- color order: RGB (ignored if grayscale)

ORB Extractor Parameters: 
- Number of Features: 1500
- Scale Levels: 8
- Scale Factor: 1.2000000476837158
- Initial Fast Threshold: 20
- Minimum Fast Threshold: 7

Left camera to Imu Transform (Tbc): 
[-0.99952501, 0.0075019184, -0.029890131, 0.045574836;
 0.029615344, -0.03439736, -0.99896932, -0.071161799;
 -0.008522328, -0.99938005, 0.034158852, -0.044681255;
 0, 0, 0, 1]

IMU frequency: 200 Hz
IMU gyro noise: 0.00015999999595806003 rad/s/sqrt(Hz)
IMU gyro walk: 2.2000000171829015e-05 rad/s^2/sqrt(Hz)
IMU accelerometer noise: 0.00279999990016222 m/s^2/sqrt(Hz)
IMU accelerometer walk: 0.00085999997099861503 m/s^3/sqrt(Hz)

.pnged to load image at: /home/sph/Downloads/dataset-room4_512_16/mav0/cam0/data/1520531124150444163
./tum_vi.sh: line 17: 11368 Segmentation fault      (core dumped) ./Examples/Monocular-Inertial/mono_inertial_tum_vi Vocabulary/ORBvoc.txt Examples/Monocular-Inertial/TUM_512.yaml "$pathDatasetTUM_VI"/dataset-room4_512_16/mav0/cam0/data Examples/Monocular-Inertial/TUM_TimeStamps/dataset-room4_512.txt Examples/Monocular-Inertial/TUM_IMU/dataset-room4_512.txt dataset-room4_512_monoi
sph@sph-System-Product-Name:~/Documents/ORB_SLAM3$ ./tum_vi.sh 
Launching Room 4 with Monocular-Inertial sensor
num_seq = 1
file name: dataset-room4_512_monoi
Loading images for sequence 0.../home/sph/Downloads/dataset-room4_512_16/mav0/cam0/data
Examples/Monocular-Inertial/TUM_TimeStamps/dataset-room4_512.txt
LOADED!
Loading IMU for sequence 0...LOADED!

-------

ORB-SLAM3 Copyright (C) 2017-2020 Carlos Campos, Richard Elvira, Juan J. Gómez, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
ORB-SLAM2 Copyright (C) 2014-2016 Raúl Mur-Artal, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
This program comes with ABSOLUTELY NO WARRANTY;
This is free software, and you are welcome to redistribute it
under certain conditions. See LICENSE.txt.

Input sensor was set to: Monocular-Inertial

Loading ORB Vocabulary. This could take a while...
Vocabulary loaded!

Creation of new map with id: 0
Creation of new map with last KF id: 0
Seq. Name: dataset-room4_512_monoi

Camera Parameters: 
- fx: 190.97846984863281
- fy: 190.97331237792969
- cx: 254.93170166015625
- cy: 256.89743041992188
- bf: 0
- k1: 0
- k2: 0
- p1: 0
- p2: 0
- k3: -0.0020532361231744289
- fps: 20
- color order: RGB (ignored if grayscale)

ORB Extractor Parameters: 
- Number of Features: 1500
- Scale Levels: 8
- Scale Factor: 1.2000000476837158
- Initial Fast Threshold: 20
- Minimum Fast Threshold: 7

Left camera to Imu Transform (Tbc): 
[-0.99952501, 0.0075019184, -0.029890131, 0.045574836;
 0.029615344, -0.03439736, -0.99896932, -0.071161799;
 -0.008522328, -0.99938005, 0.034158852, -0.044681255;
 0, 0, 0, 1]

IMU frequency: 200 Hz
IMU gyro noise: 0.00015999999595806003 rad/s/sqrt(Hz)
IMU gyro walk: 2.2000000171829015e-05 rad/s^2/sqrt(Hz)
IMU accelerometer noise: 0.00279999990016222 m/s^2/sqrt(Hz)
IMU accelerometer walk: 0.00085999997099861503 m/s^3/sqrt(Hz)

.pnged to load image at: /home/sph/Downloads/dataset-room4_512_16/mav0/cam0/data/1520531124150444163
./tum_vi.sh: line 17: 11882 Segmentation fault      (core dumped) ./Examples/Monocular-Inertial/mono_inertial_tum_vi Vocabulary/ORBvoc.txt Examples/Monocular-Inertial/TUM_512.yaml "$pathDatasetTUM_VI"/dataset-room4_512_16/mav0/cam0/data Examples/Monocular-Inertial/TUM_TimeStamps/dataset-room4_512.txt Examples/Monocular-Inertial/TUM_IMU/dataset-room4_512.txt dataset-room4_512_monoi

The tum_vi.sh file is

#!/bin/bash
pathDatasetTUM_VI='/home/sph/Downloads' #Example, it is necesary to change it by the dataset path
#------------------------------------
# Monocular-Inertial Examples
echo "Launching Room 4 with Monocular-Inertial sensor"
./Examples/Monocular-Inertial/mono_inertial_tum_vi Vocabulary/ORBvoc.txt Examples/Monocular-Inertial/TUM_512.yaml "$pathDatasetTUM_VI"/dataset-room4_512_16/mav0/cam0/data Examples/Monocular-Inertial/TUM_TimeStamps/dataset-room4_512.txt Examples/Monocular-Inertial/TUM_IMU/dataset-room4_512.txt dataset-room4_512_monoi

The exec file is in the git

https://github.com/shanpenghui/my_orbslam3.git

Thanks for helping.