eth3d / badslam Goto Github PK

Thanks for your dataset SLAM Benchmark. I wonder if you have the ROS bag version of your dataset?

Hi, guys. nice job!
But I have some problem to run it. I need your help.
Thx.
environment: Ubuntu18.04 with cuda 9.2 gtx1070.
g2o from https://github.com/RainerKuemmerle/g2o master branch.
I make the badslam and badslam_test with no problem. I tested on the latest master branch and 1.01.
The problem is:

1. badslam_test run ended with Aborted (core dumped), which outputs:

########################################################
[ RUN ] Optimization.PoseGraphOptimizer
2019-07-23 16:43:44.619 ( 46.584s) [ 2A579F80]pose_graph_optimizer.cc:127 INFO| - Performing pose graph optimization ...
int g2o::csparse_extension::cs_cholsolsymb(const cs_di*, number_t*, const cs_dis*, number_t*, int*): cholesky failed!
Cholesky failure, writing debug.txt (Hessian loadable by Octave)
2019-07-23 16:43:44.621 ( 46.586s) [ 2A579F80]pose_graph_optimizer.cc:131 INFO| - Pose graph optimization done
double free or corruption (out)
Aborted (core dumped)
########################################################

I look into it, I find it is related with g2o csparse_extension.cpp function cs_cholsolsymb.
with this lines:
########################################################
N = cs_chol_workspace (A, S, work, x) ; /* numeric Cholesky factorization */
if (!N) {
cerr << PRETTY_FUNCTION << ": cholesky failed!" << endl;
/assert(0);/
}
########################################################

I am not familiar with g2o, and just start learning. I think it is not the cuda core problem which writed in the document. But I cant solve the csparse_solver problem, maybe another solver(Cholmod solver) will resolve it.

2. The badslam with realsense live frame work properly at the first few frames, the following frames come with incorrect results. I think it is caused by the failure of Pose graph optimization. output warning:16:37:41.236 direct_ba_alternating.c:249 WARN| Pose estimation not converged (not_converged_count: 37, call_counter: 204239)

This problem may relate with
#18

hello,friends. i have successfully compiled "badslam&&badslam_test". But when i run "./applications/badslam/badslam_test". some error occurs,as shown below. can you give me some advice? thank you!! it is tested on CUDA10.0 &&&& GTX 1070 !!

Hi guys,

I have a Kinect Sensor v2 and I can read RGB-D which from sensor coming data with OPENNI library. Is that enough for use in BadSlam?

Given that pairwise frame tracking seems to be very fast, in the odometry part, one could try to always track wrt. multiple previous keyframes (separately) instead of only wrt. the latest keyframe. This would allow to average the results, and also enable detecting tracking issues (when the results of tracking wrt. different keyframes differ too much).

This perhaps only applies to the case of not using an IMU, since using an IMU might robustify the local tracking to the point where such measures would not really help anymore.

When I use Kinect for Azure run badslam, I got the following message:
6:03:53.713 direct_ba_alternating.c:249 WARN| Pose estimation not converged (not_converged_count: 253, call_counter: 35820)
16:03:53.756 direct_ba_alternating.c:249 WARN| Pose estimation not converged (not_converged_count: 254, call_counter: 36157)
16:03:54.082 direct_ba_alternating.c:249 WARN| Pose estimation not converged (not_converged_count: 255, call_counter: 36821)

could you anyone know why?

Hi, I actually have manually installed DLib to /usr/local/lib, how should I define DLib_DIR ?

I don't have
DLibConfig.cmake
dlib-config.cmake
but with the following dlib.pc

% cat /usr/local/lib/pkgconfig/dlib.pc 
libdir=/usr/local/lib
includedir=/usr/local/include

Name: dlib
Description: Numerical and networking C++ library
Version: 19.17.0
Libs: -L${libdir} -ldlib
Cflags: -I${includedir}
Requires: libpng

The current ERROR is:

 CMake Error at applications/badslam/CMakeLists.txt:7 (find_package):
   By not providing "FindDLib.cmake" in CMAKE_MODULE_PATH this project has
   asked CMake to find a package configuration file provided by "DLib", but
   CMake did not find one.

   Could not find a package configuration file provided by "DLib" with any of
   the following names:

     DLibConfig.cmake
     dlib-config.cmake

   Add the installation prefix of "DLib" to CMAKE_PREFIX_PATH or set
   "DLib_DIR" to a directory containing one of the above files.  If "DLib"
   provides a separate development package or SDK, be sure it has been
   installed.

Can anybody help to fix this issue?

Thank you
Pei

Currently, there is no in-built option to record the video when using live input.

Hi,

I'm trying to run BADSLAM on one of the mannequin datasets and run into the following error:
Do you have any idea what I can do ? Any help is appreciated, thanks.

Stack trace:
31 0x559502adb54a ./applications/badslam/badslam(+0x3054a) [0x559502adb54a]
30 0x7f4ee0b96b97 __libc_start_main + 231
29 0x559502ad3ed1 ./applications/badslam/badslam(+0x28ed1) [0x559502ad3ed1]
28 0x559502add471 ./applications/badslam/badslam(+0x32471) [0x559502add471]
27 0x559502b30052 ./applications/badslam/badslam(+0x85052) [0x559502b30052]
26 0x7f4ee2dffe95 QWidget::setVisible(bool) + 405
25 0x7f4ee2dfcd5f QWidgetPrivate::show_helper() + 63
24 0x7f4ee2dfccf0 QWidgetPrivate::showChildren(bool) + 240
23 0x7f4ee2dffe95 QWidget::setVisible(bool) + 405
22 0x7f4ee2dfcd5f QWidgetPrivate::show_helper() + 63
21 0x7f4ee2dfccf0 QWidgetPrivate::showChildren(bool) + 240
20 0x7f4ee2dffe95 QWidget::setVisible(bool) + 405
19 0x7f4ee2dfcd5f QWidgetPrivate::show_helper() + 63
18 0x7f4ee2dfccf0 QWidgetPrivate::showChildren(bool) + 240
17 0x7f4ee2dffe95 QWidget::setVisible(bool) + 405
16 0x7f4ee2dfcd43 QWidgetPrivate::show_helper() + 35
15 0x7f4ee2df91cd QWidgetPrivate::sendPendingMoveAndResizeEvents(bool, bool) + 445
14 0x7f4ee1f1ec18 QCoreApplication::notifyInternal2(QObject*, QEvent*) + 264
13 0x7f4ee2dc99f0 QApplication::notify(QObject*, QEvent*) + 688
12 0x7f4ee2dc285c QApplicationPrivate::notify_helper(QObject*, QEvent*) + 156
11 0x7f4ee2e009c2 QWidget::event(QEvent*) + 2162
10 0x7f4ee2e1efc1 QOpenGLWidget::resizeEvent(QResizeEvent*) + 177
9 0x7f4ee2df9215 QWidgetPrivate::sendPaintEvent(QRegion const&) + 37
8 0x7f4ee1f1ec18 QCoreApplication::notifyInternal2(QObject*, QEvent*) + 264
7 0x7f4ee2dc99f0 QApplication::notify(QObject*, QEvent*) + 688
6 0x7f4ee2dc285c QApplicationPrivate::notify_helper(QObject*, QEvent*) + 156
5 0x7f4ee2e00348 QWidget::event(QEvent*) + 504
4 0x7f4ee2e1f145 /home/towerman/Qt/5.12.0/gcc_64/lib/libQt5Widgets.so.5(+0x1bc145) [0x7f4ee2e1f145]
3 0x7f4ee3984ef9 vis::RenderWidgetOpenGL::paintGL() + 889
2 0x559502ca0313 ./applications/badslam/badslam(+0x1f5313) [0x559502ca0313]
1 0x559502c9cfae ./applications/badslam/badslam(+0x1f1fae) [0x559502c9cfae]
0 0x7f4ee36e4163 loguru::StreamLogger::~StreamLogger() + 163
14:39:46.997 render_window.cc:963 FATL| Cuda Error: unknown error

Originally posted by @Young532042725 in #2 (comment):

@puzzlepaint hi, there are some problem occur, when building(make). would you please give me some advice? thank you very much! There are cmake and make

cmake -DCMAKE_CUDA_FLAGS="-arch=sm_61" ..
-- Boost version: 1.58.0
-- Found the following Boost libraries:
-- serialization
-- filesystem
-- system
-- Symbol prefix:
-- Boost version: 1.58.0
DLib library found, using it from the system
CMake Warning at applications/badslam/CMakeLists.txt:21 (find_package):
By not providing "Findrealsense2.cmake" in CMAKE_MODULE_PATH this project
has asked CMake to find a package configuration file provided by
"realsense2", but CMake did not find one.

Could not find a package configuration file provided by "realsense2" with
any of the following names:

realsense2Config.cmake
realsense2-config.cmake

Add the installation prefix of "realsense2" to CMAKE_PREFIX_PATH or set
"realsense2_DIR" to a directory containing one of the above files. If
"realsense2" provides a separate development package or SDK, be sure it has
been installed.

CMake Warning at applications/badslam/CMakeLists.txt:24 (find_package):
By not providing "Findk4a.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "k4a", but
CMake did not find one.

Could not find a package configuration file provided by "k4a" with any of
the following names:

k4aConfig.cmake
k4a-config.cmake

Add the installation prefix of "k4a" to CMAKE_PREFIX_PATH or set "k4a_DIR"
to a directory containing one of the above files. If "k4a" provides a
separate development package or SDK, be sure it has been installed.

CMake Warning at applications/badslam/CMakeLists.txt:25 (find_package):
By not providing "Findk4arecord.cmake" in CMAKE_MODULE_PATH this project
has asked CMake to find a package configuration file provided by
"k4arecord", but CMake did not find one.

Could not find a package configuration file provided by "k4arecord" with
any of the following names:

k4arecordConfig.cmake
k4arecord-config.cmake

Add the installation prefix of "k4arecord" to CMAKE_PREFIX_PATH or set
"k4arecord_DIR" to a directory containing one of the above files. If
"k4arecord" provides a separate development package or SDK, be sure it has
been installed.

-- The following OPTIONAL packages have been found:

• PythonInterp

-- The following REQUIRED packages have been found:

• ZLIB
• GLEW
• Qt5Core
• Qt5Gui (required version >= 5.5.1)
• Qt5OpenGL
• Qt5Widgets
• Qt5
• Qt5X11Extras
• SuiteSparse
• Eigen3
• g2o
• OpenGL
• Threads
• X11
• OpenCV
• Boost
• DLib
• opengv

-- The following OPTIONAL packages have not been found:

• realsense2
• k4a
• k4arecord

-- Configuring done
-- Generating done
-- Build files have been written to: /home/gp/BAD-SLAM/badslam/build

Originally posted by @Young532042725 in #2 (comment)

Hi guys,

I am trying to run BAD_SLAM on my laptop with GTX 850M (Compute Capability 5.0) in Ubuntu 1804.

I have passed "make -j badslam" and "make -j badslam_test".

However, I met an error while running Auto_tunning as shown below

I tried to solve this with this method, but it was not working: #34

Please help me with this and I can provide more logs if needed. Thanks in advance!

Hi, I followed the steps in the readme. However, it encounters error as follows:

/home/B0229/Desktop/study/badslam-master/applications/badslam/third_party/DBoW2/src/FBrief.cpp:14:29: fatal error: DVision/DVision.h: No such file or directory
 #include <DVision/DVision.h>
                             ^
compilation terminated.
[ 31%] Built target png
/home/B0229/Desktop/study/badslam-master/applications/badslam/third_party/DBoW2/src/FORB.cpp:16:27: fatal error: DUtils/DUtils.h: No such file or directory
 #include <DUtils/DUtils.h>
                           ^
compilation terminated.
[ 31%] Automatic MOC for target libvis
applications/badslam/third_party/DBoW2/CMakeFiles/DBoW2.dir/build.make:101: recipe for target 'applications/badslam/third_party/DBoW2/CMakeFiles/DBoW2.dir/src/FORB.cpp.o' failed
make[3]: *** [applications/badslam/third_party/DBoW2/CMakeFiles/DBoW2.dir/src/FORB.cpp.o] Error 1
make[3]: *** Waiting for unfinished jobs....
applications/badslam/third_party/DBoW2/CMakeFiles/DBoW2.dir/build.make:88: recipe for target 'applications/badslam/third_party/DBoW2/CMakeFiles/DBoW2.dir/src/FBrief.cpp.o' failed
make[3]: *** [applications/badslam/third_party/DBoW2/CMakeFiles/DBoW2.dir/src/FBrief.cpp.o] Error 1

It seems DUtils/DUtils.h and DVision/DVision.h are missing

Any suggestion to fix it?

Thanks~

Hello there,

I met a question while I was evaluating my result.

I ran the test datasets from this website "https://www.eth3d.net/slam_datasets" and got my results.

Then I used "evaluate_ate.py" to evaluate my result. I added the addresses of my result file and imu file in the command. However, the ATE RMSE is about 0.98 m which is obviously incorrect.

I have two questions:

1. There are 4 files named imu.txt in each dataset, what is the difference?

2. I found an option named offset in "evaluate_ate.py", but I do not understand how to use it, I think probably this is what I should use to correct my evaluation.

Sorry for bothering you if there is some relative information I missed.

Thanks for your time, I am looking forward to your response.

Best regards,
Arete

hi, friends. Today, i build it on a server, named aaaa, my personal account name is yjs. As is shown below. The aaaa is already installed CUDA. in aaaa run "env | grep "cuda" " has shown. But the CMakeLists.txt indicate FATAL_ERROR "CMake did not find CUDA, which is required to build this program.". As is shown below. can you give me some help? thanks!!!

When I compile this slam, I was confronted with the following errors:
[ 94%] Building CXX object CMakeFiles/libvis_test.dir/libvis/src/libvis/test/lm_optimizer.cc.o
In file included from /home/manifold/badslam/./libvis/src/libvis/lm_optimizer.h:35:0,
from /home/manifold/badslam/libvis/src/libvis/test/lm_optimizer.cc:34:
/usr/include/cublasXt.h:57:10: fatal error: driver_types.h: no this file or directory
#include "driver_types.h"
^~~~~~~~~~~~~~~~
compilation terminated.
CMakeFiles/libvis_test.dir/build.make:182: recipe for target 'CMakeFiles/libvis_test.dir/libvis/src/libvis/test/lm_optimizer.cc.o' failed
make[2]: *** [CMakeFiles/libvis_test.dir/libvis/src/libvis/test/lm_optimizer.cc.o] Error 1
Could anyone know why?
Thank you very much

Currently, the program will not detect when it fails to track the camera pose, giving wrong results. It would be good if it could determine failure and state something like "tracking lost" instead.

If tracking continues after a failure, the program should start building a new model, while trying to merge the new model with the old one by re-localizing against it, if possible.

Currently, the keyframes are stored in a std::vector, and removing keyframes leaves nullptr entries in that vector. This is likely not to be handled correctly in all parts of the codebase, leading to crashes. It would most likely be preferable to store the keyframes in an unordered_map instead (mapping keyframe_id -> keyframe_pointer) and delete removed keyframes from it entirely.

Currently, non-keyframes almost do not contribute to the final result (other than helping initialize the pose estimation for their follow-up keyframe). It might be worth trying to fuse their depth data with the closest keyframe to possibly obtain denser and less noisy depth in keyframes (while however also introducing possible occlusion issues with this reprojection, if something is projected into a keyframe which is actually not visible from its viewpoint).

It is often desirable to obtain a surface (mesh) reconstruction rather than only a point cloud, which is the current reconstruction output of BAD SLAM. Some possible options might be:

• Integrate Poisson Surface Reconstruction as a post-processing step (such that it can be conveniently invoked with a single click from the GUI)

• Integrate SurfelMeshing for live meshing during the reconstruction. A good integration might not be straightforward however, since:

  • BAD SLAM is constantly creating new surfels and merging surfels in existing areas, which SurfelMeshing might have trouble keeping up with.
  • For SLAM, one wants to sparsify the surfels a bit to keep it fast, while for reconstruction, the surfels should be as dense as possible.

This can be tested with the "large_loop_3" dataset of the ETH3D SLAM benchmark.

I'd like to point out that you really have a great project!
Unfortunaltely I am unable to find the Windows binaries.
From the Readme:

For Windows, an executable compiled with Visual Studio 2019 is provided

Could you please provide the link for the binary?

Best

After getting badslam running with the K4A (it runs in real time on my notebook which is impressive!) I was wondering if it is possible to replay a recording (.mkv file) generated with the k4arecorder command line tool. While replay works with the k4aviewer I am not sure if it can be replayed with the SDK. Another approach would be to extract image sequences from the mkv and process them to match the ETH3D SLAM dataset format. Has this been done before?

Hi,

Thanks for your great work!
I'm trying to compile badslam, however some errors that may relate to the shipped DBoW2 package, appear as follows:
/home/qin/Documents/codes/3rdparty/bin_3rdparty/DLib/include/DLib/../DVision/BRIEF256.h: In member function ‘void DVision::BRIEF_t<Bits>::compute(const cv::Mat&, const std::vector<cv::KeyPoint>&, std::vector<std::bitset<Bits> >&, bool) const’: /home/qin/Documents/codes/3rdparty/bin_3rdparty/DLib/include/DLib/../DVision/BRIEF256.h:207:11: error: ‘cvtColor’ is not a member of ‘cv’ cv::cvtColor(image, aux, cv::COLOR_RGB2GRAY);

Note:

1. I have pre-compiled DLib and installed the compiled stuff to the directory '/home/qin/Documents/codes/3rdparty/bin_3rdparty/DLib' using '-DCMAKE_INSTALL_PREFIX';
2. My opencv version is 3.2.0 as shown below:
   $ apt list -a libopencv-dev Listing... Done libopencv-dev/bionic-updates,bionic-security,now 3.2.0+dfsg-4ubuntu0.1 amd64 [installed,automatic] libopencv-dev/bionic 3.2.0+dfsg-4build2 amd64

Another way to reproduce the error:

1. Copy the shipped directory DBoW2 to other place, e.g., /home/DoW2;
2. $ cd /home/DoW2 && mkdir build && cd build && cmake .. && make -j8

The same errors will occur.

Is there any solution to solve this compilation problem? Many thanks!

Best regards,
Qin Shi

If using surfel densification, especially if setting the minimum observation count to 1, there can be quite some noise in surfaces. This is likely because of surfels with roughly correct position but a wrong normal vector, which won't be affected by the free-space outlier removal.

If they were included in that outlier removal, then it might possibly cause issues for the reconstruction of thin surfaces, or generally object edges, however, since in these cases, surfels with strongly differing normals are close together but should not cancel each other out.

An alternative way to reduce the apparent noise might be to store the surfel normal in the vertex data used for rendering, and hide a surfel if its normal points away from the camera (i.e., backface culling for surfels).

Hi,

we are trying to run your project on windows 10, but face the following issue: When we try to build the solution in Visual Studio, we get an error in file facade_iterator_category.h of the Boost library. It says
2>Unknown compiler version - please run the configure tests and report the results
2>D:\xyz\projects\xyz\BADSLAM\dependencies\boost_1_54_0\boost/iterator/detail/facade_iterator_category.hpp(166): error C2039: 'assert_not_arg': is not a member of 'boost::mpl'

We use Visual Studio 2012 to build the Boost library (version 1.54.0) and then build the whole Badslam project in Visual Studio 2017.
Do you have an idea what could be the reason for this? Any help would be appreciated. Thanks a lot!

Recent depth cameras such as the Microsoft Azure Kinect, the Occipital Structure Core, and the Intel RealSense D435i all contain an IMU. It would be very helpful for robust pose tracking to integrate an IMU term into the cost function.

Currently, the program can only load a single dataset, which may limit the ability to create large reconstructions. It would be nice to be able to load multiple datasets. At first, this could be limited to datasets from the same camera, such that still only a single set of camera intrinsics needs to be considered.

Hello, thank you for sharing this great work !

I have a question about the surfel position optimization.

During the optimization, the surfels are only allowed to move along their normal directions and a parametrization P_new = P_old + t * normal is used. In the code, I believe the update is done here (without the descriptor residual).

The thing that is bothering me is that the jacobian used for this optimization is the one w.r.t. the depth (in this function) and not w.r.t to the parameter "t".

Could you help me understand the reason why the increment to the depth is directly applied to the parameter "t" ?

I am trying to run badslam on TUM rgbd dataset. In the readme I found that the calibration.txt only has pinhole camera model without any distortion parameter.
Will the code work if I simply add some distortion parameter after camera intrinsics? Like

fx fy cx cy d1 d2 d3 d4 d5

In rgbd_video_io_tum_dataset.h I didn't find any piece of code that can read distortion parameter.
Also in the readme seems for tum rgbd I don't need to input depth calibration file?
I really appreciate your answer!

Hi guys,nice job!
I'm trying to run BADSLAM on one of the cable_1 datasets and run into the following error:
Do you have any idea what I can do ? Any help is appreciated, thanks.
bruce@bruce-desktop:~/slam/badslam-master/build$ ./applications/badslam/badslam /home/bruce/slam/badslam-master/dataset/cables_1/ --gui_run
time file:line v|
15:26:48.718 loguru.cpp:587 INFO| arguments: ./applications/badslam/badslam /home/bruce/slam/badslam-master/dataset/cables_1/ --gui_run
15:26:48.718 loguru.cpp:590 INFO| Current dir: /home/bruce/slam/badslam-master/build
15:26:48.718 loguru.cpp:592 INFO| stderr verbosity: 2
15:26:48.718 loguru.cpp:593 INFO| -----------------------------------
Segment error (core dumped)

Can this project be compiled on the Jetson nano?

Can the software in the current state somehow be used together with the Azure Kinect (k4a) device?
I have tried it with release 1.0.1 under Ubuntu 18.04 with no success. Or is that only possible when building from source? I am also struggling with compiling it or rather getting all the dependencies installed. More detailed instructions on how to install and configure everything would be tremendously helpful.
Edit: It seams the release build 1.0.1 is not compatible with the K4A:
input_azurekinect.h:144 FATL| Azure Kinect input requested, but the program was compiled without Azure Kinect support.

I've tried to set the CMAKE_CUDA_FLAGS using "-DCMAKE_CUDA_FLAGS="-arch=sm_61" or other versions and I've had the CUDA10.1 installed with Ubuntu 14.04, yet I still couldn't get my cuda properly set.

I've also tried adding
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -gencode arch=compute_61,code=sm_61")
message("CMAKE_CUDA_FLAGS: ${CMAKE_CUDA_FLAGS}")

to my CMakelist and it shows:
CMAKE_CUDA_FLAGS: -arch=sm_61 -gencode arch=compute_61,code=sm_61

hi friends, i am using this SLAM,
"make -j badslam"
"make -j badslam_test" are all ok.
But, when i run the "./build_RelWithDebInfo/applications/badslam/badslam_test", some errors occur!
I use an Nvidia GeForce GT730 graphic card. CUDA 10.0. Would you please give me some advice? Thanks!!

I am trying to use the application with Azure Kinect through live input. After hassling with dependencies I am stuck at a point which I think not far from the the successful end. Here is the current state:

• Building & compiling is OK without an error. All necessary packages are found. Optional packages realsense and structure are not found which I don't intend to use.
• k4aviewer succesfully shows depth and RGB inputs.
• When I run ./badslam and select Azure K4A Live Input this is the error message

time                            file:line     v| 
15:02:22.304              loguru.cpp:587   INFO| arguments: ./badslam
15:02:22.304              loguru.cpp:590   INFO| Current dir: /home/omer/projects/badslam/build_RelWithDebInfo/applications/badslam
15:02:22.304              loguru.cpp:592   INFO| stderr verbosity: 2
15:02:22.304              loguru.cpp:593   INFO| -----------------------------------
15:02:22.304                 main.cc:439   WARN| No auto-tuning file found -> using default parameters. GPU performance is thus probably slightly worse than it could be.
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
[2019-12-02 15:02:28.858] [error] [t=7833] /home/vsts/work/1/s/extern/Azure-Kinect-Sensor-SDK/src/calibration/calibration.c (391): get_camera_calibration(). cJSON_Parse error .50256603956222534,0.5046575665473938,0.49263244867324829,0.492673397064209,4.5602192878723145,3.1884105205535889,0.17583206295967102,4.8896327018737793,4.7015571594238281,0.89637881517410278,0,0,3.2404401281382889E-5,-0.0001623422431293875],"ModelType":"CALIBRATION_LensDistortionModelBrownConrady"},"Location":"CALIBRATION_CameraLocationD0","Purpose":"CALIBRATION_CameraPurposeDepth","MetricRadius":1.7399997711181641,"Rt":{"Rotation":[1,0,0,0,1,0,0,0,1],"Translation":[0,0,0]},"SensorHeight":1024,"SensorWidth":1024,"Shutter":"CALIBRATION_ShutterTypeUndefined","ThermalAdjustmentParams":{"Params":[0,0,0,0,0,0,0,0,0,0,0,0]}},{"Intrinsics":{"ModelParameterCount":14,"ModelParameters":[0.50009459257125854,0.50738322734832764,0.46948739886283875,0.62577122449874878,0.75097411870956421,-2.8726491928100586,1.6259859800338745,0.62961763143539429,-2.710111141204834,1.5602593421936035,0,0,-0.000392389134503901,0.0010423335479572415],"ModelType":"CALIBRATION_LensDistortionModelBr
[2019-12-02 15:02:28.858] [error] [t=7833] /home/vsts/work/1/s/extern/Azure-Kinect-Sensor-SDK/src/sdk/k4a.c (154): calibration_create(device->depthmcu, &device->calibration) returned failure in k4a_device_open()
Stack trace:
5       0x7f3704ef188f clone + 63
4       0x7f370760e6db /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7f370760e6db]
3       0x7f370583466f /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xbd66f) [0x7f370583466f]
2       0x560aff517d68 ./badslam(+0x74d68) [0x560aff517d68]
1       0x560aff530ce9 ./badslam(+0x8dce9) [0x560aff530ce9]
0       0x7f370783cf52 loguru::StreamLogger::~StreamLogger() + 162
15:02:28.859    input_azurekinect.cc:283   FATL| WARNING: Failed to open K4A device!
Aborted

• Result of badslam_test can be seen in this link. (Since it was too long, I've linked the file)
• Since it says Cuda Error: too many resources requested for launch I've followed the instructions in the README. Auto tuning CUDA block size.
  To do that I need a dataset, live input would not work as it is stated in the README. I've downloaded Cable1 from ETH3D SLAM benchmark. The autotune asks for associated.txt calibration.txt depth(folder) rgb(folder). So I downloaded all of cables_1_mono.zip, cables_1_stereo.zip, cables_1_rgbd.zip, cables_1_imu.zip, cables_1_raw.zip and combine the data in them to get associated.txt calibration.txt depth(folder) rgb(folder).

Then I run this command to auto tune.
python3 scripts/auto_tune_parameters.py ~/projects/badslam/build_RelWithDebInfo/applications/badslam/badslam ~/projects/cables_1 --sequential_ba --sequential_loop_detection

Output is

Running: /home/omer/projects/badslam/build_RelWithDebInfo/applications/badslam/badslam /home/omer/projects/cables_1 --sequential_ba --sequential_loop_detection --auto_tuning_iteration 0
time                            file:line     v| 
14:45:06.990              loguru.cpp:587   INFO| arguments: /home/omer/projects/badslam/build_RelWithDebInfo/applications/badslam/badslam /home/omer/projects/cables_1 --sequential_ba --sequential_loop_detection --auto_tuning_iteration 0
14:45:06.990              loguru.cpp:590   INFO| Current dir: /home/omer/projects/badslam/applications/badslam
14:45:06.990              loguru.cpp:592   INFO| stderr verbosity: 2
14:45:06.990              loguru.cpp:593   INFO| -----------------------------------
14:45:07.049 rgbd_video_io_tum_datas:146    ERR| Cannot read calibration file at /home/omer/projects/cables_1/calibration.txt
14:45:07.049                 main.cc:515    ERR| Could not read dataset at /home/omer/projects/cables_1
14:45:07.056              loguru.cpp:467   INFO| atexit
Program call failed (return code: 1)

It says it cannot read calibration file and dataset. I am not sure if there is a problem in calibration file. I've also checked permissions of file, it's fine. Here is the content of calibration.txt

726.28741455078 726.28741455078 354.6496887207 186.46566772461

I can provide more logs if needed. Thanks in advance!

I'm exploring the Dataset Playback. It works fine with ETH cables_1 dataset. I've mimicked the structure of dataset with the recording I get from Azure Kinect but I get the following error. Since it works on other dataset I've a feeling that I made a mistake while constructing the dataset rather than having a CUDA issue. At the very bottom you can find more info about the dataset folder I use.

time                            file:line     v| 
12:25:06.568              loguru.cpp:587   INFO| arguments: ./applications/badslam/badslam
12:25:06.568              loguru.cpp:590   INFO| Current dir: /home/omer/projects/badslam/build_RelWithDebInfo
12:25:06.568              loguru.cpp:592   INFO| stderr verbosity: 2
12:25:06.568              loguru.cpp:593   INFO| -----------------------------------
Gtk-Message: 12:25:08.444: GtkDialog mapped without a transient parent. This is discouraged.
12:25:14.757      gui_main_window.cc:1600  INFO| Read dataset with 601 frames
12:25:15.605        loop_detector.cc:170   INFO| Loop detector: Loading vocabulary (from ./applications/badslam/resources/brief_k10L6.voc) ...
Stack trace:
7       0x7f0be9de188f clone + 63
6       0x7f0bec4fe6db /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7f0bec4fe6db]
5       0x7f0bea72466f /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xbd66f) [0x7f0bea72466f]
4       0x5649af370f19 ./applications/badslam/badslam(+0x75f19) [0x5649af370f19]
3       0x5649af33c8db ./applications/badslam/badslam(+0x418db) [0x5649af33c8db]
2       0x5649af33c164 ./applications/badslam/badslam(+0x41164) [0x5649af33c164]
1       0x5649af33e696 ./applications/badslam/badslam(+0x43696) [0x5649af33e696]
0       0x7f0bec72cf52 loguru::StreamLogger::~StreamLogger() + 162
12:25:20.197       cuda_buffer_inl.h:85    FATL| Cuda Error: invalid pitch argument
Aborted (core dumped)

The structure of dataset folder

.
├── associated.txt
├── calibration.txt
├── depth
            ├── color0001.png
             ...
|── rgb
            ├── color0001.png
             ...

Content of associated.txt

0001 rgb/color0001.png 0001 depth/depth0001.png
0002 rgb/color0002.png 0002 depth/depth0002.png
...

I've filled calibration.txt with the values printed in here, Line 236-239.

600.9438476562 600.7403564453 639.6210937500 366.5878906250

Thank you very much. I was suspecting the compute capability as well but I thought I had a 5.3. I will try doing that.

Hello guys, good job! I faced the following problem:

When I launch ./applications/badslam/badslam_test, I get this message:

[==========] Running 14 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 14 tests from Optimization
[ RUN ] Optimization.AlternatingGeometryOptimizationWithGeometricResidual
Stack trace:
15 0x7f98ad9fa41d clone + 109
14 0x7f98ae9716ba /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba) [0x7f98ae9716ba]
13 0x7f98ae294c80 /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb8c80) [0x7f98ae294c80]
12 0x7f98aff42dae /home/me/Documents/bosch_projects/BAD_SLAM/BAD_SLAM_full/build/liblibvis.so(+0x86dae) [0x7f98aff42dae]
11 0x428e8f ./applications/badslam/badslam_test() [0x428e8f]
10 0x7f98b5bdafa0 testing::UnitTest::Run() + 64
9 0x7f98b5be2b43 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::)(), char const) + 67
8 0x7f98b5bdac7f testing::internal::UnitTestImpl::RunAllTests() + 575
7 0x7f98b5bda9b5 testing::TestCase::Run() + 181
6 0x7f98b5bda8d8 testing::TestInfo::Run() + 280
5 0x7f98b5bda78a testing::Test::Run() + 186
4 0x7f98b5be2663 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::)(), char const) + 67
3 0x4297b2 ./applications/badslam/badslam_test() [0x4297b2]
2 0x5650c5 ./applications/badslam/badslam_test() [0x5650c5]
1 0x44e9e1 ./applications/badslam/badslam_test() [0x44e9e1]
0 0x7f98afc9ed73 loguru::StreamLogger::~StreamLogger() + 163
11:15:59.004 cuda_image_processing.c:192 FATL| Cuda Error: invalid device function
Aborted (core dumped)

It is probably related to the fact that I use an Nvidia GeForce 950M graphic card with 5.0 compute capability. I also use CUDA 8.0. What can be done to get through this issue?

Thanks for this great project! I have no problem running with the eth3D dataset and get results. However, when I was trying another dataset (Stanford copyroom), used by Open3D, I am having some troubles to get the results.

Here is the structure of my files:

\copyroom
---\rgb
------000001.png
        ....
---\depth
------000001.png
        ....
---associated.txt
---calibration.txt

The format of depth image is 16 bit unsigned char, and the format of the color image is 8 bit unsigned char 3-channel. Since the unit of depth is in millimeter, I used the raw to metric depth scale of 0.001. The timestamp for the images are simply ordered as 1, 2, 3, 4...
I have tried disabling and enabling the motion model choice, but none of the options work.

Here is a screen shot of the reconstruction interface. Would appreciated any help or advice on what configuration I should use instead. Thanks!

After testing with some other dataset I found that the system shows that

u16 is not supported by the Qt image IO.

Is uint16 not the right format for Qt?

It would be good to set up a continuous integration service which verifies that the code compiles both on Linux and on Windows.

I notices the paper has partitioned the benchmark datasets into 3 categories (easy, medium, hard), but where can I find which dataset belongs to which category?

Currently pre-built binary for Windows do not include Azure Kinect option. I would really appreciate if you can release a version which includes Azure Kinect input option since I'm having problem with compiling currently.

Currently, a keyframe is simply created for every Nth frame. It would be nice to be able to choose from multiple keyframe selection strategies. For example, a very simple addition to the current scheme would be to not select a keyframe in case the current camera pose is very similar to the latest created keyframe. This would prevent constantly creating new keyframes when the camera pose stays nearly constant.

Right now, keyframes take up a lot of GPU memory, which limits the size of the reconstructions that can be made. It might be helpful to introduce the possibility to store keyframe data on the CPU while it is not used to be able to make larger reconstructions. This would likely also make the process very slow as soon as this is used, however, due to the frequent memory transfers between the CPU and GPU which will be needed then.

It would be great if example auto_tuning_result.txt will be included in the repository.
For now, it is hard to guess what to do if badslam crashes with Cuda Error: out of memory under the auto_tune_parameters.py

Many thanks!

hi, when i execute "make -j badslam", why it said "make: *** No rule to make target 'badslam'. Stop. "

when I test with large_loop1 data in your benchmark dataset, the pose tracking part not converge and output wrong pose, it may need more robust front end to tracking the pose

Currently, the program always does global bundle adjustment, which means that it becomes slow after some time. It would be good to be able to do local bundle adjustment, affecting only the keyframes and surfels close to the current camera pose, such that it can remain fast.

There is already some code to support disabling surfels and keyframes, but it has not been used for a while, so it might not work properly.

Hi,
I am new to CUDA programming and learning this open-source project.
Recently I was reading the source code of this project. I have a question about it. I will appreciate it if you can read this issue and answer my question.

In libvis/cuda/, the basic data structures are defined in the files under this directory.

For /applications/badslam/src/badslam/cuda_depth_processing.cu, the function ComputeNormalsCUDA has a argument CUDABuffer_<T>* out_normals. I can understand this step: the pointer of out_normals is passed into this function.

When this function launches the kernel function ComputeNormalsCUDAKernel(), it pass *out_normals as an object into the kernel function. However, the kernel function receives an object as input in its argument list, instead of a reference. My question is: how could the modifications on *out_normals in the kernel function affect the out_normals?

In other words, in the kernel function, it computes the normals and modifies the corresponding element of out_normals by using out_normals(y, x) = xxx, but it is not the reference of the object. It is a copy of the object.

Is it because when the object out_normals is passed into the kernel function, the copy of *out_normals copied its data head address, instead of allocating new memory space in GPU? So that when the copy of *out_normals use the operator out_normals(y, x), it modified the corresponding data with respect to the original data head pointer address?

Thank you very much for your time to read this question!