Evaluation about sfa3d

[iliiliiliili]

@deepmeng, could you share evaluation results here? As far as I know, there are no available KITTI evaluation results for this method. Thanks

[maudzung]

Hi @deepmeng

Yeb, this work has a better performance compared to the YOLO-based methods in both accuracy and speed.
Please contribute the full evaluation tool if you make it. Thanks.

[bilgecanpullinen]

Hi @deepmeng and @maudzung
I couldn't figure out how I can evaluate (write the prediction results as txt files). Could you share your evaluation file.
Thanks

[deepmeng]

@bilgecanpullinen OK. The code in my hand is chaotic. I will fine-tune and share the code.

[deepmeng]

Hi @iliiliiliili and @bilgecanpullinen . I forked this repo and I added the evaluation tools. I didn't merge my code to this repo because my code is not concise. You can refer to this repo (https://github.com/reinforcementdriving/Super-Fast-Accurate-3D-Object-Detection), extract and adapt the related code to evaluate the prediction results.

[bilgecanpullinen]

Thank you I have tried but now I have some problem with the dataloader.; eval.py line 112 val_dataloader, val_dataset = create_val_dataloader(configs) gives me a value error, which is too many values to unpack(expected 2). But you have used at your train.py the same way for create_train_dataloader. So do you have any idea how I can manage this one. deepmeng ***@***.***>, 30 Mar 2021 Sal, 05:51 tarihinde şunu yazdı:

…

Hi @iliiliiliili <https://github.com/iliiliiliili> and @bilgecanpullinen <https://github.com/bilgecanpullinen> . I forked this repo and I added the evaluation tools. I didn't merge my code to this repo because my code is not concise. You can refer to this repo ( https://github.com/reinforcementdriving/Super-Fast-Accurate-3D-Object-Detection), extract and adapt the related code to evaluate the prediction results. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIUM3GN4XDV5TBCYOF2OFDTTGE4C5ANCNFSM4TOLZ7LQ> .

[iliiliiliili]

Hi @iliiliiliili and @bilgecanpullinen . I forked this repo and I added the evaluation tools. I didn't merge my code to this repo because my code is not concise. You can refer to this repo (https://github.com/reinforcementdriving/Super-Fast-Accurate-3D-Object-Detection), extract and adapt the related code to evaluate the prediction results.

Thank you, @deepmeng. Could you add evaluation results on KITTI validation subset with your code to README? It would be very helpful 🙂

[deepmeng]

@bilgecanpullinen
Please try to modify the function 'create_val_dataloader(configs)' at line 54 in 'sfa/data_process/kitti_dataloader.py':
'return val_dataloader, val_dataset'.

Thank you I have tried but now I have some problem with the dataloader.; eval.py line 112 val_dataloader, val_dataset = create_val_dataloader(configs) gives me a value error, which is too many values to unpack(expected 2). But you have used at your train.py the same way for create_train_dataloader. So do you have any idea how I can manage this one. deepmeng @.***>, 30 Mar 2021 Sal, 05:51 tarihinde şunu yazdı:
…
Hi @iliiliiliili https://github.com/iliiliiliili and @bilgecanpullinen https://github.com/bilgecanpullinen . I forked this repo and I added the evaluation tools. I didn't merge my code to this repo because my code is not concise. You can refer to this repo ( https://github.com/reinforcementdriving/Super-Fast-Accurate-3D-Object-Detection), extract and adapt the related code to evaluate the prediction results. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUM3GN4XDV5TBCYOF2OFDTTGE4C5ANCNFSM4TOLZ7LQ .

[bilgecanpullinen]

Thank you again. Everything worked good until the kitti-eval now. Sorry for asking again to you but it seems you are using the same kitti-eval-python tool. I am receiving ERROR: The function received no value for the required argument: label_split_file Usage: evaluate.py evaluate LABEL_PATH RESULT_PATH LABEL_SPLIT_FILE <flags> optional flags: --current_class | --coco | --score_thresh For detailed information on this command, run: evaluate.py evaluate --help which is from the label_split_file and I am sure that my path is correct, I tried with from https://xiaozhichen.github.io/files/mv3d/imagesets.tar.gz file as well. Do you have any suggestions. deepmeng ***@***.***>, 30 Mar 2021 Sal, 12:10 tarihinde şunu yazdı:

…

@bilgecanpullinen <https://github.com/bilgecanpullinen> Please try to modify the function 'create_val_dataloader(configs)' at line 54 in 'sfa/data_process/kitti_dataloader.py': 'return val_dataloader, val_dataset'. Thank you I have tried but now I have some problem with the dataloader.; eval.py line 112 val_dataloader, val_dataset = create_val_dataloader(configs) gives me a value error, which is too many values to unpack(expected 2). But you have used at your train.py the same way for create_train_dataloader. So do you have any idea how I can manage this one. deepmeng *@*.***>, 30 Mar 2021 Sal, 05:51 tarihinde şunu yazdı: … <#m_2157938398064785683_> Hi @iliiliiliili <https://github.com/iliiliiliili> https://github.com/iliiliiliili and @bilgecanpullinen <https://github.com/bilgecanpullinen> https://github.com/bilgecanpullinen . I forked this repo and I added the evaluation tools. I didn't merge my code to this repo because my code is not concise. You can refer to this repo ( https://github.com/reinforcementdriving/Super-Fast-Accurate-3D-Object-Detection), extract and adapt the related code to evaluate the prediction results. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 (comment) <#16 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUM3GN4XDV5TBCYOF2OFDTTGE4C5ANCNFSM4TOLZ7LQ . — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIUM3GJUC6FNGKAY7U63JN3TGGIRJANCNFSM4TOLZ7LQ> .

[deepmeng]

Thank you again. Everything worked good until the kitti-eval now. Sorry for asking again to you but it seems you are using the same kitti-eval-python tool. I am receiving ERROR: The function received no value for the required argument: label_split_file Usage: evaluate.py evaluate LABEL_PATH RESULT_PATH LABEL_SPLIT_FILE optional flags: --current_class | --coco | --score_thresh For detailed information on this command, run: evaluate.py evaluate --help which is from the label_split_file and I am sure that my path is correct, I tried with from https://xiaozhichen.github.io/files/mv3d/imagesets.tar.gz file as well. Do you have any suggestions. deepmeng @.>, 30 Mar 2021 Sal, 12:10 tarihinde şunu yazdı:
…
@bilgecanpullinen https://github.com/bilgecanpullinen Please try to modify the function 'create_val_dataloader(configs)' at line 54 in 'sfa/data_process/kitti_dataloader.py': 'return val_dataloader, val_dataset'. Thank you I have tried but now I have some problem with the dataloader.; eval.py line 112 val_dataloader, val_dataset = create_val_dataloader(configs) gives me a value error, which is too many values to unpack(expected 2). But you have used at your train.py the same way for create_train_dataloader. So do you have any idea how I can manage this one. deepmeng @.>, 30 Mar 2021 Sal, 05:51 tarihinde şunu yazdı: … <#m_2157938398064785683_> Hi @iliiliiliili https://github.com/iliiliiliili https://github.com/iliiliiliili and @bilgecanpullinen https://github.com/bilgecanpullinen https://github.com/bilgecanpullinen . I forked this repo and I added the evaluation tools. I didn't merge my code to this repo because my code is not concise. You can refer to this repo ( https://github.com/reinforcementdriving/Super-Fast-Accurate-3D-Object-Detection), extract and adapt the related code to evaluate the prediction results. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 (comment) <#16 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUM3GN4XDV5TBCYOF2OFDTTGE4C5ANCNFSM4TOLZ7LQ . — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUM3GJUC6FNGKAY7U63JN3TGGIRJANCNFSM4TOLZ7LQ .

I used this tool (https://github.com/prclibo/kitti_eval). If you can't use the kitti eval tool properly, I have no idea.

[bilgecanpullinen]

Thank you again. I am having another problem and sorry for asking it again but I have tried many things. g++ -o evaluate_object_3d_offline evaluate_object_3d_offline.cpp -O3 evaluate_object_3d_offline.cpp:12:10: fatal error: boost/numeric/ublas/matrix.hpp: No such file or directory #include <boost/numeric/ublas/matrix.hpp> ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. This is my error while I try to compile. Do you remember having an issue like this. deepmeng ***@***.***>, 30 Mar 2021 Sal, 15:07 tarihinde şunu yazdı:

…

Thank you again. Everything worked good until the kitti-eval now. Sorry for asking again to you but it seems you are using the same kitti-eval-python tool. I am receiving ERROR: The function received no value for the required argument: label_split_file Usage: evaluate.py evaluate LABEL_PATH RESULT_PATH LABEL_SPLIT_FILE optional flags: --current_class | --coco | --score_thresh For detailed information on this command, run: evaluate.py evaluate --help which is from the label_split_file and I am sure that my path is correct, I tried with from https://xiaozhichen.github.io/files/mv3d/imagesets.tar.gz file as well. Do you have any suggestions. deepmeng *@*. *>, 30 Mar 2021 Sal, 12:10 tarihinde şunu yazdı: … <#m_6312827631054034898_> @bilgecanpullinen <https://github.com/bilgecanpullinen> https://github.com/bilgecanpullinen <https://github.com/bilgecanpullinen> Please try to modify the function 'create_val_dataloader(configs)' at line 54 in 'sfa/data_process/kitti_dataloader.py': 'return val_dataloader, val_dataset'. Thank you I have tried but now I have some problem with the dataloader.; eval.py line 112 val_dataloader, val_dataset = create_val_dataloader(configs) gives me a value error, which is too many values to unpack(expected 2). But you have used at your train.py the same way for create_train_dataloader. So do you have any idea how I can manage this one. deepmeng @.*>, 30 Mar 2021 Sal, 05:51 tarihinde şunu yazdı: … <#m_2157938398064785683_> Hi @iliiliiliili <https://github.com/iliiliiliili> https://github.com/iliiliiliili https://github.com/iliiliiliili and @bilgecanpullinen <https://github.com/bilgecanpullinen> https://github.com/bilgecanpullinen https://github.com/bilgecanpullinen . I forked this repo and I added the evaluation tools. I didn't merge my code to this repo because my code is not concise. You can refer to this repo ( https://github.com/reinforcementdriving/Super-Fast-Accurate-3D-Object-Detection), extract and adapt the related code to evaluate the prediction results. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 <#16> (comment) <#16 (comment) <#16 (comment)>>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUM3GN4XDV5TBCYOF2OFDTTGE4C5ANCNFSM4TOLZ7LQ . — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 (comment) <#16 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUM3GJUC6FNGKAY7U63JN3TGGIRJANCNFSM4TOLZ7LQ . I used this tool (https://github.com/prclibo/kitti_eval). If you can't use the kitti eval tool properly, I have no idea. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIUM3GJEVPQ3FCGGZ36P453TGG5H3ANCNFSM4TOLZ7LQ> .

[deepmeng]

Thank you again. I am having another problem and sorry for asking it again but I have tried many things. g++ -o evaluate_object_3d_offline evaluate_object_3d_offline.cpp -O3 evaluate_object_3d_offline.cpp:12:10: fatal error: boost/numeric/ublas/matrix.hpp: No such file or directory #include <boost/numeric/ublas/matrix.hpp> ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. This is my error while I try to compile. Do you remember having an issue like this. deepmeng @.***>, 30 Mar 2021 Sal, 15:07 tarihinde şunu yazdı:
…
Thank you again. Everything worked good until the kitti-eval now. Sorry for asking again to you but it seems you are using the same kitti-eval-python tool. I am receiving ERROR: The function received no value for the required argument: label_split_file Usage: evaluate.py evaluate LABEL_PATH RESULT_PATH LABEL_SPLIT_FILE optional flags: --current_class | --coco | --score_thresh For detailed information on this command, run: evaluate.py evaluate --help which is from the label_split_file and I am sure that my path is correct, I tried with from https://xiaozhichen.github.io/files/mv3d/imagesets.tar.gz file as well. Do you have any suggestions. deepmeng @. >, 30 Mar 2021 Sal, 12:10 tarihinde şunu yazdı: … <#m_6312827631054034898_> @bilgecanpullinen https://github.com/bilgecanpullinen https://github.com/bilgecanpullinen https://github.com/bilgecanpullinen Please try to modify the function 'create_val_dataloader(configs)' at line 54 in 'sfa/data_process/kitti_dataloader.py': 'return val_dataloader, val_dataset'. Thank you I have tried but now I have some problem with the dataloader.; eval.py line 112 val_dataloader, val_dataset = create_val_dataloader(configs) gives me a value error, which is too many values to unpack(expected 2). But you have used at your train.py the same way for create_train_dataloader. So do you have any idea how I can manage this one. deepmeng @.>, 30 Mar 2021 Sal, 05:51 tarihinde şunu yazdı: … <#m_2157938398064785683_> Hi @iliiliiliili https://github.com/iliiliiliili https://github.com/iliiliiliili https://github.com/iliiliiliili and @bilgecanpullinen https://github.com/bilgecanpullinen https://github.com/bilgecanpullinen https://github.com/bilgecanpullinen . I forked this repo and I added the evaluation tools. I didn't merge my code to this repo because my code is not concise. You can refer to this repo ( https://github.com/reinforcementdriving/Super-Fast-Accurate-3D-Object-Detection), extract and adapt the related code to evaluate the prediction results. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 <#16> (comment) <#16 (comment) <#16 (comment)>>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUM3GN4XDV5TBCYOF2OFDTTGE4C5ANCNFSM4TOLZ7LQ . — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 (comment) <#16 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUM3GJUC6FNGKAY7U63JN3TGGIRJANCNFSM4TOLZ7LQ . I used this tool (https://github.com/prclibo/kitti_eval). If you can't use the kitti eval tool properly, I have no idea. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUM3GJEVPQ3FCGGZ36P453TGG5H3ANCNFSM4TOLZ7LQ .

Well. I'm not sure your experimental environment. You must make sure you have installed the Boost library. Maybe you can try 'sudo apt-get install libboost-all-dev'.

[bilgecanpullinen]

Okay, I don't have the sudo right for that environment but I will manage it from here. Thank you :) deepmeng ***@***.***>, 30 Mar 2021 Sal, 16:09 tarihinde şunu yazdı:

…

Thank you again. I am having another problem and sorry for asking it again but I have tried many things. g++ -o evaluate_object_3d_offline evaluate_object_3d_offline.cpp -O3 evaluate_object_3d_offline.cpp:12:10: fatal error: boost/numeric/ublas/matrix.hpp: No such file or directory #include <boost/numeric/ublas/matrix.hpp> ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. This is my error while I try to compile. Do you remember having an issue like this. deepmeng *@*.***>, 30 Mar 2021 Sal, 15:07 tarihinde şunu yazdı: … <#m_-3983213664647106577_> Thank you again. Everything worked good until the kitti-eval now. Sorry for asking again to you but it seems you are using the same kitti-eval-python tool. I am receiving ERROR: The function received no value for the required argument: label_split_file Usage: evaluate.py evaluate LABEL_PATH RESULT_PATH LABEL_SPLIT_FILE optional flags: --current_class | --coco | --score_thresh For detailed information on this command, run: evaluate.py evaluate --help which is from the label_split_file and I am sure that my path is correct, I tried with from https://xiaozhichen.github.io/files/mv3d/imagesets.tar.gz file as well. Do you have any suggestions. deepmeng *@*. *>, 30 Mar 2021 Sal, 12:10 tarihinde şunu yazdı: … <#m_6312827631054034898_> @bilgecanpullinen <https://github.com/bilgecanpullinen> https://github.com/bilgecanpullinen <https://github.com/bilgecanpullinen> https://github.com/bilgecanpullinen <https://github.com/bilgecanpullinen> https://github.com/bilgecanpullinen <https://github.com/bilgecanpullinen> Please try to modify the function 'create_val_dataloader(configs)' at line 54 in 'sfa/data_process/kitti_dataloader.py': 'return val_dataloader, val_dataset'. Thank you I have tried but now I have some problem with the dataloader.; eval.py line 112 val_dataloader, val_dataset = create_val_dataloader(configs) gives me a value error, which is too many values to unpack(expected 2). But you have used at your train.py the same way for create_train_dataloader. So do you have any idea how I can manage this one. deepmeng @.*>, 30 Mar 2021 Sal, 05:51 tarihinde şunu yazdı: … <#m_2157938398064785683_> Hi @iliiliiliili <https://github.com/iliiliiliili> https://github.com/iliiliiliili https://github.com/iliiliiliili https://github.com/iliiliiliili and @bilgecanpullinen <https://github.com/bilgecanpullinen> https://github.com/bilgecanpullinen https://github.com/bilgecanpullinen https://github.com/bilgecanpullinen . I forked this repo and I added the evaluation tools. I didn't merge my code to this repo because my code is not concise. You can refer to this repo ( https://github.com/reinforcementdriving/Super-Fast-Accurate-3D-Object-Detection), extract and adapt the related code to evaluate the prediction results. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 <#16> <#16 <#16>> (comment) <#16 <#16> (comment) <#16 (comment) <#16 (comment)>>>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUM3GN4XDV5TBCYOF2OFDTTGE4C5ANCNFSM4TOLZ7LQ . — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 <#16> (comment) <#16 (comment) <#16 (comment)>>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUM3GJUC6FNGKAY7U63JN3TGGIRJANCNFSM4TOLZ7LQ . I used this tool (https://github.com/prclibo/kitti_eval). If you can't use the kitti eval tool properly, I have no idea. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 (comment) <#16 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIUM3GJEVPQ3FCGGZ36P453TGG5H3ANCNFSM4TOLZ7LQ . Well. I'm not sure your experimental environment. You must make sure you have installed the Boost library. Maybe you can try 'sudo apt-get install libboost-all-dev'. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIUM3GP7ACZNWV4GCXGGQBLTGHEQPANCNFSM4TOLZ7LQ> .