krrish94 / icra2017

Code release for the ICRA 2017 paper "Reconstructing vehicles from a single image: shape priors for road scene understanding"

✨ Hi there 👋

My work is best accessed at https://krrish94.github.io

• 🔭 I work on [robotics / computer vision / graphics] + deep learning
• 🌱 I’m currently learning tons of cool stuff in all the above areas (and beyond).
• 👯 I’m frequently looking for collaborators on my projects. If that's you, let's get in touch!
• 💬 Ask me about research or grad school advice (and anything else -- although I might only be able to reply meaningfully if the ask is specific, and within my areas of expertise).
• 📫 How to reach me: Email me (and vey rarely, DM me on Twitter).
• 😄 Pronouns: he / him
• ⚡ Fun fact: You can get one such summary too -- just create a GitHub repo with the name exactly matching your username, and anything you write in the repo's README will show up here.

icra2017's Issues

Missing keypoints

Hi, when we could not detect all the 36 keypoints, say keypoint no. 1 is not detected when we are supposed to see it according to the azimuth, should we force the visibility to 0? It does not seem to make a difference by changing the visibility, the output shape will be wrong. It seems that the shape optimization will always use some fixed set of keypoints. Is there a way to work around the missing detections?

Maybe the only way is that we should infer the position of keypoint no. 1 based on the keypoints we have detected? Then in that case we have to always generate 36 keypoint positions?

More questions about localization and tracking

Hi, thank you so much for the code release! They are quite helpful!

As shown in the example image, we can not only see the 2D prior shape but also we can convert it to 3D and know its distance in the monocular scene. In addition, we can know the id of the observed car. I think ground plane estimation and some tracking algorithm is used here, could you please tell us more about it?

Thank you so much!

Do I need intrinsics matrix if I want to run in my own image?

It seems that i must have camera intrinsics if i want to run in my own image?
If I just want to draw 2D wireframe, it seeems camera intrinsics is not necessary?

Training on custom data

Is there any plan to release dataset, training code for the keypoint network (modified stacked hourglass)? The keypoint network currently is written in torch, which is deprecated. Are there plans to rewrite this in another framework, say pytorch? Asking because, if not, I could possibly look into it.

Secondly, is there any plan to update the repo on how to customize for objects such as chair, laptop etc., which was mentioned in the write up?

OSError: cache/poseAdjusterOutput.txt not found.

When I run "demo.py" I get the following error, do you know what is the cause and how should I fix it? Thank you very much for your help and contribution!

F0522 02:57:01.814723 28287 rotation.h:591] Check failed: pt != result (0x7ffe6a9637f0 vs. 0x7ffe6a9637f0) Inplace rotation is not supported.
*** Check failure stack trace: ***
@ 0x7fee72d480cd google::LogMessage::Fail()
@ 0x7fee72d49f33 google::LogMessage::SendToLog()
@ 0x7fee72d47c28 google::LogMessage::Flush()
@ 0x7fee72d4a999 google::LogMessageFatal::~LogMessageFatal()
@ 0x556a855040c9 (unknown)
@ 0x556a85502dad (unknown)
@ 0x556a85501d78 (unknown)
@ 0x556a85500df4 (unknown)
@ 0x556a85500092 (unknown)
@ 0x556a854ff5c1 (unknown)
@ 0x556a854feb89 (unknown)
@ 0x556a855214c6 (unknown)
@ 0x556a855473f6 (unknown)
@ 0x556a85535119 (unknown)
@ 0x556a85548f67 (unknown)
@ 0x556a8555b0ed (unknown)
@ 0x556a8555bb82 (unknown)
@ 0x556a85560604 (unknown)
@ 0x556a8552f0e6 (unknown)
@ 0x556a8552fc99 (unknown)
@ 0x556a854f98df (unknown)
@ 0x7fee70e2cbf7 __libc_start_main
@ 0x556a854f907a (unknown)
Traceback (most recent call last):
File "demo.py", line 279, in
predictedPose = np.loadtxt(poseAdjusterOutput)
File "/usr/local/lib/python3.6/dist-packages/numpy/lib/npyio.py", line 968, in loadtxt
fh = np.lib._datasource.open(fname, 'rt', encoding=encoding)
File "/usr/local/lib/python3.6/dist-packages/numpy/lib/_datasource.py", line 269, in open
return ds.open(path, mode, encoding=encoding, newline=newline)
File "/usr/local/lib/python3.6/dist-packages/numpy/lib/_datasource.py", line 623, in open
raise IOError("%s not found." % path)
OSError: cache/poseAdjusterOutput.txt not found.