xiandaguo / openstereo

Hi there 👋

I'm Xianda Guo, a PhD student at the School of Computer Science, Wuhan University.

Previous Efforts

• Gait Recognition:GREW -> DyGait -> SPOSGait
• 🚙 Monocular Depth Estimation: MonoViT -> CompletionFormer -> DiffusionDepth
• 🚙 Stereo Matching: OpenStereo
• 🚙 Surround Depth Estimation: SSDepth
• 🚙 End-to-End Driving: MaskFuser -> GenAD

Current Interests

Gait Recognition, 🚙 Autonomous Driving

• Gait Recognition
• 🚙 Depth Estimation
• 🚙 Stereo Matching
• 🚙 End-to-End Driving...

Collaborations

If you want to work with me (in person or remotely), feel free to drop me an email at [email protected]. I could support GPUs if we are a good fit.

openstereo's Issues

Evaluation on ETH3D, MB and KITTI

Hi, nice work!

I failed to find config files on ETH3D, MB, and KITTI. Could you please provide some commands on how to test on these datasets with the models trained on SceneFlow (if change data_cfg.name=ETH3D, then it cannot reload the checkpoint from SceneFlow)?

Middlebury dataset download issue

Middlebury reference data sets the configuration of the tutorial to download the data sets, but the links in the tutorial (http://vision.middlebury.edu/stereo/data) in the corresponding page has multiple data sets: 2001, 2003, 2005, 2006, 2014, 2021. I downloaded them all, but when I unzipped them, there were no TrainingQ folders. So can you tell me the detailed download path?

请问你这个模型针对的IGEV还是所有的呢？所有的立体匹配模型裁剪成320*736效果都比256*512要好吗

Is this article a review and finishing work of stereo matching？

Hi, this work is amazing! Thank you for your work. I want to ask two questions.
1.Is this article a review and finishing work of stereo matching? I don’t seem to see a network model named openstereo.
2.the data augumentation method ablation experiment seems not obviously. I can not select best combination of data augumentation method.
Wish your early reply.

Mistake in paper table7

The results in the table do not seem to be on the same test set.

In STTR, it's evaluated on the flyingthings subset of sceneflow.
In IGEV, it's evaluated on the whole sceneflow

BTW, what is the set used in StereoBase?

There are some issues with the configuration and reproduction of Raft-stereo

Nice work!
However, I found that there is an issue with the data preprocessing section in the configuration file of the raft-stereo, which cannot run smoothly. After aligning with the data preprocessing section of the original raft-stereo, it can run normally.
However, there is a significant difference between EPE and the original version, and the convergence is also slow.
Could you please review the configuration and training process of the following raft-stereo again?

This configuration can run through the code. However, whether or not data regularization is turned off, the effect is not ideal.

关于复现StereoBase

麻烦问一下，StereoBase在OpenStereo中没有开源实现吗？

Inference on single image

Hello,

Thank you for this amazing repository, I wanted to know is there any way to perform inference on a single image after training the models?
I unable to find anywhere it saves images after validation or testing.

Plan about model deployment

Is there any plan to add related implementation of model deployment?

Some model checkpoints are failed to be downloaded

Hi! Thank you for the amazing work!

I've noticed that certain checkpoints on Baidu Pan, such as FADNet_sceneflow.pt and RAFT, failed to download properly.
The problem seems to be that after downloading and extracting the checkpoints, the folder appears to be empty.
Would you mind confirming this issue? Thanks!

What does it mean to show your submission ID in arxiv print?

Configuration file for the DrivingStereo dataset.

Does OpenStereo support DrivingStereo datasets? The ".yaml" configuration file for the DrivingStereo dataset could not be found.

KITTI benchmark sota is based on which model?

KITTI benchmark sota is based on which model? Is there open-source pre-trained parameters?

The training strategies

The paper mentions that “each model was trained 90 epochs using a batch size of 64”, is this the same in the final training of Stereobase？What are the training strategies on the sceneflow and KITTI datasets respectively?

FADNet tensor size mismatched

Hi!

I attempted to validate the ETH3D dataset on FADNet using the pretrained weight: FADNet_sceneflow.pt. However, I encountered a dimension mismatch issue due to the image size of ETH3D being (513, 888). The problem arises specifically in the line concat5 = torch.cat((upconv5, upflow6, conv5b), 1), where the sizes of the three elements are as follows:

upconv5: torch.Size([1, 512, 18, 28])
upflow6: torch.Size([1, 1, 18, 28])
conv5b: torch.Size([1, 512, 17, 28])

Is additional padding or resizing required here or do you have any suggestions on how to address this, for those who would like to inference any other dataset instead of SceneFlow (like ETH3D, Kitti2012, etc). Thanks!

Which pretrained model exactly is stereo base?

In you paper I see some thing like this:

but which one in the model zoo exactly is this one?

Have you tried converting the PyTorch model via ONNX to a TensorRT engine for faster inference?

How to configure the "StereoBase" baseline model net work in your paper?

I can not find any definition of the "StereoBase" network arch, could you give me some advice to try it out?

sttr, epe is tool large

hi, i have downloaded your pretrained model, in a test of sttr model, there is a error when load the checkpoint file, i ignore this error, at last, the epe is 49(px).

感谢，请问你们文章中复现的网络是FADNet还是FADNet++呢？因为注意到FADNet++好像没有完全开源。