meteorshowers / rcf-pytorch

Richer Convolutional Features for Edge Detection model in pytorch CVPR2017

✨ meteorshowers ✨,　                

• 🔭 I am working on NN-Planning in XPENG Motors for the next generation Autonomous Driving Platform.
• 🔭 I used to work with Dr. Xiaozhi Chen on Autonomous Driving 3D perception in DJI (I lead the bevdet team from 0-1).
• 🌱 I used to work on multi-modal AICG (nlp-cv) work with Shenjian Zhao in ByteDance search-nlp group.

Blogs 1. HITNet 谷歌首个高精度实时端到端双目匹配网络结构 2. ActiveStereoNet：谷歌首个基于深度学习的结构光双目匹配系统 3. 推断速度提高几十倍，谷歌研究员提出实时端到端双目系统深度学习网络stereonet

rcf-pytorch's Issues

hello，Where is the file of vgg16convs.mat and caffe-fsds.mat?？？

@meteorshowers

About Different Optimizer

Hello, I have paid attention to this work for some time, and appreciate your wonderful work. I have some questions to insult you. And I will appreciate it if you can give some illustrations.

I notice that you also code for Adam optimizer when training. So I wonder if there are some differences between those two kinds of optimizers in performance? Is there anyone better than the other when using the same lr for different layers?

mean_bgr [104.00699, 116.66877, 122.67892]

您好，我看代码“im -= np.array((104.00698793,116.66876762,122.67891434))”，这个是图像RGB的平均值，能否问下，这个平均值好像是Pascal VOC2012的吗？如果对一个新的数据集做边缘检测，是否需要进行同样的统计呢？期待你的回复。谢谢。

Some question about batch_size

thank you for your code. I found the batch_size = 1 in most edge detection code, also in RCF-pytorch, so is it must to set the batch_size = 1, not batch_size > 1 ?

How to extract PASCAL and HED-BSDS and evaluate.py?

I created a data folder and placed the downloaded PASCAL and HED-BSDS tar.gz files in it. Then, I extracted them into folders PASCAL and HED-BSDS. I ran the train_RCF.py and then got errors about missing files, so I then created a new folder named HED-BSDS_PASCAL and placed the HED-BSDS and PASCAL folders in it

After running the training again, it says no such file or directory data/HED-BSDS_PASCAL/HED-BSDS/train/aug_gt_scale_0.5/90.0_1_0/143090.png

I see there is 143090.jpg in there instead. Do I have to change all the png to jpg in bsds_pascal_train_pair.lst?

Also, where is evaluate.py?

请问，您那个视频怎么做的？

请问，您那个视频是怎么制作的？是一帧一帧的么

请问这个程序能否不训练直接进行测试？

python train.py 是训练加评价，如果只测试的话请问如何操作？

Where is the file of test.lst ??

vgg16convs.mat ?

hello, I want to know what vgg16convs.mat in your data_loader.py is. Generally, we use a pretrained model, that is a vgg16.pth file. Look forward to your response. Thanks.

请问可以把那个86MB的gif图片改掉吗？

不知道怎么想的，放那么大一张图片，打开页面要缓冲半天。几秒钟的不就行了？

收敛时候的loos值

请问算法收敛了之后，avg的loss大概是多少～。

wrong kernel size bilinear interpolation on block 5

hi @meteorshowers

Just want to inform you that there is small mistake on your implementation.
File : models.py line : 141

weight_deconv5 = make_bilinear_weights(32, 1).cuda()
Based on the original code
https://github.com/yun-liu/rcf/blob/master/examples/rcf/train_val.prototxt
https://github.com/yun-liu/rcf/blob/master/examples/rcf/test.prototxt

it should be
weight_deconv5 = make_bilinear_weights(16, 1).cuda()

it doesn't matter a lot, but if someone want to use the original pretrained model, it will produce significant different results.

Thank you

Where is the file of vgg16convs.mat and caffe-fsds.mat?

请问，需要用matlab工具箱进行最极大抑制吗？

Note: Before evaluating the predicted edges, you should do the standard non-maximum suppression (NMS) and edge thinning. We used Piotr's Structured Forest matlab toolbox available here.
这个是https://github.com/yun-liu/rcf 中的note。

请教下 itersize 作用？

迭代10次，更新权重，请问itersize =10, batch size =1 与 itersize =1， batch size =10，有什么区别么？谢谢！

输出图片尺寸不一致

你好，经过你们的预训练模型，推理得到的图片尺寸和原始的不一致
外圈有一层黑边，请问怎么去掉，相关代码在哪里？

请问训练自己的数据集为什么LOSS不收敛，结果图都是全黑或者全白。

May I ask why the LOSS does not converge when training our own data set, the result graphs are all black or all white.

Could you give more information about `make_bilinear_weights`?

RCF-pytorch/models.py

Line 209 in 5efe804

def make_bilinear_weights(size, num_channels):

Nice work!

Function make_bilinear_weights makes me confused, could you give us more information or references about this function?

Thanks a lot!

关于数据加载，因为cv2和PIL读出来的图片size不一致

cv2读图片出来的shape是（H, W, C)， 而PIL把图片读出来的shape是(W, H, C），所以您在data_loader这一块，这两种方式的混用，会影响到最后的loss计算吧

女朋友看来管理很严格。。。。。

Failed to reproduce results

I failed to reproduce the results on the README using your training script.
The scores were

ods: 0.7926  (README says I should get 0.808)
ois: 0.8073   (README says I should get 0.811)

The trained weights can be found here https://drive.google.com/file/d/10N9ohhzJvHdbrYfVd_1zzJ-vp46XEiZN/view?usp=sharing.
The code I used is here https://github.com/yuyu2172/RCF-pytorch.

If possible, could you upload the weights of the trained model you used to produce the results on README?
Also, could you upload VGG16 pretrained weights?

IndexError: too many indices for array

你好，请问换成自己的test数据集后，为何一直提示 IndexError: too many indices for array呢？大神求助~

损失很大

我的损失很大，一般都在万以上，这样会不会造成梯度爆炸呢？
我的做法：

1 输入：下载了BSDBS500数据，使用原图和二值图标签训练，使用源代码中的方法，没有归一化输入，只是减每个通道均值
2 中间处理：模型超参数使用代码原始的参数，没有修改，batch 1 iter 10 lr 1-e6
3 输出：损失1w-2w，图相train文件夹里除了第一个有轮廓其他都是黑的或灰的，test文件夹下根本没有轮廓
希望有成功运行的人，能指点一下正确的处理过程，多谢！
1输入
2中间处理
3输出

Can't reproduce the report results

Hello, I used your codes to train the RCF, with no changes to the codes, the best ODS I got is 0.778 with single scale, which is far from the model your uploaded. Moreover, the visualization results still have many texture edges, which is also far beyond your visualized examples. I have tried pytorch1.2 and pytorch0.4.1, but still have the same problems. Have your ever met such problems in your reproduction?

vgg16convs.mat and caffe-fsds.mat?

Hello, I can't find these two files vgg16convs.mat and caffe-fsds.mat, which are needed in utils.py. Could you help me?

evaluate.py

你好，
感谢你做的贡献，但是我没有在文件夹里面找到evaluate.py，请问可以分享一下吗

Loss值

得到如图所示的loss，感觉这个loss降得好快啊，第一次跑训练不太懂是不是哪里搞错了，还是这个原本就是这样，感谢解答。

vgg16convs.mat

这个资源能给个链接？

evaluate.py?

hello,
i can't find the evaluate.py file to test the project, do you know where it is?

also, do you have a pretrained model somewhere?

thanks

Why my debug image output is all black?

Is it normal?
The last three output maps are always all black after many epoches.

lb[np.logical_and(lb>0, lb<128)] = 2

why lb[np.logical_and(lb>0, lb<128)] = 2? in data_loader.py { class BSDS_RCFLoader(data.Dataset):/def getitem(self, index):}

不好意思，我没有找到evaluate.py

我在训练了该模型，但是loss值很不稳定，训练完后没有找到预测文件，麻烦了，谢谢

数据集从哪里下载？

您好，请问这个数据集在哪里下载呢？能提供一个下载地址吗？

请问有训练好的模型吗？

请问训练好的模型什么时候可以放出来？

训练时候图片大小不固定问题的解决?

When I train, I find the loss is not stable, in one epoch, it can convergence; but when it comes to another epoch, the loss become very big.

Help

/home/dongbo/anaconda3/lib/python3.7/site-packages/torch/nn/_reduction.py:49: UserWarning: size_average and reduce args will be deprecated, please use reduction='none' instead.
warnings.warn(warning.format(ret))
Epoch: [0/30][0/49006] Time 3.022 (avg:3.022) Loss 3202.693115 (avg:3202.693115)
Traceback (most recent call last):
File "train_RCF.py", line 340, in
main()
File "train_RCF.py", line 212, in main
save_dir = join(TMP_DIR, 'epoch-%d-training-record' % epoch))
File "train_RCF.py", line 239, in train
for i, (image, label) in enumerate(train_loader):
File "/home/dongbo/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 637, in next
return self._process_next_batch(batch)
File "/home/dongbo/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 658, in _process_next_batch
raise batch.exc_type(batch.exc_msg)
ValueError: Traceback (most recent call last):
File "/home/dongbo/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 138, in _worker_loop
samples = collate_fn([dataset[i] for i in batch_indices])
File "/home/dongbo/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 138, in
samples = collate_fn([dataset[i] for i in batch_indices])
File "/home/dongbo/RCF-pytorch/data_loader.py", line 59, in getitem
img = prepare_image_cv2(img)
File "/home/dongbo/RCF-pytorch/data_loader.py", line 17, in prepare_image_cv2
im -= np.array((104.00698793,116.66876762,122.67891434))
ValueError: non-broadcastable output operand with shape () doesn't match the broadcast shape (3,)

About other version of the code

Hello，do you have writed the code to add rcf to resnet (pytorch)?

I have tried,but its not good.

迭代次数设置和加载模型

您好，最近对于RCF算法十分感兴趣，但是由于自身计算机能力较弱因此操作过程中遇到了很多困扰。请问训练模型时迭代次数在哪里能够控制；训练好的模型，应该输入到那个文件用来做边缘检测？

Meaning of "adam1e-4-tunelr"

Thanks for this repo. I am not clear about one thing in result part. What is meaning of "adam1e-4-turnlr" ? If we use adam as the optimizer, do we need to turn learning rate during traning ?

Thanks.

不能复现结果

您好，我用扩充过的BSDS500数据集训练，ODS最高到0.778，请问您训练的时候最大迭代次数和stepsize设置的是多少呢，我用BSDS500加PASCAL数据集训练的时候出来的是全灰的图，不知道是哪里有问题

How to predict single image?

Hi , how can I predict a single image?

where is the file of train_multi_gpu.py?

where is the file of train_multi_gpu.py?

函数cross_entropy_loss_RCF有疑问

对交叉熵两项加权，你用的是torch.nn.functional..binary_cross_entropy(input, target, weight=None, size_average=None, reduce=None, reduction='mean'),但是其中参数weight是对batch加权的，不是对交叉项加权。不知道我的理解是否正确。
参见pytorch文档
https://pytorch.org/docs/stable/nn.html?highlight=torch%20nn%20functional%20binary#torch.nn.BCELoss

adddilation-stridewith8

您好，我最近在学习您的RCF-pytorch 代码， 遇到了一点问题：adddilation-stridewith8 是什么意思？ 是要将涉及到dilation卷积的步长调整到8？ 比如， 在您的 models.py 文件中定义的类 class FlipConv2d(_ConvNd):
def init(self, in_channels, out_channels, kernel_size, stride=1,
padding=0, dilation=1, groups=1, bias=True):
是要将 stride=1 调整为 8。 不知我的理解是否正确？ 提前多谢您的指导哈，祝您有一个愉快的中秋。

评价工具

请问这个有评价工具吗？

Question about the RCF edge detection framework.

Thanks for your great work and kind sharing.

I successfully reproduce the edge detection result, as
2018.jpeg:

2018.png:

After NMS process, it turns like:

However, when it switch the normal convolution to some special convolution in the same backbone(VGG16), the result is kinda wacky:

After same NMS process of edge thinning, it turns like more coarser in detail when compared with result above :

Also, after evaluation, the ODS-F is 0.685891 and OIS-F is 0.703985, both are much less than the paper, 0.788864 and 0.806692.

Could any one please point out where is the potential problem lie in, cause I am really confused about it, and have no much experience in edge detection.

Thanks in advance !

training data format

hi!
i wanted to train something with your code and see results
BUT!
i totally don't understand what I need to do with data to run train_RCF.py since there is no instructions in readme
Can you type them, please?

ps as i searched other issues i had information that evaluate.py is still missing ;)

About the version of this code

Whether this repository code is the fast version of the RCF？Or it is the slow version of the RCF.

请问train_multi_gpu.py在哪里

edge results are different for the three channels

Hello, I observed that the content of the three channels of your edge result map (.jpg) is different. I understand that the three channels should be the same because they are all borders. I hope to get your reply. Thank you very much.