bbenligiray / rml-cnn

Semi-Supervised Robust Deep Neural Networks for Multi-Label Classification

RML-CNN

Code covering some of the experiments in the following paper:

Cevikalp, H., Benligiray, B., Gerek, O. N.. (2019). Semi-Supervised Robust Deep Neural Networks for Multi-Label Image Classification. In Pattern Recognition.

Use this to create a nus_wide.h5 and this to create a ms_coco.h5 file. Download resnet101_weights_tf.h5 from here. Put these in /rml-cnn, then run run.sh to run the experiments.

Alternatively, you can use the loss functions in ml_loss with any model/dataset you like, see main.py for reference. Note that your final layer's activation should be None for all loss functions (including softmax).

rml-cnn's Issues

About the calculation of top-3 index function --关于top-3指标函数计算的问题

I use the evaluation function given by you, in index TOP-3, and the calculation results are as follows:
P_C 65.06 R_C 79.06 F_C 65.92 --- P_O 65.06 R_O 67.81 F_O 66.41

But I found that there was a problem, because in index ALL, the calculation results are as follows:
P_C 79.54 R_C 66.22 F_C 71.70 --- P_O 82.17 R_O 70.42 F_O 75.84

The calculation results of P and R in index ALL and index TOP-3 are contradictory.
I think it's because of our different understanding of top-3 calculation.
我觉得是我们对计算top-3指标的理解不同造成的


如果预测的前三名结果，有一个或两个或三个的概率均低于0.5（没有使用sigmoid时低于0），但是在代码里面全部赋值为1(正标签)，这样做是有问题的，我们应该做一个判断，只有概率大于0.5（使用sigmoid时大于0）的情况才赋值为1...如果这样做的话，我的实验将会和论文《Learning Spatial Regularization with Image-level Supervisions for Multi-label Image Classification》的情况相同，不会出现ALL和TOP-3里面，P和R相互矛盾
If one or two or three of the predicted top three results are less than 0.5 (less than 0 when sigmoid is not used), but all of them are assigned to 1 (positive label) in the code, this is problematic. We should make a judgment that only when the probability is greater than 0.5 (more than 0 when sigmoid is used), the value will be assigned to 1... If this is done, my experiment will be compared with The situation of paper 《Learning Spatial Regularization with Image-level Supervisions for Multi-label Image Classification》is the same. There will be no contradiction between P and R in ALL and TOP-3.

rml-cnn/metrics.py

Line 22 in d3f9f28

top3[ind_example, top_pred_inds[k]] = 1

My code is as follows:

我个人认为你需要对代码进行修改，并且我认为修改后的实验结果，会比现在好
I personally think you need to modify the code, and I think the modified experiment results will be better than now

How to run this model？

Dear Sir:
I've read your paper and I'm very interested in your experiments... but I need some help from you.

1. How to store datasets and ResNet in folders?
   2 .Can you give me some advice on training the model from scratch?

How to Compute mAP

rml-cnn/metrics.py

Line 64 in d3f9f28

rec = tp / np.sum(ground_truth)

When I calculate mAP with labels=[[1,0,1,0,0,0,0,1], [0,1,0,0,0,0,0,0,0,1,0],preds=[[0.6,0.2,0.7,0.3,0.2,0.3,0.4,0.3], [0.2,0.8,0.1,0.2,0.3,0.4,0.7,0.3]], the result is NAN.

I would like to ask what is the value of the parameter labels of function calculate_mAP (labels, preds)?