kakaoenterprise / learning-debiased-disentangled Goto Github PK

It appears that the result of the BAR dataset was in the paper during the submission phase(https://openreview.net/forum?id=-oUhJJILWHb) and was also appreciated by the reviewers. At the same time, it was removed from the final camera-ready version. Is there any particular reason for this? Could you provide the code for replicating the result on the BAR dataset?

At the dataset files description of cmnist and cifar10c
there's typo conlict -> conflict

Hello, thank you very much for the open-source code. I tried to reconstruct CMNIST in the 1pt pre-training model you provided according to the information in the appendix but found that it seemed unable to reconstruct. Could you please provide the code of the reconstruction part?

Hi,
Thanks for sharing the code, it's really helpful. From your code, I can get the acc of bias align, bias conflic and total acc. But could I know how can I calculate the unbiased accuracy? Thanks!

Hello dear authors,

I was wondering if I could have some clarifications on GCE.

1. GCE is almost an interpolation between CCE and MAE. Their respective derivates are the following:
   
2. While CCE dynamically attributes different "weight" (1/f(x)) according to the difficulty of the sample, MAE is uniform across all samples.
3. Usually, "difficult" samples are bias-conflicting samples. Thus, the gradient is higher CCE for this scenario.
4. On the other hand, GCE attributes less gradient in the same scenario. Thus, GCE is less attentive to "difficult" samples, which in turn allocates more attention to "easier" samples in expectation. Therefore, GCE is less focused on intrinsic attributes and seeks after shortcuts

The reason I doubt my statement is due to your clarifications on GCE in OpenReview.

I thought GCE was more attentive to the bias element not because it emphasizes the easy samples but de-emphasizes the hard ones.

Thank you for your attention and great paper!

Hi,

Thanks for open-sourced the code! I made a local copy of the code base as well as the CMNIST dataset. Simply by running the model of MLP and MLP_disentangled with provided Linux command lines as instructed in README. The training loss of either case is not decreasing and hence the test accuracy is similar with a naive classifier (~10%). Could you look into this issue?

Thank you!

Dear authors,
I want to know the data setting of validation set for CMNIST. Does the validation set have the same bias degree with training set? Thanks in advance.

Thanks very much!

When I run the command

python train.py --dataset cmnist --exp=cmnist_0.5_ours --lr=0.01 --percent=0.5pct --curr_step=10000 --lambda_swap=1 --lambda_dis_align=10 --lambda_swap_align=10 --use_lr_decay --lr_decay_step=10000 --lr_gamma=0.5 --train_ours --tensorboard

the output is

...
valid_d: 0.11319999396800995 || test_d: 0.11349999904632568 
 97%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████▌   | 48496/50000 [12:32<00:16, 88.77it/s
48500 model saved ...                                                                                                                                         
valid_d: 0.11319999396800995 || test_d: 0.11349999904632568 
 98%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████▋  | 48996/50000 [12:40<00:10, 96.10it/s]
49000 model saved ...
                                                                                                                                                            
49000 model saved ...                                                                                                                                         
valid_d: 0.11319999396800995 || test_d: 0.11349999904632568 
 99%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊ | 49500/50000 [12:49<00:07, 71.10it/s
49500 model saved ...                                                                                                                                         
valid_d: 0.11319999396800995 || test_d: 0.11349999904632568 

The test acc is 0.1135. However in the paper, the result is 65.22. I don't know what's wrong. Can anyone help me?

Hi,
Thanks for open-sourced the code! Can you please make the data generating code public? I hope to further modify the dataset to test the relationship between bias rate and performance. Your code seems clear from the released dataset, I would love to refer to it.

Dear authors,
What's the official name for your method? I cannot find the name in your paper. I want to compare your method in my paper.
Thanks in advance.

Shaohua