ylsung / pytorch-adversarial-training Goto Github PK

In your experiment for Cifar10, the l_inf perturbed model has a perturbation size of 4/255 = 0.0157. The achieved robust accuracy achieved in their paper (https://arxiv.org/abs/1805.12152) is 57.79% as in table 4, whereas your experiment achieves a robust accuracy of 63.8%

Could you please give some explanation about this difference.

Hi, thanks for the good work!
I'm tring to use the attack for my own purpose.
It seems that no normalization, such as transforms.Normalize(), was used, so the input ranges from 0-1.
https://github.com/louis2889184/pytorch-adversarial-training/blob/1103fe300dc08f740b6870aebdd40a87d5690a45/cifar-10/main.py#L206-L210
As far as I know, it's comon pratice to normalize a tensor image with given mean and standard deviation. Then the input would have bigger range.
If so, when
https://github.com/louis2889184/pytorch-adversarial-training/blob/1103fe300dc08f740b6870aebdd40a87d5690a45/cifar-10/src/attack/fast_gradient_sign_untargeted.py#L113
the perturbated images are total different. In my case, the attack destoryed the training process and the model went crazy actually.
My question is what should I do to solve it?
Besides, any other changes I should make if the attack is to be used for general purposes?

Especailly, adv_trained checkpoint for cifar10.

I think the default value of alpha in cifar10 should be 2/255 instead of 2 since you choose default epsilon to be 4/255?

Hi, first thanks for your great work!

I wanna know more information about the updated checkpoint of pgd trained Madry's model on cifar-10. Was this checkpoint stored when the whole 76000 iterations were down? I ran PGD-20 attack to your trained model and the accuracy is 50.05% while it's 47.04% reported in the leaderboard from Madrylab's cifar-10 challenge. Is there any possible reason for such a difference?

Thanks for your attention. Looking forward to your reply.

Hi! Thanks for sharing the code!

Got a question here. In the train function, when you do opt.step(), the weights will be updated with both the loss in the main function and the loss in the function where you generate the perturbation, right?
But I don't think that the weights should be updated with the information from the loss in the generator function.

Hi, thanks for your great work!

I ran the code without adv_train and adv_test in cifar-10 and got the acc about 87%. It is the same with the value reported in the page of this repository, but not the same as the value reported in Madry's paper[1], which is 95.2%.

So, I am confused about such results. Especially as reported, the result of l_inf training model got the robustness acc 63.8% v.s. 55.97% in Madry's model. Are they some differences between this implementation and the paper.

Thanks for your attention and looking forward to your reply!

[1] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083, 2017.

Hi,

I run the code and test adv acc is 0.18% after training. Does this mean that the model is still not robust after AT?

Thanks,
Xiang