changdaeoh / blackvip Goto Github PK

Dear authors,

thanks for the great work and the very interesting method!

What hyperparameter is crucial (or how did you tune them) when adapting BlackVIP to custom datasets? Because when I use the default hyperparameters the training seems to diverge and the loss couldn't go down.

Thanks in advance.

I want to run the code for VP (white-box), and use the code according to your suggestion: "sh tl_bench.sh oxford_flowers 100 40.0", and I also change the tl_bench.sh file like this:

Is it right? But it reports an error like this:

If I still run the code like this "sh tl_bench.sh oxford_flowers 100 0.5 0.2 0.02 1.0". It can run without errors.
I just wonder how to evaluate VP(white-box), e.g. for oxford_flowers datasets, the parameters, and the tl_bench.sh file.

Thanks!

We are interested in your work, and we want to reproduce the results. But we find that it runs extremely slow, e.g., about 2 days 8h with a single RTX3090.

We set CFG=vit_b16 and ptb=vit-base, and others followed the configuration.md.

Is something wrong?

We are trying to reproduce the results of VPOUR, but we failed to reproduce the results of VPOUR (Blackbox VP using SPSA-GC). For example, the result we reproduced on EUROSAT dataset is 51.1, which is far below 70.9 reported in the paper. Would you mind checking if the hyperparameters provided are correct?