GithubHelp home page GithubHelp logo

adaptive-auto-attack's Introduction

Adaptive AutoAttack

"Automatic Discovery of Adaptive Attacks on Adversarial Defenses"

Chengyuan Yao, Pavol Bielik, Petar Tsankov, Martin Vechev

arxiv: https://arxiv.org/abs/2102.11860v1

Setup:

python version==3.8.X

1. pip install -r requirement.txt
2. pip install git+https://github.com/RobustBench/[email protected]
3. download folder for models from: https://drive.google.com/file/d/1ajCXluAPUPiyLe2ka9i41jj1YQ6u7YX-/view?usp=sharing (2.2GB)
4. unzip the folder and replace the folder zoo/saved_instances

Reproduce main result:

Models need to be selected in the script, and epsilon need to be adjusted.

python bmA_eval.py (A^3 on group A models)
python bmB_eval.py (A^3 on group B models)
python bmC_eval.py (A^3 on group C models)
python autoattack_eval.py (AutoAttack on defenses)
python aa_detector_eval.py (AutoAttack on defenses with detector)

Reproduce scalability result:

python bmA_scalability.py

Reproduce ablation study:

Run Loss formulation comparison:

python progression_comparison.py --algo 0 --dir=<result_dir_name, eg loss_noloss1>

Run TPE vs Random comparison:

python progression_comparison.py --algo 1 --dir=<result_dir_name, e.g tpe_random1>  

Analyze result (need to update the result directory names in the script)

python progression_analysis.py --algo <bool> --dual <bool, to plot TPE vs Random and loss formulation together>

Reproduce the scatter plot on attack-scores:

python density_plot.py --dir=test

adaptive-auto-attack's People

Contributors

lavanova avatar

Stargazers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.