mesunhlf / upc-tf

[CVPR 2020] Universal Physical Camouflage Attacks on Object Detectors

Overview

This is the official Tensorflow implementation of the universal physical camouflage (UPC) method proposed in Universal Physical Camouflage Attacks on Object Detectors. The project page (including demo & dataset) is here.

Prerequisites

python 2.7
scipy 1.2.2
opencv-python 4.1.2
tensorflow 1.12rc1
easydict 1.6

Faster-RCNN our code uses the repository of tf-faster-rcnn as an example to attack.
(1) please download the repository from tf-faster-rcnn and compile the code according to its README.md.
(2) run the tools/demo.py to test the installation.
(3) copy the folder lib from tf-faster-rcnn into the root directory of UPC-tf except lib/nets/network.py file, which is uploaded and modified for feeding the tensor in our pipeline.

Run the Code

Download the training data from this link.
For changing the parameters, please refer to the settings in flags.py.
For runing the training code, please execute bash run_main.sh.

Test the Patterns

Requirements: Autodesk 3dsMax 2018 and V-ray 3.6.
(1) train the camouflage patterns
(2) download the dataset AttackScenes and release the human.zip and scenes.zip to get human models and scenes files.
(3) use Autodesk 3dsMax software to paste (i.e. uv mapping function) the trained patterns on human models.
(4) export the human model as (.fbx) format from 3dsMax software and import it into each virtual scene (.3max).
(5) install the V-ray 3.6 plug-ins (for 3dsMax), and render the images from pre-defined cameras.
(6) run the faster-rcnn demo tools/demo.py to test the rendered images.

We also provide the rendered images, please click here.

Pipeline and Examples

Dataset

We collect the first standardized dataset, named AttackScenes, for fairly evaluating the performace of physical attacks under a controllable and reproducible environment.

Environments AttackScenes includes different virtual scenes under various physical conditions.
Cameras For each virtual scene, 18 cameras are placed for capturing images from different viewpoints.
Illuminations The illuminations are accessible at 3 levels, and can be adjusted by controlling the strength of light sources.

Citation

If you find this project is useful for your research, please consider citing:

@inproceedings{Huang2020UPC,
  title={Universal Physical Camouflage Attacks on Object Detectors},
  author={Lifeng Huang and Chengying Gao and Yuyin Zhou and Cihang Xie and Alan L. Yuille and Changqing Zou and Ning Liu},
  booktitle={CVPR},
  year={2020}
}

upc-tf's Issues

Where to find training images?

The paper says that 200 human images are collected to generate universal patterns. Where to download these training images? I notice this repo doesn't contain.

flags.DEFINE_string('attack_srcdir', 'data/person/train/', 'The dir containing the images to train the patterns')

Correct checkpoint file to use?

Based on the parameters in flags.py, I've assumed this is the correct model to download and set for the training:
http://download.tensorflow.org/models/vgg_16_2016_08_28.tar.gz

However, I get the following error when attempting to use this model:
Tensor name "vgg_16/bbox_pred/biases" not found in checkpoint files data/imagenet_weights/vgg_16.ckpt

Is there another checkpoint file I should be using?

UPDATE: Found the checkpoint data in one of their links. I feel as if this should be made clearer to people setting up the project for the first time, as the readme implies you only need the library data from that repo.

tensorflow似乎不支持3.5以下的python，环境似乎有冲突请问是怎么回事呢

Ran out of GPU memory

When I ran the "main.py", the terminal presented the "Ran out of GPU memory when allocating 0 bytes for..."
I tried to run the "nvidia-smi" and found out that it occupied the 10G memory of my GPU.
Is it normal?

Faster RCNN library imports not being recognised

When trying to run the code I'm getting the following error:
ImportError: No module named lib.model.config

This is despite compiling and copying over the lib folder from the faster rcnn code. I know the files are in the correct place as my IDE recognises the imports and the error doesn't show up if I attempt to run it in Python 3. This happens on both my local machine and Google Colab so I know it's not just my environment. Any reason why Python 2 isn't recognising it properly, despite being the prerequisite?

ImportError: libcudart.so.11.0: cannot open shared object file: No such file or directory

ImportError: libcudart.so.11.0: cannot open shared object file: No such file or directory

Train Dataset for CAR

Hi... It's very interesting work.

I have a question. Where or how can we get the train dataset for car ?
Because in this link we can only get the person dataset:

Download the training data from this link.

Also if we can get the generated attack pattern shows in the video, i think will be great