ivmcl / aognets

Official implementation for our CVPR19 paper, AOGNets: Compositional Grammatical Architectures for Deep Learning

AOGNets: Compositional Grammatical Architectures for Deep Learning

This project provides source code for our CVPR19 paper AOGNets. The code is still under refactoring. Please stay tuned.

Installation

Requirements

Ubuntu 16.04 LTS (although not tested yet, other OS should work too if PyTorch can be installed successfully)

Python 3 (Anaconda is recommended)

CUDA 9 or newer

PyTorch 0.4 or newer

NVIDIA APEX

NVIDIA NCCL.

Clone the repo

$ git clone https://github.com/iVMCL/AOGNets.git
$ cd AOGNets
$ pip install requirements.txt

Some tweaks

USE pillow-simd to speed up pytorch image loader (assume Anaconda is used)

$ pip uninstall pillow
$ conda uninstall --force jpeg libtiff -y
$ conda install -c conda-forge libjpeg-turbo
$ CC="cc -mavx2" pip install --no-cache-dir -U --force-reinstall --no-binary :all: --compile pillow-simd

ImageNet dataset preparation

• Download the ImageNet dataset to YOUR_IMAGENET_PATH and move validation images to labeled subfolders

  • The script may be helpful.

• Create a datasets subfolder under your cloned AOGNets and a symlink to the ImageNet dataset

$ cd AOGNets
$ mkdir datasets
$ ln -s PATH_TO_YOUR_IMAGENET ./datasets/

Perform training on ImageNet dataset

NVIDIA Apex is used for FP16 training.

E.g.,

$ cd AOGNets
$ ./examples/train_fp16.sh aognet_s configs/aognet_imagenet_12M.yaml first_try

See more configuration files in AOGNets/configs. Change the GPU settings in train_fp16.sh

Perform testing with pretrained models

• Pretrained AOGNet_12M in ImageNet on Google Drive

• Pretrained AOGNet_40M in ImageNet on Google Drive

• More coming soon

• Remarks: The provided pretrained models are obtained using the latest refactored code and the performance are slightly different from the results in the paper.

E.g.,

$ cd AOGNets
$ ./examples/test_fp16.sh aognet_s AOGNet_12M_PATH

Citations

Please consider citing the AOGNets paper in your publications if it helps your research.

@inproceedings{AOGNets,
  author    = {Xilai Li and Xi Song and Tianfu Wu},
  title     = {AOGNets: Compositional Grammatical Architectures for Deep Learning},
  booktitle = {{IEEE} Conference on Computer Vision and Pattern Recognition, {CVPR}},
  year      = {2019},
  url       = {https://arxiv.org/pdf/1711.05847.pdf}
}

Contact

Please feel free to report issues and any related problems to Xilai Li (xli47 at ncsu dot edu), Xi Song (xsong.lhi at gmail.com) and Tianfu Wu (twu19 at ncsu dot edu).

License

AOGNets related codes are under RESEARCH ONLY LICENSE.

aognets's Issues

not convergence

./examples/train_fp16.sh aognet_s configs/aognet_imagenet_12M.yaml first_try

we use 4 P40 GPUs, so change the "train_fp16.sh" like this:

Change accordingly

GPUS=0,1,2,3
NUM_GPUS=4
NUM_WORKERS=4

the log is below:
FP16_Optimizer received torch.cuda.FloatTensor with torch.Size([206])
FP16_Optimizer received torch.cuda.HalfTensor with torch.Size([824, 206, 1, 1])
FP16_Optimizer received torch.cuda.FloatTensor with torch.Size([824])
FP16_Optimizer received torch.cuda.FloatTensor with torch.Size([824])
FP16_Optimizer received torch.cuda.FloatTensor with torch.Size([824])
FP16_Optimizer received torch.cuda.FloatTensor with torch.Size([824])
FP16_Optimizer received torch.cuda.FloatTensor with torch.Size([824])
FP16_Optimizer received torch.cuda.HalfTensor with torch.Size([1000, 824])
FP16_Optimizer received torch.cuda.FloatTensor with torch.Size([824])
FP16_Optimizer received torch.cuda.FloatTensor with torch.Size([824])
FP16_Optimizer received torch.cuda.FloatTensor with torch.Size([824])
FP16_Optimizer received torch.cuda.HalfTensor with torch.Size([1000, 824])
FP16_Optimizer received torch.cuda.HalfTensor with torch.Size([1000])
FP16_Optimizer received torch.cuda.HalfTensor with torch.Size([1000])
torch.Size([256, 3, 224, 224])
torch.Size([256, 3, 224, 224])
torch.Size([256, 3, 224, 224])
torch.Size([256, 3, 224, 224])
Epoch: [0][0/1252] Time 12.503 (12.503) Speed 81.903 (81.903) Data 0.984 (0.984) Loss 6.9218750000 (6.9219) Prec@1 0.195 (0.195) Prec@5 0.586 (0.586) lr 0.000064
Epoch: [0][100/1252] Time 2.100 (2.206) Speed 487.515 (464.191) Data 0.001 (0.011) Loss nan (nan) Prec@1 0.000 (0.003) Prec@5 0.000 (0.009) lr 0.006454
Epoch: [0][200/1252] Time 2.116 (2.155) Speed 483.939 (475.125) Data 0.001 (0.006) Loss nan (nan) Prec@1 0.000 (0.001) Prec@5 0.000 (0.004) lr 0.012843
Epoch: [0][300/1252] Time 2.103 (2.137) Speed 487.036 (479.160) Data 0.001 (0.004) Loss nan (nan) Prec@1 0.000 (0.001) Prec@5 0.000 (0.003) lr 0.019233

Improve links to papers in readme

About the link to the AOGNets paper in the readme, it currently points to the PDF version. It would be better if it were to point to the HTML version instead, i.e. https://arxiv.org/abs/1711.05847 . Please consider making this change. Readers can always click on the PDF link themselves.

Secondly, the link to "Mixture Normalization: A Lightweight Integration of Feature Normalization and Attention" is broken. What should it be pointing to?

Add code and models

https://arxiv.org/pdf/1711.05847v3.pdf says:

The code and models are available at https://github.com/iVMCL/AOGNets

So where are they? The last update was over a month ago. It has been over 1.5 years since its first preprint, and there is evidently no code to show for it. Multiple grants have been used too.

model used for detection

Hello! I'm interested in this work. I use AOGNets to faster R-CNN as your paper intorduced. I use the code from https://github.com/jwyang/faster-rcnn.pytorch/tree/pytorch-1.0. And I make a aog.py in lib/model/faster_rcnn you do. The issue is that when I run train_val to train the model end to end, The loss is so high normally. I want to know where is the problem. Is it related to FP-optimizer? If yes, what should I do for this issues?
I'm looking forward to your reply!
Best wishes!

I use the pretrained model in your google driver.