GithubHelp home page GithubHelp logo

transhp's Introduction

TransHP

[NeurIPS 2023] The official implementation of TransHP: Image Classification with Hierarchical Prompting.

image

Environment

Our TransHP uses PyTorch 1.8.0 and timm 0.4.12. They can be easily installed by:

pip install torch==1.8.0+cu111 torchvision==0.9.0+cu111 torchaudio==0.8.0 -f https://download.pytorch.org/whl/torch_stable.html
pip install timm==0.4.12

The minimum hardware requirement of our TransHP is 8 V100.

Dataset

Download and extract ImageNet train and val images from http://image-net.org/. The directory structure is the standard layout for the torchvision datasets.ImageFolder, and the training and validation data is expected to be in the train/ folder and val/ folder respectively.

/path/to/imagenet/
  train/
    class1/
      img1.jpeg
    class2/
      img2.jpeg
  val/
    class1/
      img3.jpeg
    class2/
      img4.jpeg

Train

python -m torch.distributed.launch --nproc_per_node=8 --use_env main.py \
--model deit_small_hi_patch16_224 --batch-size 128 --data-path /path/to/imagenet/ \
--output_dir ./ckpt/TransHP/

Test

We release our trained model and corresponding logs, you should download and save it. Then you can test its performance by

python -m torch.distributed.launch --nproc_per_node=8 --use_env main.py --eval \
--resume released_checkpoint.pth --model deit_small_hi_patch16_224 \
--data-path /path/to/imagenet/

This should give

* Acc@1 78.514 Acc@5 93.614 loss 0.948

Note that this is a bit different from our reported performance (78.65) in our paper due to the randomness in the reproducing.

Known issues

  1. If the loss is NaN, please check facebookresearch/deit#29.

  2. One limitation of our TransHP is its need to select prompting blocks and tune the balance parameters. Any advice or follow-up work to solve this problem is welcomed.

  3. Another limitation is most of our experiments are performed on a lightweight Vision Transformer (ViT) due to insufficient computing resources. I am happy to see more experiments on larger ViTs or bigger datasets.

Citation

@inproceedings{
    wang2023transhp,
    title={Trans{HP}: Image Classification with Hierarchical Prompting},
    author={Wenhao Wang and Yifan Sun and Wei Li and Yi Yang},
    booktitle={Thirty-seventh Conference on Neural Information Processing Systems},
    year={2023},
    url={https://openreview.net/forum?id=vpQuCsZXz2}
}

Acknowledgement

We implement our TransHP based on DeiT. Our baseline is a lightweight ViT, i.e. ViT-small.

transhp's People

Contributors

wangwenhao0716 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

transhp's Issues

What do these twelve losses represent?

When I try to train, I get a running error in the loss function, because the data dimension is incorrect, what do the twelve losses represent?Would you please answer it. BTW, I use the dataset CIFAR100 instead of image net1k, does that make a big difference?I would appreciate your help.
d91f414ca71b257d2ad799084bb6dfb0

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.