GithubHelp home page GithubHelp logo

christina200 / l2p-pytorch Goto Github PK

View Code? Open in Web Editor NEW

This project forked from jh-lee-kr/l2p-pytorch

0.0 0.0 0.0 59 KB

PyTorch Implementation of Learning to Prompt (L2P) for Continual Learning @ CVPR22

License: Apache License 2.0

Shell 1.10% Python 98.90%

l2p-pytorch's Introduction

L2P PyTorch Implementation

This repository contains PyTorch implementation code for awesome continual learning method L2P,
Wang, Zifeng, et al. "Learning to prompt for continual learning." CVPR. 2022.

The official Jax implementation is here.

Environment

The system I used and tested in

  • Ubuntu 20.04.4 LTS
  • Slurm 21.08.1
  • NVIDIA GeForce RTX 3090
  • Python 3.8

Usage

First, clone the repository locally:

git clone https://github.com/JH-LEE-KR/l2p-pytorch
cd l2p-pytorch

Then, install the packages below:

pytorch==1.12.1
torchvision==0.13.1
timm==0.6.7
pillow==9.2.0
matplotlib==3.5.3

These packages can be installed easily by

pip install -r requirements.txt

Data preparation

If you already have CIFAR-100 or 5-Datasets (MNIST, Fashion-MNIST, NotMNIST, CIFAR10, SVHN), pass your dataset path to --data-path.

The datasets aren't ready, change the download argument in datasets.py as follows

CIFAR-100

datasets.CIFAR100(download=True)

5-Datasets

datasets.CIFAR10(download=True)
MNIST_RGB(download=True)
FashionMNIST(download=True)
NotMNIST(download=True)
SVHN(download=True)

Training

To train a model via command line:

Single node with single gpu

python -m torch.distributed.launch \
        --nproc_per_node=1 \
        --use_env main.py \
        <cifar100_l2p or five_datasets_l2p> \
        --model vit_base_patch16_224 \
        --batch-size 16 \
        --data-path /local_datasets/ \
        --output_dir ./output 

Single node with multi gpus

python -m torch.distributed.launch \
        --nproc_per_node=<Num GPUs> \
        --use_env main.py \
        <cifar100_l2p or five_datasets_l2p> \
        --model vit_base_patch16_224 \
        --batch-size 16 \
        --data-path /local_datasets/ \
        --output_dir ./output 

Also available in Slurm system by changing options on train_cifar100_l2p.sh or train_five_datasets.sh properly.

Multinode train

Distributed training is available via Slurm and submitit:

pip install submitit

To train a model on 2 nodes with 4 gpus each:

python run_with_submitit.py <cifar100_l2p or five_datasets_l2p> --shared_folder <Absolute Path of shared folder for all nodes>

Absolute Path of shared folder must be accessible from all nodes.
According to your environment, you can use NCLL_SOCKET_IFNAME=<Your own IP interface to use for communication> optionally.

Evaluation

To evaluate a trained model:

python -m torch.distributed.launch --nproc_per_node=1 --use_env main.py <cifar100_l2p or five_datasets_l2p> --eval

Result

Test results on a single gpu.

Split-CIFAR100

Name Acc@1 Forgetting
Pytorch-Implementation 83.77 6.63
Reproduce Official-Implementation 82.59 7.88
Paper Results 83.83 7.63

5-Datasets

Name Acc@1 Forgetting
Pytorch-Implementation 80.22 3.81
Reproduce Official-Implementation 79.68 3.71
Paper Results 81.14 4.64

Here are the metrics used in the test, and their corresponding meanings:

Metric Description
Acc@1 Average evaluation accuracy up until the last task
Forgetting Average forgetting up until the last task

License

This repository is released under the Apache 2.0 license as found in the LICENSE file.

Cite

@inproceedings{wang2022learning,
  title={Learning to prompt for continual learning},
  author={Wang, Zifeng and Zhang, Zizhao and Lee, Chen-Yu and Zhang, Han and Sun, Ruoxi and Ren, Xiaoqi and Su, Guolong and Perot, Vincent and Dy, Jennifer and Pfister, Tomas},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={139--149},
  year={2022}
}

l2p-pytorch's People

Contributors

jh-lee-kr avatar christina200 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.