GithubHelp home page GithubHelp logo

cl_pytorch's Introduction

CL-Pytorch: Continual Learning Framework for Pytorch

This codebase implements some SOTA continual / incremental / lifelong learning methods by PyTorch.

By the way, this is also the official repository of Adapter Learning in Pretrained Feature Extractor for Continual Learning of Diseases. MICCAI2023

One step baseline method:

  • Finetune: Baseline for the upper bound of continual learning which updates parameters with data of all classes available at the same time.

Continual methods already supported:

  • Finetune: Baseline method which simply updates parameters when new task data arrive.(with or without memory replay of old class data.)
  • iCaRL: Incremental Classifier and Representation Learning. [paper]
  • GEM: Gradient Episodic Memory for Continual Learning. NIPS2017 [paper]
  • UCIR: Learning a Unified Classifier Incrementally via Rebalancing. CVPR2019[paper]
  • BiC: Large Scale Incremental Learning. [paper]
  • PODNet: PODNet: Pooled Outputs Distillation for Small-Tasks Incremental Learning. [paper]
  • WA: Maintaining Discrimination and Fairness in Class Incremental Learning. CVPR2020 [paper]
  • Dark Experience for General Continual Learning: a Strong, Simple Baseline. NeurIPS [paper]
  • DER: Dynamically Expandable Representation for Class Incremental Learning. CVPR2021[paper]
  • L2P: Learning to Prompt for continual learning. CVPR2022[paper]
  • DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning. ECCV2022 [paper]
  • CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learning. CVPR2023 [paper]
  • ACL: Adapter Learning in Pretrained Feature Extractor for Continual Learning of Diseases. MICCAI2023 [paper]

Contrastive model pretraining methods already supported:

  • MoCov2: Improved Baselines with Momentum Contrastive Learning. [paper]
  • SimSiam: Exploring Simple Siamese Representation Learning. [paper]

Coming soon:

  • LwF: Learning without Forgetting. ECCV2016 [paper]
  • EWC: Overcoming catastrophic forgetting in neural networks. PNAS2017 [paper]
  • LwM: Learning without Memorizing. [paper]
  • Layerwise Optimization by Gradient Decomposition for Continual Learning. CVPR2021[paper]
  • FOSTER: Feature Boosting and Compression for Class-incremental Learning. ECCV 2022 [paper]
  • Class-Incremental Continual Learning into the eXtended DER-verse. TPAMI 2022 [paper]
  • Continual Learning with Bayesian Model based on a Fixed Pre-trained Feature Extractor. MICCAI2021[paper]
  • Continual Learning of New Diseases with Dual Distillation and Ensemble Strategy. MICCAI2020[paper]

How to Use

Prepare environment

conda create -n CL_Pytorch python=3.8
conda activate CL_Pytorch
pip install -r requirement.txt

Run experiments

  1. Edit the hyperparameters in the corresponding options/XXX/XXX.yaml file

  2. Train models:

python main.py --config options/XXX/XXX.yaml
  1. Test models with checkpoint (ensure save_model option is True before training)
python main.py --checkpoint_dir logs/XXX/XXX.pkl

If you want to temporary change GPU device in the experiment, you can type --device #GPU_ID in terminal without changing 'device' in .yaml config file.

Add datasets and your method

Add corresponding dataset .py file to datasets/. It is done! The programme can automatically import the newly added datasets.

we put continual learning methods inplementations in /methods/multi_steps folder, pretrain methods in /methods/pretrain folder and normal one step training methods in /methods/singel_steps.

Supported Datasets:

  • Natural image datasets: CIFAR-10, CIFAR-100, ImageNet100, ImageNet1K, ImageNet-R, TinyImageNet, CUB-200

  • Medical image datasets: MedMNIST, path16, Skin7, Skin8, Skin40

More information about the supported datasets can be found in datasets/

We use os.environ['DATA'] to access image data. You can config your environment variables in your computer by editing ~/.bashrc or just change the code.

Reproduce Results

More details can be found in Reproduce_results.md.

References

We sincerely thank the following works for providing help.

https://github.com/zhchuu/continual-learning-reproduce

https://github.com/G-U-N/PyCIL

https://github.com/GT-RIPL/CODA-Prompt

https://github.com/aimagelab/mammoth

ToDo

  • Results need to be checked: ewc
  • Methods need to be modified: mas, lwf
  • Multi GPU processing module need to be add.
  • A detailed documentation is coming soon

cl_pytorch's People

Contributors

fatflower avatar giantjun avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

cl_pytorch's Issues

Link to PATh16 dataset

Hi,
Great code.

Can you please provide the link to the path16 dataset? The link mentioned in the paper needs to be fixed.

Thanks

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.