ritwikkanodia / global-pruning-with-minimum-threshold

Global Magnitude Pruning With Minimum Threshold (GPMT)

This GitHub repository is the official repository for the paper "Global Magnitude Pruning With Minimum Threshold Is All We Need".

Set Up

0. Clone this repository.
1. Using Python 3.6.9, create a virtual environment venv with python -m venv myenv and run source myenv/bin/activate.
2. Install requirements with pip install -r requirements.txt for venv.
3. Create a folder which has LabelSmoothing.py, prune_GP.py (or prune_GPMT.py), model_list.py, and the base model.

Training with GP and GPMT

To run the global magnitude pruning without minimum threshold (MT), run the prune_GP.py file. To run the global magnitude pruning with MT, run the prune_GPMT.py file. Finetuning code is included in both the files itself.

Note - you should change the base model's location and the dataset's location in the the prune_GP.py and prune_GPMT.py files before running them.

To run the prune_GP.py file, run the command-

python3 prune_GP.py

To run the prune_GPMT.py file, run the command-

python3 prune_GPMT.py

Dense Models:

This model is the base model that we used for our ResNet-50 on ImageNet experiments.

Architecture	Parameters	Sparsity (%)	Top-1 Acc (%)	Model Links
Resnet-50	25.50M	0.00	77.04	Base Model

GP and GP+MT Pruned Models:

These models are the checkpoints of pruned Resnet-50 on ImageNet models by GP and GP+MT.

Pruning Method	Sparsity (%)	Top-1 Acc (%)	Model Links
GP	80	76.84	Pruned Model
GP + 0.05% MT	80	76.81	Pruned Model
GP	90	75.44	Pruned Model
GP + 0.05% MT	90	75.42	Pruned Model
GP	95.30	71.63	Pruned Model
GP + 0.005% MT	95.30	71.55	Pruned Model
GP	98.05	62.12	Pruned Model
GP + 0.005% MT	98.05	61.83	Pruned Model