GithubHelp home page GithubHelp logo

aygong / snip-tensorflow Goto Github PK

View Code? Open in Web Editor NEW
0.0 1.0 0.0 1.61 MB

Code for the project "SNIP: Single-Shot Network Pruning"

Jupyter Notebook 16.53% Python 80.07% MATLAB 3.40%
machine-learning network-pruning

snip-tensorflow's Introduction

CS-439 Optimization for Machine Learning - Class Project

    

Open In Colab

Group Members

Introduction

This project focuses on a network pruning algorithm proposed on ICLR 2019, namely SNIP for single-shot network pruning. We reproduced the experiments of the original paper with a similar experimental setup. In particular, we evaluated the algorithm on two datasets with seven network architectures given by the reference. Our implementation is based on the GitHub repo provided by the authors of the original paper. To further study its robustness, we validated the effectiveness of the algorithm on an additional dataset and nine network architectures, provided numerical and visualization results with respect to a wide range of configurations, and compared our result with the original paper.

Requirements

Dependencies

You can reproduce our results by two ways, using either Google Colab Pro or your local system.

  • Reproduce our results on Colab.

    • Colab contains most of needed packages, and is easy and highly recommended for reproducing our results.

    • Note that our codes requires tensorflow-gpu 1.14.0, which is not pre-installed in Colab. Be sure to run

      # Install required packages
      %tensorflow_version 2.x
      !pip install gast==0.2.2
      !pip uninstall -y tensorflow
      !pip install tensorflow-gpu==1.14.0

      at the beginning of the Colab notebook run.ipynb to setup the environment correctly.

  • Reproduce our results on your local system.

    • To reproduce our results, you should install the following packages.
      • gast 0.2.2
      • matplotlib 3.3.4
      • numpy 1.19.2
      • imgaug 0.4.0
      • tensorflow-gpu 1.14.0
    • GPUs with at least 25.46 GB RAM are required. Otherwise, you will get RuntimeError: CUDA out of memory.

Datasets

Download the following datasets and move the files to the corresponding folders.

  • Download the Kuzushiji-MNIST dataset and move the files into ./KMNIST.
  • Download the MNIST dataset and move the files into ./MNIST.
  • Download the CIFAR-10 dataset and move the files into ./cifar-10-batches-py.

How to Reproduce Our Experiments

You can reproduce our experiments in two ways, using either Google Colab Pro or your local machine.

  • Reproduce our experiments on Colab.

    • First, create a folder called Mini-Project in your Google Drive, i.e., /gdrive/MyDrive/Mini-Project.

    • Second, make sure the structure of the folder is the same as the Folder Structure section.

      • git clone our repo to your local machine and then move the code to the folder.
      • Download the three datasets and then move the files to the folder.
    • Third, click Change runtime type under the menu Runtime, and select GPU as Hardware accelerator and High-RAM as Runtime shape.

    • Last, run the provided Jupyter notebook run.ipynb under the root directory with Run all.

  • Reproduce our experiments on your local machine.

    • First, install all the dependencies mentioned in the Requirement section.
    • Second, create a folder called Mini-Project on your local machine, i.e., ./Mini-Project.
    • Third, make sure the structure of the folder is the same as the Folder Structure section.
      • git clone our repo to your local machine and then move the code to the folder.
      • Download the three datasets and then move the files to the folder.
    • Last, run the provided Jupyter notebook run.ipynb under the root directory with Run all.

Folder Structure

./Mini-Project
├── README.md
|
├── results.xlsx                    # All the results from our experiments
|
├── report.pdf                      # Our report
|
├── run.ipynb                       # The Jupyter notebook to run the algorithm and reproduce our results.
|
├── snip/                           # Folder contains all the sources files to run the algorithm
|   ├── train.py                    # Define the training process
|   ├── test.py                     # Define the test process
|   ├── prune.py                    # Define the pruning process which is executed before training
|   ├── network.py                  # Define multiple neural network architectures
|   ├── model.py                    # Define the entire model, including the network, loss, optimization, and input/output processing, etc.
|   ├── mnist.py                    # Process the MNIST dataset
|   ├── main.py                     # Main file that parses arguments and initializes the entire training and testing procedure
|   ├── kmnist.py                   # Process the KMNIST dataset
|   ├── helpers.py                  # Several auxiliary functions
|   ├── dataset.py                  # Process the dataset to feed the model
|   ├── cifar.py                    # Process the CIFAR-10 dataset
|   └── augment.py                  # Perform data augmentation
|
├── cifar-10-batches-py/            # Folder for the CIFAR-10 dataset
|   ├── batches.meta
|   ├── data_batch_1
|   ├── data_batch_2
|   ├── data_batch_3
|   ├── data_batch_4
|   ├── data_batch_5
|   ├── readme.html
|   └── test_batch
|
├── KMNIST/                         # Folder for the KMNIST dataset
|   ├── t10k-images-idx3-ubyte.gz
|   ├── t10k-labels-idx1-ubyte.gz
|   ├── train-images-idx3-ubyte.gz
|   └── train-labels-idx1-ubyte.gz
|
└── MNIST/                          # Folder for the MNIST dataset
    ├── t10k-images-idx3-ubyte.gz
    ├── t10k-labels-idx1-ubyte.gz
    ├── train-images-idx3-ubyte.gz
    └── train-labels-idx1-ubyte.gz

Useful Links

The algorithm of SNIP is presented in our report and the original paper.

The following network architectures are used for evaluating.

snip-tensorflow's People

Contributors

akaqyd avatar aygong avatar skye-du avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.