GithubHelp home page GithubHelp logo

hewh16 / rpg_asynet Goto Github PK

View Code? Open in Web Editor NEW

This project forked from uzh-rpg/rpg_asynet

0.0 0.0 0.0 664 KB

Code for the paper "Event-based Asynchronous Sparse Convolutional Networks" (ECCV, 2020).

License: GNU General Public License v3.0

Shell 0.05% C++ 8.05% Python 91.76% CMake 0.13%

rpg_asynet's Introduction

Event-based Asynchronous Sparse Convolutional Networks

Video to Events

This is the code for the paper Event-based Asynchronous Sparse Convolutional Networks (PDF) by Nico Messikommer*, Daniel Gehrig*, Antonio Loquercio, and Davide Scaramuzza.

If you use any of this code, please cite the following publication:

@InProceedings{Messikommer20eccv,
  author        = {Nico Messikommer and Daniel Gehrig and Antonio Loquercio and Davide Scaramuzza},
  title         = {Event-based Asynchronous Sparse Convolutional Networks},
  journal       = {European Conference on Computer Vision. (ECCV)},
  url           = {http://rpg.ifi.uzh.ch/docs/ECCV20_Messikommer.pdf},
  year          = 2020
}

Installation

First set up an Anaconda environment:

conda create -n asynet python=3.7  
conda activate asynet

Then clone the repository and install the dependencies with pip

git clone [email protected]:uzh-rpg/rpg_asynet.git
cd rpg_asynet/
pip install -r requirements.txt

In addition sparseconvnet 0.2 needs to be installed from here.

CPP Bindings

To build the cpp bindings for the event representation tool, you can follow the instructions below:

pip install event_representation_tool/

For the bindings for asynchronous sparse convolutions, we first need to clone the 3.4.0-rc version of Eigen into the include folder. In addition, pybind11 is required.

cd async_sparse_py/include
git clone https://gitlab.com/libeigen/eigen.git --branch 3.4.0-rc1
conda install -c conda-forge pybind11

Finally, the bindings can be installed

pip install async_sparse_py/

Sparse CNN Training

The training parameters can be adjusted in the config/settings.yaml file. The following training tasks and datasets are supported:

  • Classification on NCars and NCaltech101.
  • Object Detection on Prophesee Gen1 Automotive and NCaltech101

Download location of the datasets:

To test the code, make a directory data/ in the root of the repository and download one of the datasets:

mkdir data
cd data
wget http://rpg.ifi.uzh.ch/datasets/gehrig_et_al_iccv19/N-Caltech101.zip
unzip N-Caltech101.zip
rm N-Caltech101.zip

The dataset can be configured in the dataset/name tag in config/settings.yaml. Different model types can be chosen based on the task and whether or not sparse convolutions should be used.

  • Sparse VGG for Classification and Object Detection
  • Standard VGG for Classification and Object Detection

These can be configured in the model tag in config/settings.yaml.

The following command starts the training:

CUDA_VISIBLE_DEVICES=<GPU_ID>, python train.py --settings_file config/settings.yaml

By default, a folder with the current date and time is created in log/ containing the corresponding tensorboard files.

Unit Tests

To test the different asynchronous and sparse layers, multiple unit tests are implemented in unittests/:

  • Asynchronous Sparse Convolution Layer 2D: sparse_conv2D_test.py
  • Asynchronous Sparse Convolution Layer 2D CPP Implementation: sparse_conv2D_cpp_test.py
  • Asynchronous Sparse Max Pooling Layer: sparse_max_pooling_test.py
  • Asynchronous Sparse VGG: sparse_VGG_test.py.
    There are three paths in the script sparse_VGG_test.py specified with 'PATH_TO_MODEL' and 'PATH_TO_DATA', which need to be replaced with the NCaltech Classification dataset and the sparse classification model trained on N-Caltech.

To run the unittests call

python -m unittest discover -s unittests/ -p '*_test.py'

Evaluation

There is one script in the evaluation/sliding_window_flops.py folder for computing the number of FLOPs. The command to execute the script is:

python -m evaluation.sliding_window_flops --setting config/settings.yaml
    --save_dir <PATH_TO_DIR> --num_events 1 --num_samples 500
    --representation histogram  --use_multiprocessing

The output of the script are the numbers of FLOPs for the four processing modes (Asyn Sparse Conv, Asyn Conv, Sparse Conv, Standard Conv).

rpg_asynet's People

Contributors

messikommernico avatar danielgehrig18 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.