victorleelk / vsst

Imbalanced Video Summarization

VSST

Here we provide the code for the following paper:

   Imbalanced Video Summarization via Spatio-Temporal Deep Architecture

Updates

• Nov 21, 2016
  • Release the first version VSST (branch: master) compatible with caffe toolbox

Two-stream deep ConvNet trained on the SumMe and CLA dataset

First, we provide our trained two-stream CNN models on the SumMe dataset,

"Spatial net model (VSST-RGB&MV)"
"Temporal net model (VSST-RGB&MV)"
"Spatial net model (VSST-imbalance)"
"Temporal net model (VSST-imbalance)"

and CLA dataset.

"Spatial net model (VSST-RGB&MV)"
"Temporal net model (VSST-RGB&MV)"
"Spatial net model (VSST-imbalance)"
"Temporal net model (VSST-imbalance)"

VSST demo code

Here, a matlab demo code for VSST is provided.

• Step 1: RGB Extraction
  You need extract the RGB frames of the video by yourself.
• Step 2: MV Extraction
  You need download the motion vector code and compile it by yourself. Motion Vector
• Step 3: caffe
  You need download the public caffe toolbox. Our VSST code is compatatible with caffe toolbox.
  
• Step 4: train two-stream deep Convnet
  
• Step 5: Extract feature for each video frame through two-stream deep ConvNet
  
• Step 6: use SVR to predict the summmary score for each frame