Inference time of detection and segmentation with benchmark.py about blitznet

[dvornikita]

Unfortunately, this information is not enough to tell how this matches out results. We don't have 980 ti to test either.
We do take into account NMS, but we don't consider image resize in the beginning.
I would recommend you to try again running the benchmark.py to compare with our results.
That's what I get after running:
$ python benchmark.py --run_name=BlitzNet300_VOC07+12 --image_size=300 --batch_size=1 --x4 --head=nonshared --seg_filter_size=3 --ckpt=1

Mean=40.20ms; Std=4.70ms; FPS=24.9

This is Titan X (not Pascal). If we don't predict detection but only segmentation, it is faster by roughly 1 FPS. You should be able to run the code of benchmark too. Write me if you get errors.

[revilokeb]

For running the benchmark.py I have done the following (now also on Titan X Maxwell):

1. Extracted VOC07 test to DATASETS_ROOT
2. Downloaded VOC07+12 (https://drive.google.com/open?id=0B7XqhdpFpfcIcTNtVU9VeHRLcG8), put into CKPT_ROOT and extracted
3. Then: python3 benchmark.py --run_name=BlitzNet300_VOC07+12 --ckpt=1

This is the first part of the stdout:
CUDA_VISIBLE_DEVICES=2 python3 benchmark.py --run_name=BlitzNet300_VOC07+12 --ckpt=1
[INFO]: Created a loader VOC07 test with 4952 images
2017-09-19 20:59:58.722855: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 0 with properties:
name: GeForce GTX TITAN X
major: 5 minor: 2 memoryClockRate (GHz) 1.2155
pciBusID 0000:0a:00.0
Total memory: 11.92GiB
Free memory: 11.80GiB
2017-09-19 20:59:58.722886: I tensorflow/core/common_runtime/gpu/gpu_device.cc:976] DMA: 0
2017-09-19 20:59:58.722892: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 0: Y
2017-09-19 20:59:58.722899: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX TITAN X, pci bus id: 0000:0a:00.0)
INFO:tensorflow:Restoring parameters from /media/revilokeb/bigdata/models/blitznet/archive/BlitzNet300_VOC07+12/model.ckpt-1000
[INFO]: Restoring parameters from /media/revilokeb/bigdata/models/blitznet/archive/BlitzNet300_VOC07+12/model.ckpt-1000
2017-09-19 21:00:01.498638: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Assign requires shapes of both tensors to match. lhs shape= [21] rhs shape= [64]
[[Node: save/Assign_277 = Assign[T=DT_FLOAT, _class=["loc:@ssd/Conv_6/biases"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](ssd/Conv_6/biases, save/RestoreV2_277/_1)]]

There seems to be a mismatch in tensor sizes, 21 probably being the number of Pascal VOC classes. But what could 64 be on the right hand side? Any idea?

[revilokeb]

Ok, sorry @dvornikita! My bad :-(

You had already posted it: python benchmark.py --run_name=BlitzNet300_VOC07+12 --image_size=300 --batch_size=1 --x4 --head=nonshared --seg_filter_size=3 --ckpt=1

This is working also on my side - result: Mean=43.74ms; Std=3.95ms; FPS=22.9 for Titan X Maxwell

Again sorry for the confusion and thank you for posting what I needed to do!

python benchmark.py --run_name=BlitzNet300_VOC07+12 --image_size=300 --batch_size=1 --x4 --head=nonshared --seg_filter_size=3 --ckpt=1

[INFO]: Mean=301.54ms; Std=33.36ms; FPS=3.3

[dvornikita]

The experiments were conducted on Nvidia Titan-X (not pascal). If you use another gpu, or a cpu, the results will differ