SSD+MobileNet： FPS about pelee

[foralliance]

@Robert-JunWang

There seem to be two ways of calculating FPS.

1.via the 'time' model of caffe with batch_size = 1：

build/tools/caffe time -model=models/VGGNet/VOC0712/refinedet_vgg16_320x320/deploy.prototxt -gpu=0

2.get the time interval and estimate the FPS.
for example, got the following lines after I run examples/ssd/score_ssd_pascal.py:

I0713 20:50:25.756376 117599 net.cpp:684] Ignoring source layer mbox_loss
I0713 20:51:32.472930 117599 solver.cpp:531] Test net output #0: detection_eval = 0.723217
I0713 20:51:32.473057 117599 solver.cpp:325] Optimization Done.

The total time spent on evaluating 4952 test images is about 67 seconds, and thus FPS = 4952 / 67 = 74.

---

Which method should be used to calculate FPS? Is there any other way?

[Robert-JunWang]

The FPS of intel is got by Caffe time. The FPS of iPhone is calculated by the average time of 100 images processed by the CoreML model. This time includes the image pre-processing time, but it does not include the time of the post-processing part (decoding the bounding-boxes and performing non-maximum suppression). Usually, post-processing is done on the CPU, which can be executed asynchronously with the other parts that are executed on mobile GPU. Hence, the actual speed should be very close to our test result.

[Robert-JunWang]

I think both methods are fine, as long as you evaluate all the models in the same way.

[foralliance]

@Robert-JunWang
thanks your reply
The difference between the two approaches is a bit big.

[ujsyehao]

@foralliance what is batch_size in your method2?
I think batch size affect speed much.