columbiadvmm / tamp Goto Github PK

Hi All,

Thanks for open source your design and providing interface to caffe implementation! I have a question about evaluating your performance in terms of training time of alexnet.

Here is the result of time consumption of traditional caffe over alexnet:
"""
I0208 16:09:37.450343 18112 caffe.cpp:401] Average time per layer:
I0208 16:09:37.450346 18112 caffe.cpp:404] data forward: 1.36319 ms.
I0208 16:09:37.450350 18112 caffe.cpp:407] data backward: 0.0024288 ms.
I0208 16:09:37.450354 18112 caffe.cpp:404] conv1 forward: 12.9353 ms.
I0208 16:09:37.450358 18112 caffe.cpp:407] conv1 backward: 17.3249 ms.
I0208 16:09:37.450361 18112 caffe.cpp:404] relu1 forward: 2.56323 ms.
I0208 16:09:37.450364 18112 caffe.cpp:407] relu1 backward: 3.82991 ms.
I0208 16:09:37.450368 18112 caffe.cpp:404] norm1 forward: 2.89879 ms.
I0208 16:09:37.450371 18112 caffe.cpp:407] norm1 backward: 4.46302 ms.
I0208 16:09:37.450381 18112 caffe.cpp:404] pool1 forward: 2.44754 ms.
I0208 16:09:37.450404 18112 caffe.cpp:407] pool1 backward: 8.88663 ms.
I0208 16:09:37.450408 18112 caffe.cpp:404] conv2 forward: 16.7941 ms.
I0208 16:09:37.450412 18112 caffe.cpp:407] conv2 backward: 39.8721 ms.
I0208 16:09:37.450414 18112 caffe.cpp:404] relu2 forward: 1.65409 ms.
I0208 16:09:37.450417 18112 caffe.cpp:407] relu2 backward: 2.42135 ms.
I0208 16:09:37.450438 18112 caffe.cpp:404] norm2 forward: 2.13315 ms.
I0208 16:09:37.450440 18112 caffe.cpp:407] norm2 backward: 3.20389 ms.
I0208 16:09:37.450443 18112 caffe.cpp:404] pool2 forward: 1.60964 ms.
I0208 16:09:37.450446 18112 caffe.cpp:407] pool2 backward: 5.85156 ms.
I0208 16:09:37.450449 18112 caffe.cpp:404] conv3 forward: 8.77778 ms.
I0208 16:09:37.450453 18112 caffe.cpp:407] conv3 backward: 18.8411 ms.
I0208 16:09:37.450455 18112 caffe.cpp:404] relu3 forward: 0.600653 ms.
I0208 16:09:37.450459 18112 caffe.cpp:407] relu3 backward: 0.890301 ms.
I0208 16:09:37.450462 18112 caffe.cpp:404] conv4 forward: 6.55406 ms.
I0208 16:09:37.450465 18112 caffe.cpp:407] conv4 backward: 15.3713 ms.
I0208 16:09:37.450469 18112 caffe.cpp:404] relu4 forward: 0.601238 ms.
I0208 16:09:37.450472 18112 caffe.cpp:407] relu4 backward: 0.891581 ms.
I0208 16:09:37.450475 18112 caffe.cpp:404] conv5 forward: 4.26468 ms.
I0208 16:09:37.450479 18112 caffe.cpp:407] conv5 backward: 9.63692 ms.
I0208 16:09:37.450481 18112 caffe.cpp:404] relu5 forward: 0.404426 ms.
I0208 16:09:37.450484 18112 caffe.cpp:407] relu5 backward: 0.618806 ms.
I0208 16:09:37.450489 18112 caffe.cpp:404] pool5 forward: 0.393098 ms.
I0208 16:09:37.450491 18112 caffe.cpp:407] pool5 backward: 1.35045 ms.
I0208 16:09:37.450495 18112 caffe.cpp:404] fc6 forward: 6.99653 ms.
I0208 16:09:37.450497 18112 caffe.cpp:407] fc6 backward: 5.61412 ms.
I0208 16:09:37.450500 18112 caffe.cpp:404] relu6 forward: 0.0519008 ms.
I0208 16:09:37.450503 18112 caffe.cpp:407] relu6 backward: 0.0635392 ms.
I0208 16:09:37.450507 18112 caffe.cpp:404] drop6 forward: 0.0957504 ms.
I0208 16:09:37.450510 18112 caffe.cpp:407] drop6 backward: 0.0612928 ms.
I0208 16:09:37.450513 18112 caffe.cpp:404] fc7 forward: 1.68963 ms.
I0208 16:09:37.450516 18112 caffe.cpp:407] fc7 backward: 2.68684 ms.
I0208 16:09:37.450520 18112 caffe.cpp:404] relu7 forward: 0.0450144 ms.
I0208 16:09:37.450522 18112 caffe.cpp:407] relu7 backward: 0.070224 ms.
I0208 16:09:37.450526 18112 caffe.cpp:404] drop7 forward: 0.0842816 ms.
I0208 16:09:37.450529 18112 caffe.cpp:407] drop7 backward: 0.0615776 ms.
I0208 16:09:37.450532 18112 caffe.cpp:404] fc8 forward: 0.73568 ms.
I0208 16:09:37.450536 18112 caffe.cpp:407] fc8 backward: 0.736986 ms.
I0208 16:09:37.450538 18112 caffe.cpp:404] loss forward: 0.145184 ms.
I0208 16:09:37.450541 18112 caffe.cpp:407] loss backward: 0.0344864 ms.
I0208 16:09:37.450556 18112 caffe.cpp:412] Average Forward pass: 76.0813 ms.
I0208 16:09:37.450561 18112 caffe.cpp:414] Average Backward pass: 142.982 ms.
I0208 16:09:37.450568 18112 caffe.cpp:416] Average Forward-Backward: 219.162 ms.
I0208 16:09:37.450574 18112 caffe.cpp:418] Total Time: 2191.62 ms.
"""

Here is the result of time consumption of caffe with your circulant design over alexnet:
"""
I0208 16:11:00.087167 19202 caffe.cpp:401] Average time per layer:
I0208 16:11:00.087170 19202 caffe.cpp:404] data forward: 1.37564 ms.
I0208 16:11:00.087175 19202 caffe.cpp:407] data backward: 0.0028576 ms.
I0208 16:11:00.087180 19202 caffe.cpp:404] conv1 forward: 12.8175 ms.
I0208 16:11:00.087184 19202 caffe.cpp:407] conv1 backward: 16.9658 ms.
I0208 16:11:00.087188 19202 caffe.cpp:404] relu1 forward: 2.56445 ms.
I0208 16:11:00.087190 19202 caffe.cpp:407] relu1 backward: 3.82018 ms.
I0208 16:11:00.087193 19202 caffe.cpp:404] norm1 forward: 2.90364 ms.
I0208 16:11:00.087198 19202 caffe.cpp:407] norm1 backward: 4.49031 ms.
I0208 16:11:00.087199 19202 caffe.cpp:404] pool1 forward: 2.52628 ms.
I0208 16:11:00.087203 19202 caffe.cpp:407] pool1 backward: 9.33765 ms.
I0208 16:11:00.087205 19202 caffe.cpp:404] conv2 forward: 16.5641 ms.
I0208 16:11:00.087209 19202 caffe.cpp:407] conv2 backward: 39.1221 ms.
I0208 16:11:00.087213 19202 caffe.cpp:404] relu2 forward: 1.6541 ms.
I0208 16:11:00.087215 19202 caffe.cpp:407] relu2 backward: 2.42211 ms.
I0208 16:11:00.087218 19202 caffe.cpp:404] norm2 forward: 2.12487 ms.
I0208 16:11:00.087221 19202 caffe.cpp:407] norm2 backward: 3.1529 ms.
I0208 16:11:00.087224 19202 caffe.cpp:404] pool2 forward: 1.72133 ms.
I0208 16:11:00.087236 19202 caffe.cpp:407] pool2 backward: 6.10486 ms.
I0208 16:11:00.087258 19202 caffe.cpp:404] conv3 forward: 8.78861 ms.
I0208 16:11:00.087263 19202 caffe.cpp:407] conv3 backward: 18.4702 ms.
I0208 16:11:00.087266 19202 caffe.cpp:404] relu3 forward: 0.603286 ms.
I0208 16:11:00.087285 19202 caffe.cpp:407] relu3 backward: 0.89009 ms.
I0208 16:11:00.087288 19202 caffe.cpp:404] conv4 forward: 6.57151 ms.
I0208 16:11:00.087292 19202 caffe.cpp:407] conv4 backward: 15.254 ms.
I0208 16:11:00.087296 19202 caffe.cpp:404] relu4 forward: 0.602928 ms.
I0208 16:11:00.087299 19202 caffe.cpp:407] relu4 backward: 0.888906 ms.
I0208 16:11:00.087302 19202 caffe.cpp:404] conv5 forward: 4.26599 ms.
I0208 16:11:00.087306 19202 caffe.cpp:407] conv5 backward: 9.4658 ms.
I0208 16:11:00.087309 19202 caffe.cpp:404] relu5 forward: 0.404566 ms.
I0208 16:11:00.087313 19202 caffe.cpp:407] relu5 backward: 0.602019 ms.
I0208 16:11:00.087316 19202 caffe.cpp:404] pool5 forward: 0.431626 ms.
I0208 16:11:00.087321 19202 caffe.cpp:407] pool5 backward: 1.40352 ms.
I0208 16:11:00.087323 19202 caffe.cpp:404] fc6 forward: 6.23818 ms.
I0208 16:11:00.087327 19202 caffe.cpp:407] fc6 backward: 9.62638 ms.
I0208 16:11:00.087329 19202 caffe.cpp:404] relu6 forward: 0.0550912 ms.
I0208 16:11:00.087333 19202 caffe.cpp:407] relu6 backward: 0.069808 ms.
I0208 16:11:00.087337 19202 caffe.cpp:404] fc7 forward: 10.9973 ms.
I0208 16:11:00.087340 19202 caffe.cpp:407] fc7 backward: 9.6887 ms.
I0208 16:11:00.087343 19202 caffe.cpp:404] relu7 forward: 0.0512128 ms.
I0208 16:11:00.087347 19202 caffe.cpp:407] relu7 backward: 0.0694688 ms.
I0208 16:11:00.087350 19202 caffe.cpp:404] fc8 forward: 0.450304 ms.
I0208 16:11:00.087354 19202 caffe.cpp:407] fc8 backward: 0.733661 ms.
I0208 16:11:00.087357 19202 caffe.cpp:404] loss forward: 0.136755 ms.
I0208 16:11:00.087362 19202 caffe.cpp:407] loss backward: 0.0364608 ms.
I0208 16:11:00.087378 19202 caffe.cpp:412] Average Forward pass: 84.08 ms.
I0208 16:11:00.087383 19202 caffe.cpp:414] Average Backward pass: 152.801 ms.
I0208 16:11:00.087388 19202 caffe.cpp:416] Average Forward-Backward: 236.982 ms.
I0208 16:11:00.087394 19202 caffe.cpp:418] Total Time: 2369.82 ms.
"""

I evaluate time performance with the built-in support of caffe over single GTX1080. But I believe you did speed up since I could tell from the time-stamp of saved models by caffe. May I ask how to correctly evaluate your time performance?

Thanks!

I trained AlexNet with your design but only got 0.4568 accuracy. Just want to confirm if you got similar result as me.

CXX/LD -o .build_release/examples/cifar10/convert_cifar_data.bin
.build_release/lib/libcaffe.so：对‘structured::functor::LayerOpFunctorstructured::CPUDevice::operator()(structured::CPUDevice const&, structured::ExecutiveCore*, structured::ProcessorTape*, structured::ProcessorTape*)’未定义的引用