Do you get 130 img/s with D0 for batch=1? about efficientdet-pytorch

[rwightman]

@AlexeyAB nope, definitely not, I think that was batch of 64. Don't see a way to get anything close to that with batch=1 and pure python + pytorch eager.

Run nore recently on a Titan RTX
batch_size=128, 200img/s for forward + post-processing (top-k, nms, conversion to coco fmt)
batch_size=1, 16img/s

removing python post proc, just the box/class head outputs:
batch_size=128, 480img/s
batch_size=1, 27img/s

Based on those spreads, likely some significant optimizations could be done on post processing and also network inference (tensorrt, etc).

All these were using Apex AMP mixed prec, so tensorcores in some cases, last I looked PyTorch wasn't doing a great job of getting all the depthwise separable convs onto tensorcores...