stylize using GPU about fast-neural-style

[jayavanth]

I think you're right. The forward() just takes 300-400ms

[abhiskk]

@jayavanth are you talking about evaluation phase when you stylize using a trained model?

The implementation currently doesn't use GPU while stylizing, I will add that soon.

[jayavanth]

Yes. I just changed lines 153 and 155 to have .cuda() and it's about 3X faster now. I was wondering if it's possible to get it in the millisecond range

[abhiskk]

Okay, actually I have cleaned up the code over the past day so I don't know which version are you using. Can you tell which version you are using, or maybe do a git pull and then add your changes. Feel free to open a pull-request.

As you said I think you just need to do a style_model.cuda() and content_image = content_image.cuda() and maybe do a output = output.cpu() before you call the utils.tensor_save_bgrimage() function.

[jayavanth]

Yes, something like that. Check out PR #10

[abhiskk]

Regarding the speed, I guess you can try passing in smaller sized images but it might not make a big difference.

I think you will have to remove layers from TransformerNet to make better gains but that might result in degradation in quality of the style-transfers.

[jayavanth]

I'm passing 300x200 and it still takes about 3s to finish. Have you tried using compressed vgg?

[abhiskk]

So vgg is only used while training, the running time during evaluation is only dependent upon the TransformerNet architecture, I am not sure how this running time compares to other implementations of neural-style, do you have any idea about that?

[jayavanth]

The fastest neural-style I've seen so far is this

[abhiskk]

Okay, yes I am have come across that implementation, I haven't really benchmarked my implementation for evaluation, I will try and do that soon. Which GPU are you running it on?

[jayavanth]

I'm using TitanX

[abhiskk]

Okay, I don't have a GPU locally and the queue on the server seems to be long right now, I will test it soon. Can you check the running time only for this line. Were you testing the time taken to run the whole script earlier? I think there is some overhead to reading and writing the image and moving around between numpy and cuda.

[abhiskk]

That's good to know, so the actual computations in pytorch are comparable to tensorflow. I guess the 100ms time that implementation reported was also the time for processing.

Actually because the model was saved in CPU format and for GPU you convert it to CUDA it takes a long time as the model size is not small.

In relation to the tensorflow implementation that you mentioned, my network structure is same as that implementation so I think the forward time is comparable to that implementation. Thanks for the help.

[jayavanth]

Yes, it is comparable. The tensorflow implementation takes about 100ms per frame during forward(). It takes an additional ~5s for image i/o and ffmpeg for a 30 frame video.

[abhiskk]

Good to know, I guess it's expected pytorch is slower as compared to Tensorflow and I think there are some performance gains that can be made, specially in broadcasting because currently you have to squeeze and unsqueeze stuff. There is a broadcasting feature planned soon. Also as pytorch matures things will improve.

[abhiskk]

Fixed in 9803eab

[abhiskk]

Hey @jayavanth I benchmarked the pytorch implementation against this tensorflow implementation that you had linked earlier on a GTX1080. For the same image pytorch took 600ms and tensorflow took 700ms.

Can you give more details on how you were benchmarking the pytorch implementation, I suspect I am doing something wrong because it seems unlikely that pytorch is faster than tensorflow.

[jayavanth]

AFAIK, the performance depends on the image size, and hopefully it will plateau. What image sizes are you using? I also noticed that your implementation crashes for large images e.g. 2557x1704. However, the Tensorflow implementation handles large images with a warning message about memory being full. Maybe, the latter error has something to do with Tensorflow vs PyTorch.

[abhiskk]

I was using 1024 x 680 image for my tests. The running times of my implementation and the tensorflow implementation are almost similar so that's good.

Regarding memory yes there are some issues with pytorch right now, for a 2557x1704 the memory requirement will be a lot, around 80GB I guess and pytorch might not be able to work in a smaller memory. Going ahead I think the memory handling of pytorch will become better.

Thanks