`robust_pca` does not compute on GPU using `mxnet` backend about tensorly

[JeanKossaifi]

Hi @wdeback

Thanks for the detailed feedback!

Proper support for the context of input tensors has been added in master (033ed61).

I am planning to release a new version soon on PyPi but in the meantime the easiest way is to install TensorLy from master:

git clone https://github.com/tensorly/tensorly
cd tensorly
pip install -e .

Or directly:

pip install git+https://github.com/tensorly/tensorly

I incremented the version in the latest commit so you can check whether you are using the correct version by checking that tensorly.__version__ returns 0.3.0.

Let me know if you have any more questions or if this doesn't solve your issue!

[wdeback]

Thanks for your prompt reply, @JeanKossaifi.

Proper support for the context of input tensors has been added in master (033ed61).

I am already using the master (build from source) including the commit with propagation of contexts such as:

D = T.zeros_like(X, **T.context(X))

and I confirmed that the internal tensors (D,E, L_x, etc.) indeed live in the gpu context (see first lines in Out [9] in this notebook).

Nevertheless, most computation of the robust_pca() function itself is done on cpu which, I assume, is not the intended behavior.

Any idea how I can test this further to help you identify the issue?

[JeanKossaifi]

As of now MXNet doesn't yet have an implementation of the SVD, so for every call to the singular-value thresholding operator, the input tensor if copied to CPU, where SVD is performed by NumPy and the result copied back to GPU: my guess is most the time is wasted there..

I am experimenting with the syevd but it does not seem to be robust enough for most use cases..

[wdeback]

MXNet doesn't yet have an implementation of the SVD, so for every call to the singular-value thresholding operator, the input tensor if copied to CPU, where SVD is performed by NumPy and the result copied back to GPU

Aha, yes, that makes sense. Thanks for the heads up.

I'll keep an eye on future developments. (Very nice project, BTW.)