muximuxi / tf-decompose

Tensor decomposition implemented in TensorFlow

tf-decompose

CP and Tucker tensor decompositions implemented in TensorFlow.

Usage

import numpy as np
import tensorflow as tf
from scipy.io.matlab import loadmat
from ktensor import KruskalTensor

# Load sensory bread data (http://www.models.life.ku.dk/datasets)
mat = loadmat('data/bread/brod.mat')
X = mat['X'].reshape([10,11,8])

# Build ktensor and learn CP decomposition using ALS with specified optimizer
T = KruskalTensor(X.shape, rank=3, regularize=1e-6, init='nvecs', X_data=X)
X_predict = T.train_als(X, tf.train.AdadeltaOptimizer(0.05), epochs=20000)

# Save reconstructed tensor to file
np.save('X_predict.npy', X_predict)

Kruskal tensors

Notes on ALS gradient computation

• For CP decomposition we use alternating least squares' (ALS) over component matrices, but do not compute the exact solution as in Kolda & Bader (2009) due to the computational demands of computing large matrix inversions.
• In our tests we find inferior results to the exact solution descent method (requires inverting potentially huge matrices) implemented in scikit-tensor with ~.80 vs. ~.90 fit with decomposed rank-3 tensors on the Sensory Bread dataset.
• tf-decompose parallelized on GPU was approximately 20 times faster than scikit-tensor for a rank-200 decomposition of a random tensor with 60 million parameters.

Tucker tensors

Preliminary results: with sensory bread data, TuckerTensor.hosvd seems to perform quite poorly, while TuckerTensor.hooi and DecomposedTensor.train_als learn reconstructions with fit ~0.70.

References

Bader, Brett W., and Tamara G. Kolda. "Efficient MATLAB computations with sparse and factored tensors." SIAM Journal on Scientific Computing 30.1 (2007): 205-231.

Kolda, Tamara G., and Brett W. Bader. "Tensor decompositions and applications." SIAM review 51.3 (2009): 455-500.

Nickel, Maximilian. scikit-tensor

Also see: tensorD (code, paper). I wrote tf-decompose before this was available; I haven't used it, but you should check it out as well if you're considering using tf-decompose.