Loaded SVD model with only_unknowns=True, so need create_matrix(), but it finds 0 tuples about python-recsys

[ocelma]

Once you've created the model, and stored it in disk, you can load the SVD model again, but (unfortunately) this does not load the original input matrix, as this takes much time.

The only way for you to do so would be to load the data again from the raw file. Something like:

svd = SVD(filename=sFileTarget) # Loading already computed SVD model
svd.load_data(filename=sFileSource, sep=',', format=dictFormat) # Re-load the matrix each time!
svd.create_matrix() # And (re)create it
(...)

[jmadison222]

Great thanks for the reply. Makes sense. And that's still a big lift since compute() is the more time consuming. But now I get this error:

/appl/build/anaconda/lib/python2.7/site-packages/csc_utils/ordered_set.pyc in __getitem__(self, index)
     50         else:
     51             # assume it's a fancy index list
---> 52             return OrderedSet([self.items[i] for i in index])
     53 
     54     def copy(self):

IndexError: list index out of range

Where the code I have is this:

from recsys.algorithm.factorize import SVD
svd = SVD(filename=sFileTarget) # Loading already computed SVD model
svd.load_data(filename=sFileSource, sep=',', format=dictFormat)
svd.create_matrix()
setCustIDs = set([x[2] for x in svd.get_data().get()[0:100]]) # Get some sample customer IDs
print(setCustIDs)
[(x, svd.recommend(x, n=3, is_row=False, only_unknowns=True)) for x in setCustIDs]

Such that the last line works fine if I do the full compute. That is, this code works:

sFileSource = '/appl/cwa/data/cov_100k.out'
sFileTarget = '/appl/cwa/data/coverage.model'
# dictFormat = {'col':0, 'row':1, 'value':2, 'ids': int}
dictFormat = {'col':0, 'row':1, 'value':2}

import recsys.algorithm
recsys.algorithm.VERBOSE = True
from recsys.algorithm.factorize import SVD
svd = SVD()
svd.load_data(filename=sFileSource, sep=',', format=dictFormat)
k = 5 # Number of clusters
svd.compute(
    k=k, min_values=3, pre_normalize=None, 
    mean_center=False,
    post_normalize=True, savefile=sFileTarget
)
print(svd.get_data().get()[0:5]) # Look at data for sanity check.
setCustIDs = set([x[2] for x in svd.get_data().get()[0:100]]) # Get some sample customer IDs
print(setCustIDs) 
[(x, svd.recommend(x, n=3, is_row=False, only_unknowns=True)) for x in setCustIDs]

And both the working and non-working code are getting the same customer IDs for that print statement:

set([33, 195, 262, 198, 266, 285, 144, 254, 222, 215, 218, 123, 61, 126, 63])

Thoughts?