slow WARP evaluation about lightfm

[maciejkula]

I've opened #64 which adds a bunch of optimized evaluation functions, please have a look and see if they help.

In general, evaluating learning-to-rank metrics is expensive because you have to compute the full set of recommendations for every user. If you still find it prohibitively slow I would advise choosing a smaller test set by randomly sampling from your users.

@seanlaw you might find this useful too.

[seanlaw]

@maciejkula I was already subscribed for notifications to this issue :)

[bejjani]

Great, thanks!
How are the hyperparameters tuned (step size, regularization)?
I am thinking of using the same rank estimation approach as in WARP when computing the loss on the validation set to let it scale (specially when doing grid search + CV)

[maciejkula]

I'd just use the ranking metric score on the validation set. Especially for WARP watching validation loss (as in neural networks) is not the best approach because the algorithms selects more difficult examples the better it gets, so your raw loss values would be all over the place.

I'd also suggest random search or hyperopt for hyperparams search, but that's up to you :)