Comments (4)
I've opened #64 which adds a bunch of optimized evaluation functions, please have a look and see if they help.
In general, evaluating learning-to-rank metrics is expensive because you have to compute the full set of recommendations for every user. If you still find it prohibitively slow I would advise choosing a smaller test set by randomly sampling from your users.
@seanlaw you might find this useful too.
from lightfm.
@maciejkula I was already subscribed for notifications to this issue :)
from lightfm.
Great, thanks!
How are the hyperparameters tuned (step size, regularization)?
I am thinking of using the same rank estimation approach as in WARP when computing the loss on the validation set to let it scale (specially when doing grid search + CV)
from lightfm.
I'd just use the ranking metric score on the validation set. Especially for WARP watching validation loss (as in neural networks) is not the best approach because the algorithms selects more difficult examples the better it gets, so your raw loss values would be all over the place.
I'd also suggest random search or hyperopt for hyperparams search, but that's up to you :)
from lightfm.
Related Issues (20)
- Lightfm 1.17 - kernel dies in Jupyter, Spyder when fitting HOT 3
- How to reproduce the evaluation score in the paper.
- Recommender System with RFM ratings to put into Interaction matrix HOT 9
- Feature Request: Support for Freezing Item Embeddings while Updating User Embeddings During Training
- Incorporating item and user biases in LightFM embeddings for accurate recommendations in Vespa HOT 1
- Item and User Normalization
- Is there a way to save Dataset()? HOT 1
- How to get clusters of users from trained lightfm model on implicit rating model?
- Evaluate AUC on a subset of items
- Auc evalutation return always nan
- Can't install on Python 3.10.6 in Ubuntu HOT 1
- Implementing Incremental Training and Feedback loop using Lightfm
- LightFM kills Jupyter Kernel when fitting model
- Does calculating precision@k for one user makes sense? HOT 1
- Creating recommendations for single users without leveraging user identity features
- Can't install on Python 3.12 HOT 6
- Can't seem to install with with poetry HOT 3
- ModuleNotFoundError: No module named 'lightfm._lightfm_fast_no_openmp'
- Recommendation model with text embedding as item feature
- Same precision@k with different item embeddings?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lightfm.