Extend LightFM with ANN ordering for recommendations about lightfm

[lesterlitch]

Ok I've tidied up and done an example of using NSMlib and Annoy for item and member recommendations. I also did a speed / scaling chart. I haven't worked out how to do a bulk predict yet in order to calculate avg. P@K, but there is a qualitative example.

https://github.com/lesterlitch/misc/blob/master/Light%2Bfm%2Bannoy%2Band%2Bproduct%2Bsearch%2Bexample.ipynb

[musicformellons]

Ouch, you closed it after having me in the dark for 5 months... a small comment, please!!

[maciejkula]

Sorry! I agree that many applications will have an ANN model to accompany this. However, this is probably best accomplished with an external package.

As to guidance: you're probably right, we could add an example ipython notebook that shows how to do this. Would you like to have a stab at a first draft?

[lesterlitch]

I did a rough example using annoy for item-to-item filtering and an example of maximum inner product search with ball tree. If the approach is sound I could tidy this up for an example?

https://github.com/lesterlitch/misc/blob/master/Light%2Bfm%2Bannoy%2Band%2Bproduct%2Bsearch%2Bexample.ipynb

[maciejkula]

Sure, I'd love an example! One thing that might be useful as well is some discussion on the pros and cons of using something like Annoy or NMSlib.

[maciejkula]

Great, thanks! Would be great to include this in the examples and integrate in the documentation. Would you mind submitting a PR and we can go from there?

It might even be worth having a lightfm.contrib.ann module and actually build these functions in. I have historically been wary of including heavy(ish) external dependencies, but if we keep them optional it should be fine.

[LongbinChen]

believe it or not, using numpy's matrix multiplication is fast enough for most of the application. with a 256 vector, search from 1 M doc only cost 100ms.

[hammadkhann]

Right now I am trying to use annoy using item embedding and I am confused about how apply my filters on the result and also how to incorporate user embedding while giving recommendations ??? @lesterlitch @maciejkula

[SimonCW]

I’m closing this issue because it has been inactive for a long time. If you still encounter the problem, please open a new issue.

Thank you!