derekgreene / dynamic-nmf Goto Github PK
View Code? Open in Web Editor NEWDynamic Topic Modeling via Non-negative Matrix Factorization
License: Apache License 2.0
Dynamic Topic Modeling via Non-negative Matrix Factorization
License: Apache License 2.0
I was wondering about what kinds of runtime have you encountered in the practical application of this topic model (leaving aside the question of choosing K). In my limited experience, the scikit NMF decomposition algorithm has been extremely fast for small corpora (a matter of seconds) but it slows down drastically at higher K and larger matrices. I have a model currently running with K=20 on a sparse matrix with 4.3 million cells and it has been going for hours. Compared to standard LDA, this is significantly slower.
The scikit learn documentation mentions polynomial time complexity, which would explain the huge changes in execution time I experienced, and I would like to understand whether this is an issue for others as well.
Please forgive if this is a stupid question as I'm new to topic modeling, coding, and DTM. I've found your tutorial to be extremely helpful and user friendly. I just have one question: I'm running multiple DTM on thousands of documents and was wondering if there was a way to see which documents were the most relevant to a particular topic instead of just the arbitrary top 50 (or 100, 1000, etc)?
When I've run TM using MALLET I'm able to see the percentage of how relevant a topic is in any given document and can weed out the documents that have the highest relevance using that data. I'm wondering if there's a way to do that with your method.
I meet a problem when I try to follow your command.
python prep-text.py data/sample/month1 data/sample/month2 data/sample/month3 -o data --tfidf --norm
/home/jjc/anaconda3/lib/python3.7/site-packages/sklearn/externals/joblib/init.py:15: FutureWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.
warnings.warn(msg, category=FutureWarning)
Loaded 347 stopwords
I don't know why... could you please help me to solve the problem?
Please forgive if this is a stupid question as I'm new to topic modeling, coding, and DTM. I am a student from China and I want to get the probability that each document belongs to the topic. How can I get it ? Hope you can help me, thank you very much!!!!!!!
Hello everyone!
Thank you for providing such wonderful tool! I am studying topic model now.
I follow exactly the README instructions, but when I try 'track-dynamic-topics.py out....' I have a KeyError.
Traceback (most recent call last):
File "track-dynamic-topics.py", line 101, in
main()
File "track-dynamic-topics.py", line 57, in main
dynamic_topic_idx = assigned_window_map[window_topic_label]
KeyError: 'month1_06'
Could you please help me with it?
Thank you very much.
Regards,
Tong
Hello everyone! Thanks for the attention!
I'm using this library for an university project that has the scope to analyze the topics in Twitter's data. I used also LDA algorithm for discover topics in tweets. Now I'm interested on using this Dynamic Topic Modeling approach. So, my question is:
P.S. If I have not been too much clear, I can re-write this question more precisely.
Again, thanks a lot for the attention!
Edoardo, an Italian Computer Science student!
I want to use n-gram, to build my window topic model:
python prep-text.py data/sample/month1 data/sample/month2 data/sample/month3 -o data --tfidf --norm --ngram 3
python find-window-topics.py data/*.pkl -k 5 -o out
python display-topics.py out/month1_windowtopics_k05.pkl out/month2_windowtopics_k05.pkl out/month3_windowtopics_k05.pkl
when I display the window topics, why it is still unigram?
Hi:
In the paper, it says: "3) A ranking of every MEPs contributions relative to all window and dynamic topics in the corpus"
Is there a function that out put this result? Or we need to calculate it by self.
Hi I'm Semiha Makinist,
I' a computer engineer and I'm working about aout topic detection and I want you to ask one question about this issue. You said finding dynamic topic, but you gived k value. How can this code dynamic? I didn't understand this situation.
"python find-dynamic-topics.py out/month1_windowtopics_k05.pkl out/month2_windowtopics_k05.pkl out/month3_windowtopics_k05.pkl -k 5 -o out"
Already, Thank you indeed for helping. Have a good work and nice day.
Best regards,
Semiha Makinist
In the results of track-dynamic-topics.py, as shown in - Dynamic Topic: D01, there are 3 topics in window 3.
I am confused. The result means we got 3 topics in a time window?
how to get the topic evolution? I mean, how to know which topic in window 2 changes to topics in window 3, if we get more than 2 topics in window 2.
Hi --
From the example in the docs, it doesn't look like this is really designed to be run on streaming data, but is it possible to do so? If not, are you aware of similar dynamic topic modeling packages that could work on a data stream?
Thanks
Ben
Hi,
first of all many thanks for providing the code, but also your excellent homepage to document all of your analysis on the European Parliament. Are there any plans to enable PIP install for this model?
In my opinion your approach is the only valid Python option for including time in topic models. Meanwhile, the R package stm has become an enormous contribution for any social scientists working with textual data.
Structuring dynamic-nmf
, such that it is as easy to use as possible - e.g. like gensim, would really be awesome for workflow, teaching purposes, etc.
Best,
Carsten
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.