Comments (9)
relatedly, whats the correct / recommend way to rewrite the sum over the gammas?
as written, its increment_log_prob = log_sum_exp(gamma)
should it be
a) target+= gamma
b) target+= something something gamma
c) something else?
from example-models.
The code is way out of date. It's in
The current marginalization over the topic (k
) for a given word (n
) where is this:
for (n in 1:N) {
real gamma[K];
for (k in 1:K)
gamma[k] <- log(theta[doc[n],k]) + log(phi[k,w[n]]);
increment_log_prob(log_sum_exp(gamma)); // likelihood
}
That can be reduced to
for (n in 1:N)
target += log_sum_exp(log(theta[doc[n]]) + to_vector(log(phi[ , w[n]])));
It'd be even better to define log_phi
in vector form and reuse for each n
. It would also be worth doing this for log_theta
if the number of words per document is greater than the total number of topics.
from example-models.
@cartazio: Feel free to submit a pull request.
And a warning---you can't really do Bayesian inference for LDA because of the multimodality. You'll see that you won't satisfy convergence diagnostics running in multiple chains, and not just because of label switching.
from example-models.
@bob-carpenter thanks! Thats super helpful.
by multi-mode you mean: there are different local optima when viewed as an optimization problem / things are nonconvex? (ie vary the priors and there will be different local optima in the posterior?). I had to google around to figure out what you meant, https://scholar.harvard.edu/files/dtingley/files/multimod.pdf seemed the most clearly expositional despite the double spaced formatting :)
is there any good reading/references on how the "variational" formulations such as Mallet/VowpalWabbit etc deal with that issue? or is it just one of those things that tends to stay hidden in folklore common knowledge?
from example-models.
from example-models.
from example-models.
from example-models.
from example-models.
from example-models.
Related Issues (20)
- ARM models (radon) not found HOT 1
- Minor typo in occupancy model case study HOT 1
- `stan_demo` example 500 not working HOT 4
- Missing function in Introduction
- Error in BPA Ch.10 js_super.stan with possible fix HOT 1
- rats ref link at top of stan files is broken
- Add .data.R files for knitr models HOT 1
- Rdump format
- Reorganize the stan example repository HOT 13
- sampling() fails in Lotka-Volterra example HOT 3
- Lotka-Volterra example not running HOT 1
- BUGS Background Information for Litter Example HOT 1
- Failed to build Lotka Vollterra on Mac OS / clang HOT 9
- reduce_sum case study will be out of date once new cmdstanr pull gets merged
- Example script `fit_nyc_bym2.R` to use model `bym2_offset_only.stan` doesn't work HOT 1
- Subtle error in some capture-recapture likelihoods HOT 3
- [Error] Validate transformed params
- Survey.stan example from Cognitive Modeling - Expected end of file after end of generated quantities block. HOT 9
- Which book/chapter HOT 1
- updates to ARM model for normal_id_glm - variable name "cov" should be "x_as_mat" or something like that HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from example-models.