cscherrer / tilde.jl

WIP successor to Soss.jl

Tilde

WIP, successor to Soss.jl

For a high-level description of Tilde's design, check out this blog post

julia> m = @model (x, s) begin
        σ ~ Exponential()
        @inbounds x[1] ~ Normal(σ = σ)
        n = length(x)
        @inbounds for j = 2:n
            x[j] ~ StudentT(1.5, x[j - 1], σ)
        end
    end;
    
julia> x = zeros(3);

julia> r = rand(m(x,10))
(σ = 9.096155145583953, x = [14.876934886768867, 6.612967845518229, 2.045770246490428])

julia> x
3-element Vector{Float64}:
 14.876934886768867
  6.612967845518229
  2.045770246490428

julia> ℓ = logdensityof(m(x, 1.0) | (;x), (σ = 1.0,))
-122.91114458882001

tilde.jl's Issues

Struct arrays:

Seems like Tilde lost the ability to use struct arrays for dual numbers: Eg. in the context of

    post = model_lr(At, y) | (;y)
    as_post = as(post)
    obj(θ) = -Tilde.unsafe_logdensityof(post, transform(as_post, θ))

    ith = zeros(d)
    function δobj(x, i)
        ith[i] = 1
        sa = StructArray{ForwardDiff.Dual{}}((x, ith))
        δ = obj(sa).partials[]
        ith[i] = 0
        return δ
    end

doesn't work anymore (tries to convert to Float64)

Transport to Gaussian conditionals

@mschauer mentioned in Slack that Mitosis.jl likes to work with nonlinear models that have Gaussian conditionals. With the upcoming transport API in MeasureBase.jl, we should be able to automate this in Tilde.

If a model has a line like

x ~ foo

it can be replaced with e.g.

_x_dof = getdof(foo)
_x_raw ~ StdNormal() ^ _x_dof
x = transport_to(foo, StdNormal() ^ _x_dof, _x_raw)

Automating this will make it easier to use Mitosis for inference on Tilde models.

TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Unbiased Tilde-models

Below I create two logistic regression tilde models, full_model and hat_model.
Note that the hat_model is stochastic and its gradient is an unbiased estimate of the full model, scaled by K/N, because
each observation is picked with probability K/N.
You see that I use a trick for the global parameters which are picked alway (with probability 1) because α ~ Tilde.Normal(0,sqrt(N1/K)) gives a log-likelihood which, when scaled together with the data, goes back to α ~ Tilde.Normal(0,1).

full_model = @Tilde.model N1, C, c, y begin
    α ~ Tilde.Normal(0,1)
    cc ~ Tilde.Normal(0,1)^C
    for i in 1:N1
        v = α + cc[c[i]]
        y[i] ~ Soss.Bernoulli(logistic(v))
    end
end

hat_model = @Tilde.model N1, C, c, K, seed, y begin
    α ~ Tilde.Normal(0,sqrt(N1/K))
    cc ~ Tilde.Normal(0,sqrt(N1/K))^C
    sampler = Random.SamplerRangeNDL(1:N1)
    rng = ZZB.Rng(seed)
    for _ in 1:K
        i = rand(rng, sampler)
        v = α + cc[c[i]]
        y[i] ~ Soss.Bernoulli(logistic(v))
    end
end

It would be nice to tell Tilde to put a weight/factor on the log-likelihood, like this:

hat_model = @Tilde.model N1, C, c, K, seed, y begin
    α ~ Tilde.Normal(0,1)
    cc ~ Tilde.Normal(0,1)^C
    sampler = Random.SamplerRangeNDL(1:N1)
    rng = ZZB.Rng(seed)
    for _ in 1:K
        i = rand(rng, sampler)
        v = α + cc[c[i]]
        y[i] ~ Soss.Bernoulli(logistic(v), weight=N/K)
    end
end