sciml / scimlexpectations.jl

Fast uncertainty quantification for scientific machine learning (SciML) and differential equations

SciMLExpectations.jl: Expectated Values of Simulations and Uncertainty Quantification

This package is still under heavy construction. Use at your own risk!

SciMLExpectations.jl is a package for quantifying the uncertainties of simulations by calculating the expectations of observables with respect to input uncertainties. Its goal is to make it fast and easy to compute solution moments in a differentiable way in order to enable fast optimization under uncertainty.

Tutorials and Documentation

For information on using the package, see the stable documentation. Use the in-development documentation for the version of the documentation, which contains the unreleased features.

Example

using SciMLExpectations, OrdinaryDiffEq, Distributions, Cubature

function eom!(du, u, p, t, A)
    du .= A * u
end

u0 = [1.0, 1.0]
tspan = (0.0, 3.0)
p = [1.0; 2.0]
A = [0.0 1.0; -p[1] -p[2]]
prob = ODEProblem((du, u, p, t) -> eom!(du, u, p, t, A), u0, tspan, p)
u0s_dist = (Uniform(1, 10), truncated(Normal(3.0, 1), 0.0, 6.0))
gd = GenericDistribution(u0s_dist...)
cov(x, u, p) = x, p

sm = SystemMap(prob, Tsit5(), save_everystep = false)

analytical = (exp(A * tspan[end]) * [mean(d) for d in u0s_dist])
analytical

julia> analytical
2-element Vector{Float64}:
  1.5433991194037804
 -1.120209038276938

g(sol, p) = sol[:, end]
exprob = ExpectationProblem(sm, g, cov, gd)
sol = solve(exprob, Koopman(); quadalg = CubatureJLh(),
            ireltol = 1e-3, iabstol = 1e-3)
sol.u # Expectation of the states 1 and 2 at the final time point

2-element Vector{Float64}:
  1.5433860531082695
 -1.1201922503747408

Approximate error on the expectation

sol.resid #= 2-element Vector{Float64}: 7.193424502016654e-5 5.2074632876847327e-5 =#

scimlexpectations.jl's Issues

Feature request: extend to SteadyStateProblem

It would be great to allow SteadyStateProblems to be handled!

Expectation sensitivity Lotka Volterra

I can not get the Expectation interface (over sensitivity states) of the Lotka Volterra model to match up with my own naive implementation:

using DiffEqSensitivity
using ForwardDiff
using DiffEqUncertainty
using Distributions
using OrdinaryDiffEq
using LinearAlgebra
using Plots

function fiip(du,u,p,t)
    du[1] = dx = p[1]*u[1] - p[2]*u[1]*u[2]
    du[2] = dy = -p[3]*u[2] + p[4]*u[1]*u[2]
end
u0 = [1.0,1.0]
p = [1.5,1.0,3.0,1.0]
prob = ODEForwardSensitivityProblem(fiip,u0,(0.0,10.0),p,saveat=0:10)  
sol = solve(prob, Tsit5())
function g(sol)
    J = extract_local_sensitivities(sol,true)[2]
    det(J'*J)
end
g(sol) 
function g_manual(sol)
    det(sol[3:2:end,:]*sol[3:2:end,:]' + sol[4:2:end,:]*sol[4:2:end,:]')
end
g_manual(sol)

function fiip_forward_sensitivity(duₑₓₜ,uₑₓₜ,p,t)
    n_u = 2
    n_p = 4
    du = @view duₑₓₜ[1:n_u]
    u = @view uₑₓₜ[1:n_u]
    fiip(du,u,p,t)
    du_sen = @view duₑₓₜ[n_u+1:end]
    u_sen = @view uₑₓₜ[n_u+1:end]
    du_sen_mat = reshape(du_sen,n_u,n_p)
    u_sen_mat = reshape(u_sen,n_u,n_p)
    ForwardDiff.jacobian!(du_sen_mat, (du,p)->fiip(du,u,p,t), du, p)
    mul!(du_sen_mat,ForwardDiff.jacobian( (du,u)->fiip(du,u,p,t), du, u),u_sen_mat,1.0,1.0)
    return nothing
end
g_manual(solve(ODEProblem(fiip_forward_sensitivity,vcat(u0,zeros(length(p)*length(u0))),(0.0,10.0),p,saveat=0:10),Tsit5()))

u0_dist = [Uniform(0.9,1.1), 1.0]
p_dist = [1.5, truncated(Normal(1.5,.1),1.1, 1.9),3.0,1.0]
u0_dist_extended = vcat(u0_dist,zeros(length(p)*length(u0)))

test_batch = Float64[]
for i = 1:100_000
    u_test = [isa(ui,Distribution) ? rand(ui) : ui for ui in u0_dist_extended]
    p_test = [isa(pj,Distribution) ? rand(pj) : pj for pj in p_dist]
    prob_test = ODEProblem(fiip_forward_sensitivity,u_test,(0.0,10.0),p_test,saveat=0:10)  
    sol_test = solve(prob_test, Tsit5())
    push!(test_batch,g_manual(sol_test))
end
mean(test_batch) #3.5681520174354315e6

test_batch = Float64[]
for i = 1:100_000
    u_test = [isa(ui,Distribution) ? rand(ui) : ui for ui in u0_dist]
    p_test = [isa(pj,Distribution) ? rand(pj) : pj for pj in p_dist]
    prob_test = ODEForwardSensitivityProblem(fiip,u_test,(0.0,10.0),p_test,saveat=0:10)  
    sol_test = solve(prob_test, Tsit5())
    push!(test_batch,g(sol_test))
end
mean(test_batch) #3.5681520174354315e6
histogram(test_batch)

prob_func = function (prob, i, repeat)
    remake(prob, u0=[isa(ui,Distribution) ? rand(ui) : ui for ui in u0_dist_extended], p=[isa(pj,Distribution) ? rand(pj) : pj for pj in p_dist])
end
output_func = (sol, i) -> (g(sol), false)
monte_prob = EnsembleProblem(prob;output_func=output_func,prob_func=prob_func)

sol = solve(monte_prob,Tsit5(),EnsembleSerial(),trajectories=100_000)
mean(sol.u) #9.880744083958029e8
histogram(sol.u)

sol = solve(monte_prob,Tsit5(),EnsembleThreads(),trajectories=100_000)
mean(sol.u)#9.89137803472626e8

expectation(g, prob, u0_dist_extended, p_dist, MonteCarlo(), Tsit5(); trajectories =100_000)#9.872067067529106e8
expectation(g, prob, u0_dist_extended, p_dist, Koopman(), Tsit5())[1]  #9.884782355140978e8

Probably I made a mistake in my implementation.

Generalized initial uncertainty

Currently the interface supports uncertainty via elements of the initial state/parameters being "sampleable" (distributions). This is limiting in the sense that it assumes that the dimensions of the initial joint pdf are all independent.

I think the interface would benefit from a more general approach, possibly something similar to the prob_func arg from EnsembleProblem? However, instead of returning a new problem it returns a tuple holding an initial joint pdf draw (u0) and it's associated value f0(u0).

Requiring parameters to be AbstractVectors

I think here you are implicitly requiring u0 and p to be AbstractVectors. Hence you might aswell dispatch on that. E.g. this uses tuple parameters and throws an error:

expectation(x -> 1, 
            ODEProblem((a,b,c,d) -> nothing, [1.], (0., 1.), nothing),
            [1.,1.],
            (Uniform(-1,1), 0.), # tuple params
            Koopman(),
            Tsit5())

Wrong results for nout >1 , batch > 1

expectation() is returning the incorrect result when using nout > 1 and batch >1. nout=1 and batch > 1 works. As does nout > 1 and batch = 0

using OrdinaryDiffEq, DiffEqUncertainty, Quadrature, Cuba, Distributions

f(u,p,t) = p.*u
u0 = [10.0]
p = [-0.3]
tspan = (0.0,10.0)
prob = ODEProblem(f,u0,tspan,p)
sol = solve(prob, Tsit5())

g(sol) = [sol(4.0)[1], sol(6.0)[1]]

u0_dist = [truncated(Normal(3.0,2.0),-5,11)]

## Analytical
a = exp.(p[1]*[4.0,6.0])*mean(u0_dist[1])

## Non-batch
b = expectation(g, prob, u0_dist, p, Koopman(), Tsit5(); 
                quadalg = CubaSUAVE(), nout=2)

## batch
c = expectation(g, prob, u0_dist, p_dist, Koopman(), Tsit5(), EnsembleThreads(); 
            quadalg = CubaSUAVE(), nout=2, batch=1000)

Potential tolerance issues w/ AD

When performing AD the the magnitudes may be drastically different than the original observable of interest. Thus, the quadrature tolerances required may be quite different. I don't think there is a good way to hide this fact from the user. Need to brainstorm ideas. At a minimum we need to update the docs to reflect this.

Register

@JuliaRegistrator register()

Revive old tutorials

https://docs.sciml.ai/SciMLTutorialsOutput/stable/DiffEqUncertainty/01-expectation_introduction/
https://docs.sciml.ai/SciMLTutorialsOutput/stable/DiffEqUncertainty/02-AD_and_optimization/

These need to be updated to SciMLExpectations

centralmoment not working w/ MonteCarlo()

Numerically it doesn't make a ton of sense to solve the centralmoment problem with MonteCarlo() but it should work. Issue related to using sol.u vs sol[:] and the different solution type between MonteCarlo() and Koopman(). Related to #24

Does the Quadrature.jl/Integrals.jl dependency version spec need to be upgraded?

I got a strange situation with versions of DiffEqUncertainty and Quadrature packages (https://stackoverflow.com/questions/72794683/strange-behavior-of-diffequncertainty-jl-and-quadrature-jl). The guess for this situation is that the DiffEqUncertainty package declares its dependency as Quadrature major version 1, meanshile the current version of Quadrature pacakage is 2.1 already.

Does the Quadrature.jl/Integrals.jl dependency version spec need to be upgraded?

Expectation of an observed variable

I have a system made using ModelingToolkit that contains some observed variables. Can I extract the expectation of one of these observed variables in some way?

My two ideas on how to do it:

1. Pull out the relationship between the observed variable and the state vector and plug that code directly to the expectation function.
2. Force the observed variable into the state vector in ModelingToolkit after the structural_simplify step.

Is there a better way?

Discontinuous observable

This issue is a follow up to our slack discussion. To compute the size of the basin of attraction I am using a discontinuous observable (converged). With the default setting the quadrature does not converge. With reltol and abstol >0.08 it always finds the same (approximately correct) answer and for reltol and abstol <0.07 it doesn't converge at all. maxiters seems to have no effect.

using DiffEqUncertainty, OrdinaryDiffEq, Distributions

# Define a system [Menck, 2012]

function swing!(du,u,p,t)
    du[1] = u[2] # phase
    du[2] = -p[1]*u[2] + p[2] - p[3] * sin(u[1]) # frequency
end

tspan = (0.0,1000.0)
p = [0.1, 1, 8] # α, P, K
u_fix = [asin(p[2] / p[3]); 0.]
prob = ODEProblem(swing!, u_fix, tspan, p)


# Distribution of initial conditions

u0_dist = [Uniform(u_fix[1] - pi, u_fix[1] + pi), Uniform(-100, 100)]

# Observable for basin size

# An initial condition belongs to the basin if its frequency converges to 0
# Here we assume a trajectory is converged if its end point deviates from 0 by less than 0.1
converged(sol) = abs(sol[2,end]) > 0.1 ? 0 : 1

# MonteCarlo experiment to compute basin stability
@time expectation(converged, prob, u0_dist, p, MonteCarlo(), Tsit5(); trajectories = 1_000)  # ~1.5s
@time expectation(converged, prob, u0_dist, p, MonteCarlo(), Tsit5(), EnsembleSerial(); trajectories = 1_000) # ~3.5s

# [Gerlach, 2020] claims the Koopman algorithm is faster, but here it doesn't converge
expectation(converged, prob, u0_dist, p, Koopman(), Tsit5())

The discontinuous observable seems to be at the heart of the problem. When I replace it with a bump function the Koopman algorithm behaves reasonably.

function bump(sol)
    z = abs(sol[2,end])
    if z < .1
        return 1
    elseif z < .2
        return exp(-1 / (1 - (z-.1)^2))
    else
        return 0
    end
end

@time quad  = expectation(bump, prob, u0_dist, p, Koopman(), Tsit5(); ireltol = .05, iabstol = .05)

GenericDistribution fails with 4 arguments

GenericDistribution constructor method fails when exactly 4 distribution arguments are supplied.

MWE example below:

using SciMLExpectations, Test
import Distributions

n=8
p_array=[Distributions.Uniform(0,1) for i in (1:n)]

@testset "Generic distribution, variable length" begin
	for i in (1:n)
		gd=GenericDistribution(p_array[1:i]...)
		@test all(isa.(rand(gd), Number))
	end
end

Failure (for i==4 here) occurs at rand(gd) call, with exception:

MethodError: objects of type Distributions.Uniform{Float64} are not callable

expectation() should use p and u0 from ODEProblem

What is the rationale behind having to explicitly pass the initial conditions and parameters to expectation function if these can be embedded into the ODEProblem?

function expectation(g::Function, prob::ODEProblem, u0, p, expalg::Koopman, args...)

This makes the code a bit strange as I can have double definitions of initial conditions and one seems to be ignored.

f(u,p,t) = p.*u

u0 = [-666.666]
u0_dist = [truncated(Normal(3.0,2.0),3-1000,3+1000)]

p = [-0.3]
tspan = (0.0,10.0)
prob = ODEProblem(f,u0,tspan,p)

expectation(f -> f(1.0), prob, u0_dist, p, Koopman(), Tsit5())

Wouldn't this be enough or am I missing something?

prob = ODEProblem(f,u0_dist,tspan,p)
expectation(f -> f(1.0), prob, Koopman(), Tsit5())

Handling unstable systems

Unstable systems are causing extremely long quadrature run times. Seems like we can mitigate this with checking the status of the ODEProblem sol.

How should we handle the pull-back with unstable systems? Simply assign Inf to the pull back? Or assign Inf to all states in S(x) and then evaluate the observable.

covariance function

Extend the centralmoment methodology to include covariance

Potential issue with uncertainty quantification

I mentionned an issue I had using the library to assess the uncertainty of my DAE system solution on the forum :

https://discourse.julialang.org/t/problem-understanding-results-of-diffequncertainty/71193

I was encouraged to post an issue about it even though my code might be the problem and not the library.

The problem is basically that solving my DAE normally takes 9s

9.159245 seconds (21.20 M allocations: 1.964 GiB, 9.29% gc time, 0.10% compilation time)

whereas if I use the uncertainty library it takes way longer (for 5 trajectories) and a lot of allocations :

6588.658787 seconds (21.72 G allocations: 3.086 TiB, 52.83% gc time, 0.00% compilation time)

I am not pasting the whole discussion on the forum, to not clutter this post but the issue might be related to caching according to Chris Rackauckas

The code :

I add my code to this issue. The file interp2D.jl is used to defined interpolation functions on my main code model_MM_github.jl

I scraped the code as best as I coould, I believe it is fairly straightforward, if somehow something is not clear, do not hesitate to ask for more information.

code_github.zip

TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Reexport Quadrature?

Should we Reexport Quadrature? It is required to access the various quadalgs.

Feature request: Koopman expectation combined with collocation techniques

Is it possible to combine the Koopman expectation method with collocation techniques defined in DiffEqFlux for very efficient stochastic optimal control?

Extension to non-native ODE solvers?

Can this package be used for non-native solvers like that of Sundials, ODEInterface etc.?

Remove Distributions `extrema` type piracy

[ Info: Precompiling SciMLExpectations [afe9f18d-7609-4d0e-b7b7-af0cb72b8ea8]
WARNING: Method definition minimum(Distributions.AbstractMvNormal) in module Distributions at /root/.julia/packages/Distributions/39PV5/src/multivariate/mvnormal.jl:83 overwritten in module SciMLExpectations at /workspaces/SciMLExpectations.jl/src/SciMLExpectations.jl:23.
  ** incremental compilation may be fatally broken for this module **

WARNING: Method definition maximum(Distributions.AbstractMvNormal) in module Distributions at /root/.julia/packages/Distributions/39PV5/src/multivariate/mvnormal.jl:84 overwritten in module SciMLExpectations at /workspaces/SciMLExpectations.jl/src/SciMLExpectations.jl:24.
  ** incremental compilation may be fatally broken for this module **

WARNING: Method definition minimum(Distributions.Product{S, T, V} where V<:AbstractArray{T, 1} where T<:Distributions.Distribution{Distributions.ArrayLikeVariate{0}, S} where S<:Distributions.ValueSupport) in module Distributions at /root/.julia/packages/Distributions/39PV5/src/multivariate/product.jl:49 overwritten in module SciMLExpectations at /workspaces/SciMLExpectations.jl/src/SciMLExpectations.jl:27.
  ** incremental compilation may be fatally broken for this module **

WARNING: Method definition maximum(Distributions.Product{S, T, V} where V<:AbstractArray{T, 1} where T<:Distributions.Distribution{Distributions.ArrayLikeVariate{0}, S} where S<:Distributions.ValueSupport) in module Distributions at /root/.julia/packages/Distributions/39PV5/src/multivariate/product.jl:50 overwritten in module SciMLExpectations at /workspaces/SciMLExpectations.jl/src/SciMLExpectations.jl:28.
  ** incremental compilation may be fatally broken for this module **

No longer needed... upstreamed in JuliaStats/Distributions.jl#1319

centralmoment() vector valued support

Add support for computing central moments of vector valued observables.

Adding uncertainty to initial conditions and parameter value

Hey,

I am trying to put uncertainty in both initial condition and parameter value for the bouncing ball example. It is giving me errors.

Is it possible for you to show how that can be done on the existing code for Bouncing ball example?

Thanks in advance.

2.0 feature regressions

Transferring the documentation to 2.0 shows the following regressions

• - central moments
• - KDE as the source of uncertainty
• - batch integration, particularly with GPU

Trouble using Koopman.jl

I was running my code with Koopman.jl the other day and I got the following issue. Before when I ran my code, it was working perfectly fine.

MethodError: no method matching init(::IntegralProblem{false, Vector{Float64}, DiffEqUncertainty.var"#10#23"{DiffEqUncertainty.var"#20#33", DiffEqUncertainty.var"#21#34", typeof(loss), DiffEqUncertainty.var"#6#7"{Base.Iterators.Pairs{Symbol, CallbackSet{Tuple{ContinuousCallback{typeof(ground_condition), typeof(ground_affect!), typeof(ground_affect!), typeof(SciMLBase.INITIALIZE_DEFAULT), typeof(SciMLBase.FINALIZE_DEFAULT), Float64, Int64, Rational{Int64}, Nothing, Int64}, ContinuousCallback{typeof(stop_condition), typeof(terminate!), typeof(terminate!), typeof(SciMLBase.INITIALIZE_DEFAULT), typeof(SciMLBase.FINALIZE_DEFAULT), Float64, Int64, Rational{Int64}, Nothing, Int64}}, Tuple{}}, Tuple{Symbol}, NamedTuple{(:callback,), Tuple{CallbackSet{Tuple{ContinuousCallback{typeof(ground_condition), typeof(ground_affect!), typeof(ground_affect!), typeof(SciMLBase.INITIALIZE_DEFAULT), typeof(SciMLBase.FINALIZE_DEFAULT), Float64, Int64, Rational{Int64}, Nothing, Int64}, ContinuousCallback{typeof(stop_condition), typeof(terminate!), typeof(terminate!), typeof(SciMLBase.INITIALIZE_DEFAULT), typeof(SciMLBase.FINALIZE_DEFAULT), Float64, Int64, Rational{Int64}, Nothing, Int64}}, Tuple{}}}}}, ODEProblem{Vector{Float64}, Tuple{Float64, Float64}, true, Vector{Float64}, ODEFunction{true, typeof(ball!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, SciMLBase.StandardODEProblem}, Tuple{Tsit5{typeof(OrdinaryDiffEq.trivial_limiter!), typeof(OrdinaryDiffEq.trivial_limiter!), Static.False}}}, Vector{Float64}, Vector{Any}, Vector{Bool}, Int64}, Vector{Float64}, Vector{Float64}, Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}, ::HCubatureJL; reltol=1.0e-5, abstol=0.01, maxiters=1000000)
Closest candidates are:
init(::SciMLBase.AbstractDEProblem, ::Any...; kwargshandle, kwargs...) at C:\Users\Lakshay Arora.julia\packages\DiffEqBase\S7V8q\src\solve.jl:191

Stacktrace:
[1] solve(::IntegralProblem{false, Vector{Float64}, DiffEqUncertainty.var"#10#23"{DiffEqUncertainty.var"#20#33", DiffEqUncertainty.var"#21#34", typeof(loss), DiffEqUncertainty.var"#6#7"{Base.Iterators.Pairs{Symbol, CallbackSet{Tuple{ContinuousCallback{typeof(ground_condition), typeof(ground_affect!), typeof(ground_affect!), typeof(SciMLBase.INITIALIZE_DEFAULT), typeof(SciMLBase.FINALIZE_DEFAULT), Float64, Int64, Rational{Int64}, Nothing, Int64}, ContinuousCallback{typeof(stop_condition), typeof(terminate!), typeof(terminate!), typeof(SciMLBase.INITIALIZE_DEFAULT), typeof(SciMLBase.FINALIZE_DEFAULT), Float64, Int64, Rational{Int64}, Nothing, Int64}}, Tuple{}}, Tuple{Symbol}, NamedTuple{(:callback,), Tuple{CallbackSet{Tuple{ContinuousCallback{typeof(ground_condition), typeof(ground_affect!), typeof(ground_affect!), typeof(SciMLBase.INITIALIZE_DEFAULT), typeof(SciMLBase.FINALIZE_DEFAULT), Float64, Int64, Rational{Int64}, Nothing, Int64}, ContinuousCallback{typeof(stop_condition), typeof(terminate!), typeof(terminate!), typeof(SciMLBase.INITIALIZE_DEFAULT), typeof(SciMLBase.FINALIZE_DEFAULT), Float64, Int64, Rational{Int64}, Nothing, Int64}}, Tuple{}}}}}, ODEProblem{Vector{Float64}, Tuple{Float64, Float64}, true, Vector{Float64}, ODEFunction{true, typeof(ball!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, SciMLBase.StandardODEProblem}, Tuple{Tsit5{typeof(OrdinaryDiffEq.trivial_limiter!), typeof(OrdinaryDiffEq.trivial_limiter!), Static.False}}}, Vector{Float64}, Vector{Any}, Vector{Bool}, Int64}, Vector{Float64}, Vector{Float64}, Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}, ::Vararg{Any, N} where N; kwargs::Base.Iterators.Pairs{Symbol, Real, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:reltol, :abstol, :maxiters), Tuple{Float64, Float64, Int64}}})
@ CommonSolve C:\Users\Lakshay Arora.julia\packages\CommonSolve\TGRvG\src\CommonSolve.jl:23
[2] expectation(g::typeof(loss), prob::ODEProblem{Vector{Float64}, Tuple{Float64, Float64}, true, Vector{Float64}, ODEFunction{true, typeof(ball!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, SciMLBase.StandardODEProblem}, u0::Vector{Float64}, p::Vector{Any}, expalg::Koopman, args::Tsit5{typeof(OrdinaryDiffEq.trivial_limiter!), typeof(OrdinaryDiffEq.trivial_limiter!), Static.False}; u0_CoV::DiffEqUncertainty.var"#20#33", p_CoV::DiffEqUncertainty.var"#21#34", maxiters::Int64, batch::Int64, quadalg::HCubatureJL, ireltol::Float64, iabstol::Float64, nout::Int64, kwargs::Base.Iterators.Pairs{Symbol, CallbackSet{Tuple{ContinuousCallback{typeof(ground_condition), typeof(ground_affect!), typeof(ground_affect!), typeof(SciMLBase.INITIALIZE_DEFAULT), typeof(SciMLBase.FINALIZE_DEFAULT), Float64, Int64, Rational{Int64}, Nothing, Int64}, ContinuousCallback{typeof(stop_condition), typeof(terminate!), typeof(terminate!), typeof(SciMLBase.INITIALIZE_DEFAULT), typeof(SciMLBase.FINALIZE_DEFAULT), Float64, Int64, Rational{Int64}, Nothing, Int64}}, Tuple{}}, Tuple{Symbol}, NamedTuple{(:callback,), Tuple{CallbackSet{Tuple{ContinuousCallback{typeof(ground_condition), typeof(ground_affect!), typeof(ground_affect!), typeof(SciMLBase.INITIALIZE_DEFAULT), typeof(SciMLBase.FINALIZE_DEFAULT), Float64, Int64, Rational{Int64}, Nothing, Int64}, ContinuousCallback{typeof(stop_condition), typeof(terminate!), typeof(terminate!), typeof(SciMLBase.INITIALIZE_DEFAULT), typeof(SciMLBase.FINALIZE_DEFAULT), Float64, Int64, Rational{Int64}, Nothing, Int64}}, Tuple{}}}}})
@ DiffEqUncertainty C:\Users\Lakshay Arora.julia\packages\DiffEqUncertainty\03ndV\src\koopman.jl:104
[3] top-level scope
@ In[15]:4
[4] eval
@ .\boot.jl:360 [inlined]
[5] include_string(mapexpr::typeof(REPL.softscope), mod::Module, code::String, filename::String)
@ Base .\loading.jl:1116

I have been racking my brain with this issue. Please help.

Also, I am getting the same issue while running the example code as well. I have tried updating the packages as well.

Standardize return types

• expectation(Koopman()) returns a QuadratureSolution
• expectation(MonteCarlo()) returns a number/vector
• centralmoment returns a vector

Return DiffEq solution from expectation methods

When running koopman_expectation() and montecarlo_expectation() it would be nice to also return the ensemble solution or at least the initial conditions used by the integration scheme.

I often find myself wanting to plot the trajectories used.

Gradient of chance constraint in tutorial

Opening an issue after asking on the sciml-bridged slack channel.

In the tutorial " Optimization Under Uncertainty with DiffEqUncertainty.jl " a chance constraint is introduced, and it's gradient is evaluated by ForwardDiff, i.e. the code snippet below. However if I check the gradient with some input, say ForwardDiff.gradient(𝔼_constraint,[-1.0, 222.0, 50.0]) , I get a vector of zeros. Is the tutorial out of date or is there some other problem?

constraint_obs(sol) = sol[1,end] ≈ constraint[1] ? one(sol[1,end]) : zero(sol[1,end])
function 𝔼_constraint(θ)
    u0 = [θ[1],θ[2],θ[3], 0.0]
    expectation(constraint_obs, prob, u0, p_uncertain, Koopman(), Tsit5(),
                ireltol= 1e-9, iabstol = 1e-9,callback=constraint_cbs)[1]
end

function 𝔼_constraint_nlopt(x,∇)
    length(∇) > 0 ? ForwardDiff.gradient!(∇, 𝔼_constraint,x) : nothing
    𝔼_constraint(x) - 0.01
end

Improve docstrings

Such that makedocs warnonly = [:missing_docs] can be removed.

UndefVarError

Hello,

I'm starting out with the DiffEqUncertainty examples in https://github.com/SciML/SciMLTutorials.jl

Everything runs smoothly as long as I don't call expectation().
Both for MonteCarlo and Koopman, I get "UndefVarError: MonteCarlo/Koopman not defined".

The package is installed (I used the two lines here https://diffeq.sciml.ai/stable/analysis/uncertainty_quantification/)

Can you help me out? Thank you.

centralmoment and MonteCarlo?

Does centralmoment only work for Koopman and not MonteCarlo? For the example in the readme...

julia> centralmoment(2, g, prob, u0_dist, p_dist, MonteCarlo(), Tsit5(); trajectories = 100_000)
ERROR: type Array has no field u
Stacktrace:
 [1] getproperty(x::Vector{Float64}, f::Symbol)
   @ Base ./Base.jl:33
 [2] centralmoment(::Int64, ::typeof(g), ::ODEProblem{Vector{Float64}, Tuple{Float64, Float64}, true, Vector{Float64}, ODEFunction{true, typeof(f!), LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing}, Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, SciMLBase.StandardODEProblem}, ::Vararg{Any, N} where N; kwargs::Base.Iterators.Pairs{Symbol, Int64, Tuple{Symbol}, NamedTuple{(:trajectories,), Tuple{Int64}}})
   @ DiffEqUncertainty ~/.julia/packages/DiffEqUncertainty/IFrAT/src/koopman.jl:142
 [3] top-level scope
   @ none:1

What is the difference between `using IntegralCubature` and `using Integrals, Cubature` in the example?

Question❓

Slightly modified example from the readme

using SciMLExpectations, OrdinaryDiffEq, Distributions, Integrals, Cubature

function eom!(du, u, p, t, A)
    du .= A * u
end

u0 = [1.0, 1.0]
tspan = (0.0, 3.0)
p = [1.0; 2.0]
A = [0.0 1.0; -p[1] -p[2]]
prob = ODEProblem((du, u, p, t) -> eom!(du, u, p, t, A), u0, tspan, p)
u0s_dist = (Uniform(1, 10), truncated(Normal(3.0, 1), 0.0, 6.0))
gd = GenericDistribution(u0s_dist...)
cov(x, u, p) = x, p

sm = SystemMap(prob, Tsit5(), save_everystep = false)

analytical = (exp(A * tspan[end]) * [mean(d) for d in u0s_dist])
analytical

g(sol, p) = sol[:, end]
exprob = ExpectationProblem(sm, g, cov, gd; nout = length(u0))
sol = solve(exprob, Koopman(); quadalg = CubatureJLh(),
            ireltol = 1e-3, iabstol = 1e-3)
sol.u # Expectation of the states 1 and 2 at the final time point

fails with

ERROR: UndefVarError: `CubatureJLh` not defined
Stacktrace:
 [1] top-level scope
   @ ~/test/test.jl:25

Integrals.jl docs say

CubatureJLh: h-Cubature from Cubature.jl. Requires using Cubature.

So, shouldn't SciMLExpectations.jl work with using Integrals, Cubature too? Or is there significant difference in the scope of application of those packages?

Adding an additional Callback function

Hey,

I am trying to add one or two more callback functions for a 3D model. I am considering the stopping condition to be x and y are equal to zero or all x, y ,z are equal to zero. For some reason, I am not able to get results. Should it be like the following code?

condition_1(u, t, integrator) = u[1]
cb_1 = ContinuousCallback(condition_1, terminate!)

condition_2(u, t, integrator) = u[2] 
cb_2 = ContinuousCallback(condition_2, terminate!)

# condition_3(u, t, integrator) = u[3]
# cb_3 = ContinuousCallback(condition_3, terminate!)

cb_p = CallbackSet(cb_1, cb_2)
# cb_p = CallbackSet(cb_1, cb_2, cb_3)

Also, what if I want to make all 6 states equal to zero? For example, when x = 0 then x_dot = 0.

In anticipation of your reply.

Regards

Gradient by zygote of scalar expectation in batch mode

using ForwardDiff
using Zygote
using OrdinaryDiffEq
using DiffEqUncertainty
using DiffEqSensitivity
using Distributions
using Cubature

function f!(du,u,p,t)
    du[1] = p[1]*u[1] - p[2]*u[1]*u[2] #prey
    du[2] = -p[3]*u[2] + p[4]*u[1]*u[2] #predator
end

tspan = (0.0,10.0)
u0 = [1.0;1.0]
p = [1.5,1.0,3.0,1.0]
prob = ODEProblem(f!,u0,tspan,p,sensealg=InterpolatingAdjoint())
g(sol) = sol[1,end]

p1_3 = [1.5,3.0]
function testf!(p1_3)
    p_dist = [p1_3[1],1.0,p1_3[2],truncated(Normal(1.0,.1),.6, 1.4)]
    u0_dist = [1.0, Uniform(0.8, 1.1)]
    expectation(g, prob, u0_dist, p_dist, Koopman(), Tsit5(), quadalg=CubatureJLp(),batch=32)[1]
end
testf!(p1_3)
ForwardDiff.gradient(testf!,p1_3)
Zygote.gradient(testf!,p1_3)

automated sampling importance resampling

https://www.ncbi.nlm.nih.gov/pubmed/28887735

Problems evaluating Koopman Expectations over large parameter space

Hi, first time using the package so I apologise if I'm missing something obvious.

I'm trying to propagate parametric uncertainty through a large, stiff system of ODEs representing a chemical reaction network (previous discussion here) using Koopman Expectations, such that I have a normal distribution over each of my ODESystem parameters and I am able to observe the mean and standard deviation over my state vector at every timestep specified by saveat. Here is my setup:

# `rates` represents reaction rate constants, the parameters of the model.
rates, rate_uncertainties = calc_rate_constants(...)
p_dist = [truncated(Normal(r, ru), r-(4*u), r+(4*u)) for (r, u) in zip(rates, rate_uncertainties)]

# `ODEProblem` constructed via Catalyst.jl, shouldn't be relevant.
oprob = ODEProblem(...)

# Want mean and variance for all saved timesteps.
g(sol) = [sol[1,:]; sol[1,:].^2]
gd = GenericDistribution(p_dist...)
# I believe this is the correct expression for `h(x, u, p)` when only applying uncertainty to parameters?
h(x, u, p) = u, x
sm = SystemMap(oprob, Rosenbrock23(); abstol=1e-14, reltol=1e-14, dtmin=1e-16, saveat=0.0:1e6:2e7)
exprob = ExpectationProblem(sm, g, h, gd; nout=2)
sol = solve(exprob, Koopman())

As far as I can tell this is set up correctly, but please correct me if not.

When I run this, it gets into the solve() line and then errors with the following:

ERROR: LoadError: OutOfMemoryError()
Stacktrace:
  [1] Array
    @ ./boot.jl:459 [inlined]
  [2] signcombos(k::Int64, λ::Float64, #unused#::Val{36})
    @ HCubature ~/.julia/packages/HCubature/QvyJW/src/genz-malik.jl:34
  [3] HCubature.GenzMalik(v::Val{36}, ::Type{Float64})
    @ HCubature ~/.julia/packages/HCubature/QvyJW/src/genz-malik.jl:99
  [4] cubrule
    @ ~/.julia/packages/HCubature/QvyJW/src/HCubature.jl:37 [inlined]
  [5] hcubature_(f::Integrals.var"#19#21"{IntegralProblem{false, RecursiveArrayTools.ArrayPartition{Float64, Tuple{Vector{Float64}, Vector{Float64}}}, SciMLExpectations.var"#14#15"{SciMLExpectations.GenericDistribution{SciMLExpectations.var"#pdf_func#4"{NTuple{36, Distributions.Truncated{Distributions.Normal{Float64}, Distributions.Continuous, Float64}}}, SciMLExpectations.var"#rand_func#6"{NTuple{36, Distributions.Truncated{Distributions.Normal{Float64}, Distributions.Continuous, Float64}}}, StaticArraysCore.SVector{36, Float64}, StaticArraysCore.SVector{36, Float64}}, AutoBreakdown.var"#h#45", AutoBreakdown.var"#g#44", SciMLExpectations.SystemMap{ODEProblem{Vector{Float64}, Tuple{Float64, Float64}, true, Vector{Float64}, ODEFunction{true, SciMLBase.AutoSpecialize, ModelingToolkit.var"#f#487"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:ˍ₋arg1, :ˍ₋arg2, :t), ModelingToolkit.var"#_RGF_ModTag", ModelingToolkit.var"#_RGF_ModTag", (0x11954d43, 0xfeb7cb16, 0x36117848, 0xb62dc5d6, 0x86beea97)}, RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:ˍ₋out, :ˍ₋arg1, :ˍ₋arg2, :t), ModelingToolkit.var"#_RGF_ModTag", ModelingToolkit.var"#_RGF_ModTag", (0xf7d03d71, 0x4a56f8e0, 0x6b4c3b18, 0xa7e9f6b2, 0x184e3749)}}, UniformScaling{Bool}, Nothing, Nothing, ModelingToolkit.var"#_jac#491"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:ˍ₋arg1, :ˍ₋arg2, :t), ModelingToolkit.var"#_RGF_ModTag", ModelingToolkit.var"#_RGF_ModTag", (0xeb9be0d9, 0x805797f6, 0x35efa81e, 0xf7ba3bad, 0x26a49a9b)}, RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:ˍ₋out, :ˍ₋arg1, :ˍ₋arg2, :t), ModelingToolkit.var"#_RGF_ModTag", ModelingToolkit.var"#_RGF_ModTag", (0x6e53ff16, 0x22f787c8, 0xc15faf6f, 0x2bf5f454, 0x2964e826)}}, Nothing, Nothing, SparseArrays.SparseMatrixCSC{Float64, Int64}, Nothing, Nothing, Nothing, Nothing, Vector{Symbol}, Symbol, Vector{Symbol}, ModelingToolkit.var"#503#generated_observed#495"{Bool, ModelingToolkit.ODESystem, Dict{Any, Any}}, Nothing, ModelingToolkit.ODESystem}, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, SciMLBase.StandardODEProblem}, Tuple{Rosenbrock23{0, true, Nothing, typeof(OrdinaryDiffEq.DEFAULT_PRECS), Val{:forward}, true, nothing}}, Base.Pairs{Symbol, Any, NTuple{7, Symbol}, NamedTuple{(:saveat, :reltol, :progress_steps, :dtmin, :abstol, :progress, :kwargshandle), Tuple{Vector{Float64}, Float64, Int64, Float64, Float64, Bool, DataType}}}}}, StaticArraysCore.SVector{36, Float64}, StaticArraysCore.SVector{36, Float64}, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}}, a::StaticArraysCore.SVector{36, Float64}, b::StaticArraysCore.SVector{36, Float64}, norm::typeof(norm), rtol_::Float64, atol::Float64, maxevals::Int64, initdiv::Int64)
    @ HCubature ~/.julia/packages/HCubature/QvyJW/src/HCubature.jl:56
  [6] #hcubature#3
    @ ~/.julia/packages/HCubature/QvyJW/src/HCubature.jl:179 [inlined]
  [7] __solvebp_call(::IntegralProblem{false, RecursiveArrayTools.ArrayPartition{Float64, Tuple{Vector{Float64}, Vector{Float64}}}, SciMLExpectations.var"#14#15"{SciMLExpectations.GenericDistribution{SciMLExpectations.var"#pdf_func#4"{NTuple{36, Distributions.Truncated{Distributions.Normal{Float64}, Distributions.Continuous, Float64}}}, SciMLExpectations.var"#rand_func#6"{NTuple{36, Distributions.Truncated{Distributions.Normal{Float64}, Distributions.Continuous, Float64}}}, StaticArraysCore.SVector{36, Float64}, StaticArraysCore.SVector{36, Float64}}, AutoBreakdown.var"#h#45", AutoBreakdown.var"#g#44", SciMLExpectations.SystemMap{ODEProblem{Vector{Float64}, Tuple{Float64, Float64}, true, Vector{Float64}, ODEFunction{true, SciMLBase.AutoSpecialize, ModelingToolkit.var"#f#487"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:ˍ₋arg1, :ˍ₋arg2, :t), ModelingToolkit.var"#_RGF_ModTag", ModelingToolkit.var"#_RGF_ModTag", (0x11954d43, 0xfeb7cb16, 0x36117848, 0xb62dc5d6, 0x86beea97)}, RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:ˍ₋out, :ˍ₋arg1, :ˍ₋arg2, :t), ModelingToolkit.var"#_RGF_ModTag", ModelingToolkit.var"#_RGF_ModTag", (0xf7d03d71, 0x4a56f8e0, 0x6b4c3b18, 0xa7e9f6b2, 0x184e3749)}}, UniformScaling{Bool}, Nothing, Nothing, ModelingToolkit.var"#_jac#491"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:ˍ₋arg1, :ˍ₋arg2, :t), ModelingToolkit.var"#_RGF_ModTag", ModelingToolkit.var"#_RGF_ModTag", (0xeb9be0d9, 0x805797f6, 0x35efa81e, 0xf7ba3bad, 0x26a49a9b)}, RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:ˍ₋out, :ˍ₋arg1, :ˍ₋arg2, :t), ModelingToolkit.var"#_RGF_ModTag", ModelingToolkit.var"#_RGF_ModTag", (0x6e53ff16, 0x22f787c8, 0xc15faf6f, 0x2bf5f454, 0x2964e826)}}, Nothing, Nothing, SparseArrays.SparseMatrixCSC{Float64, Int64}, Nothing, Nothing, Nothing, Nothing, Vector{Symbol}, Symbol, Vector{Symbol}, ModelingToolkit.var"#503#generated_observed#495"{Bool, ModelingToolkit.ODESystem, Dict{Any, Any}}, Nothing, ModelingToolkit.ODESystem}, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, SciMLBase.StandardODEProblem}, Tuple{Rosenbrock23{0, true, Nothing, typeof(OrdinaryDiffEq.DEFAULT_PRECS), Val{:forward}, true, nothing}}, Base.Pairs{Symbol, Any, NTuple{7, Symbol}, NamedTuple{(:saveat, :reltol, :progress_steps, :dtmin, :abstol, :progress, :kwargshandle), Tuple{Vector{Float64}, Float64, Int64, Float64, Float64, Bool, DataType}}}}}, StaticArraysCore.SVector{36, Float64}, StaticArraysCore.SVector{36, Float64}, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}, ::Integrals.HCubatureJL, ::Integrals.ReCallVJP{Integrals.ZygoteVJP}, ::StaticArraysCore.SVector{36, Float64}, ::StaticArraysCore.SVector{36, Float64}, ::RecursiveArrayTools.ArrayPartition{Float64, Tuple{Vector{Float64}, Vector{Float64}}}; reltol::Float64, abstol::Float64, maxiters::Int64, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Integrals ~/.julia/packages/Integrals/KYdjn/src/Integrals.jl:260
  [8] #__solvebp#13
    @ ~/.julia/packages/Integrals/KYdjn/src/Integrals.jl:221 [inlined]
  [9] #solve#12
    @ ~/.julia/packages/Integrals/KYdjn/src/Integrals.jl:217 [inlined]
 [10] integrate(quadalg::Integrals.HCubatureJL, adalg::SciMLExpectations.NonfusedAD, f::Function, lb::StaticArraysCore.SVector{36, Float64}, ub::StaticArraysCore.SVector{36, Float64}, p::RecursiveArrayTools.ArrayPartition{Float64, Tuple{Vector{Float64}, Vector{Float64}}}; nout::Int64, batch::Int64, kwargs::Base.Pairs{Symbol, Real, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:reltol, :abstol, :maxiters), Tuple{Float64, Float64, Int64}}})
    @ SciMLExpectations ~/.julia/packages/SciMLExpectations/DWytJ/src/expectation.jl:148
 [11] solve(::SciMLExpectations.ExpectationProblem{SciMLExpectations.SystemMap{ODEProblem{Vector{Float64}, Tuple{Float64, Float64}, true, Vector{Float64}, ODEFunction{true, SciMLBase.AutoSpecialize, ModelingToolkit.var"#f#487"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:ˍ₋arg1, :ˍ₋arg2, :t), ModelingToolkit.var"#_RGF_ModTag", ModelingToolkit.var"#_RGF_ModTag", (0x11954d43, 0xfeb7cb16, 0x36117848, 0xb62dc5d6, 0x86beea97)}, RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:ˍ₋out, :ˍ₋arg1, :ˍ₋arg2, :t), ModelingToolkit.var"#_RGF_ModTag", ModelingToolkit.var"#_RGF_ModTag", (0xf7d03d71, 0x4a56f8e0, 0x6b4c3b18, 0xa7e9f6b2, 0x184e3749)}}, UniformScaling{Bool}, Nothing, Nothing, ModelingToolkit.var"#_jac#491"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:ˍ₋arg1, :ˍ₋arg2, :t), ModelingToolkit.var"#_RGF_ModTag", ModelingToolkit.var"#_RGF_ModTag", (0xeb9be0d9, 0x805797f6, 0x35efa81e, 0xf7ba3bad, 0x26a49a9b)}, RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:ˍ₋out, :ˍ₋arg1, :ˍ₋arg2, :t), ModelingToolkit.var"#_RGF_ModTag", ModelingToolkit.var"#_RGF_ModTag", (0x6e53ff16, 0x22f787c8, 0xc15faf6f, 0x2bf5f454, 0x2964e826)}}, Nothing, Nothing, SparseArrays.SparseMatrixCSC{Float64, Int64}, Nothing, Nothing, Nothing, Nothing, Vector{Symbol}, Symbol, Vector{Symbol}, ModelingToolkit.var"#503#generated_observed#495"{Bool, ModelingToolkit.ODESystem, Dict{Any, Any}}, Nothing, ModelingToolkit.ODESystem}, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, SciMLBase.StandardODEProblem}, Tuple{Rosenbrock23{0, true, Nothing, typeof(OrdinaryDiffEq.DEFAULT_PRECS), Val{:forward}, true, nothing}}, Base.Pairs{Symbol, Any, NTuple{7, Symbol}, NamedTuple{(:saveat, :reltol, :progress_steps, :dtmin, :abstol, :progress, :kwargshandle), Tuple{Vector{Float64}, Float64, Int64, Float64, Float64, Bool, DataType}}}}, AutoBreakdown.var"#g#44", AutoBreakdown.var"#h#45", SciMLExpectations.GenericDistribution{SciMLExpectations.var"#pdf_func#4"{NTuple{36, Distributions.Truncated{Distributions.Normal{Float64}, Distributions.Continuous, Float64}}}, SciMLExpectations.var"#rand_func#6"{NTuple{36, Distributions.Truncated{Distributions.Normal{Float64}, Distributions.Continuous, Float64}}}, StaticArraysCore.SVector{36, Float64}, StaticArraysCore.SVector{36, Float64}}, RecursiveArrayTools.ArrayPartition{Float64, Tuple{Vector{Float64}, Vector{Float64}}}}, ::SciMLExpectations.Koopman{SciMLExpectations.NonfusedAD}; maxiters::Int64, batch::Int64, quadalg::Integrals.HCubatureJL, ireltol::Float64, iabstol::Float64, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ SciMLExpectations ~/.julia/packages/SciMLExpectations/DWytJ/src/expectation.jl:134
 [12] solve(::SciMLExpectations.ExpectationProblem{SciMLExpectations.SystemMap{ODEProblem{Vector{Float64}, Tuple{Float64, Float64}, true, Vector{Float64}, ODEFunction{true, SciMLBase.AutoSpecialize, ModelingToolkit.var"#f#487"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:ˍ₋arg1, :ˍ₋arg2, :t), ModelingToolkit.var"#_RGF_ModTag", ModelingToolkit.var"#_RGF_ModTag", (0x11954d43, 0xfeb7cb16, 0x36117848, 0xb62dc5d6, 0x86beea97)}, RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:ˍ₋out, :ˍ₋arg1, :ˍ₋arg2, :t), ModelingToolkit.var"#_RGF_ModTag", ModelingToolkit.var"#_RGF_ModTag", (0xf7d03d71, 0x4a56f8e0, 0x6b4c3b18, 0xa7e9f6b2, 0x184e3749)}}, UniformScaling{Bool}, Nothing, Nothing, ModelingToolkit.var"#_jac#491"{RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:ˍ₋arg1, :ˍ₋arg2, :t), ModelingToolkit.var"#_RGF_ModTag", ModelingToolkit.var"#_RGF_ModTag", (0xeb9be0d9, 0x805797f6, 0x35efa81e, 0xf7ba3bad, 0x26a49a9b)}, RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:ˍ₋out, :ˍ₋arg1, :ˍ₋arg2, :t), ModelingToolkit.var"#_RGF_ModTag", ModelingToolkit.var"#_RGF_ModTag", (0x6e53ff16, 0x22f787c8, 0xc15faf6f, 0x2bf5f454, 0x2964e826)}}, Nothing, Nothing, SparseArrays.SparseMatrixCSC{Float64, Int64}, Nothing, Nothing, Nothing, Nothing, Vector{Symbol}, Symbol, Vector{Symbol}, ModelingToolkit.var"#503#generated_observed#495"{Bool, ModelingToolkit.ODESystem, Dict{Any, Any}}, Nothing, ModelingToolkit.ODESystem}, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, SciMLBase.StandardODEProblem}, Tuple{Rosenbrock23{0, true, Nothing, typeof(OrdinaryDiffEq.DEFAULT_PRECS), Val{:forward}, true, nothing}}, Base.Pairs{Symbol, Any, NTuple{7, Symbol}, NamedTuple{(:saveat, :reltol, :progress_steps, :dtmin, :abstol, :progress, :kwargshandle), Tuple{Vector{Float64}, Float64, Int64, Float64, Float64, Bool, DataType}}}}, AutoBreakdown.var"#g#44", AutoBreakdown.var"#h#45", SciMLExpectations.GenericDistribution{SciMLExpectations.var"#pdf_func#4"{NTuple{36, Distributions.Truncated{Distributions.Normal{Float64}, Distributions.Continuous, Float64}}}, SciMLExpectations.var"#rand_func#6"{NTuple{36, Distributions.Truncated{Distributions.Normal{Float64}, Distributions.Continuous, Float64}}}, StaticArraysCore.SVector{36, Float64}, StaticArraysCore.SVector{36, Float64}}, RecursiveArrayTools.ArrayPartition{Float64, Tuple{Vector{Float64}, Vector{Float64}}}}, ::SciMLExpectations.Koopman{SciMLExpectations.NonfusedAD})
    @ SciMLExpectations ~/.julia/packages/SciMLExpectations/DWytJ/src/expectation.jl:125

Watching my system memory usage at the time indicates that I really shouldn't be running out of memory (~7 GB / 32 GB used when it errors), so I'm really not sure what to make of the issue. I'd like this to work on systems of >2000 ODES with > 7000 parameters, but right now this crash is happening on a much smaller system of 20 ODEs with only 36 parameters (hence the Rosenbrock23 solver, I usually use QNDF).

possible test failure in upcoming Julia version 1.5

A PkgEval run for a Julia pull request which changes the generated numbers for rand(a:b) indicates that the tests of this package might fail in Julia 1.5 (and on Julia current master branch).

Also, you might be interested in using the new StableRNGs.jl registered package, which provides guaranteed stable streams of random numbers across Julia releases.

Apologies if this is a false positive. Cf. https://github.com/JuliaCI/NanosoldierReports/blob/ab6676206b210325500b4f4619fa711f2d7429d2/pkgeval/by_hash/52c2272_vs_47c55db/logs/DiffEqUncertainty/1.5.0-DEV-87d2a04de3.log

Noise Scaling in ProbInts

I am not sure if the noise scaling in ProbInts is correct. The dt scaling is asymptotically correct, I know I've checked that. However, I am not sure where I should be putting sigma. I think it's in the wrong spot because my results differ from the paper for how large it has to be in order to see significant effects.

@finmod

Zygote return Any[] for expectation()

using OrdinaryDiffEq, Distributions, DiffEqUncertainty,
        ForwardDiff, Zygote, FiniteDiff, DiffEqSensitivity

function simplependulum(du,u,p,t)
    g, L = p
    du[1] = u[2]
    du[2] = -(g/L)*sin(u[1])
end

u0 = [π/2, 0.0]
p = [9.807, 1.0]
tspan = (0.0,5.0)
prob = ODEProblem(simplependulum, u0, tspan, p)

u0_dist = [Uniform(π/2.1, π/1.9), 0.0]
p_dist = [9.807, truncated(Normal(1.0,.1), 0.6, 1.4)]

g(sol) = sol[1,end]

# ∂𝔼/∂u0
test_u0(q) = expectation(g, prob, q, p_dist, Koopman(), Tsit5())[1]
@time df1 = ForwardDiff.gradient(test_u0,u0) #Float64[]
@time df2 = FiniteDiff.finite_difference_gradient(test_u0,u0) #Float64[]
@time df3 = Zygote.gradient(test_u0,u0)    #any[]

# ∂𝔼/∂p
test_p(q) = expectation(g, prob, u0_dist, q, Koopman(), Tsit5())[1]
@time df1 = ForwardDiff.gradient(test_p,p) #Float64[]
@time df2 = FiniteDiff.finite_difference_gradient(test_p,p) #Float64[]
@time df3 = Zygote.gradient(test_p,p) @any[]

However, the following properly returns Float64[] for some reason

# mixed u0/p
function test_mixed(q) 
    _u = [u0_dist[1], q[1]]
    _p = [q[2], p_dist[1]]
    expectation(g, prob, _u, _p, Koopman(), Tsit5())[1]
end

x0 = [u0[2], p[1]]
@time df1 = ForwardDiff.gradient(test_mixed,x0) #Float64[]
@time df2 = FiniteDiff.finite_difference_gradient(test_mixed,x0) #Float64[]
@time df3 = Zygote.gradient(test_mixed,x0) #Float64[]

Error using Koopman package

Hey,

I am using the SciMLExpectations package, while my MonteCarlo() function is working perfectly but my Koopman() is giving me this issue

MethodError: no method matching logpdf(::Int64, ::Float64)

Some of the types have been truncated in the stacktrace for improved reading. To emit complete information
in the stack trace, evaluate TruncatedStacktraces.VERBOSE[] = true and re-run the code.

Closest candidates are:
logpdf(::UnivariateMixture, ::Real) at C:\Users\lakshay.julia\packages\Distributions\YQQXX\src\mixtures\mixturemodel.jl:363
logpdf(::DiscreteNonParametric, ::Real) at C:\Users\lakshay.julia\packages\Distributions\YQQXX\src\univariate\discrete\discretenonparametric.jl:110
logpdf(::Chi, ::Real) at C:\Users\lakshay.julia\packages\Distributions\YQQXX\src\univariate\continuous\chi.jl:92
...

Stacktrace:
[1] (::SciMLExpectations.var"#2#5")(::Tuple{Int64, Float64})
@ SciMLExpectations .\none:0
[2] MappingRF
@ .\reduce.jl:95 [inlined]
[3] foldl_impl(op::Base.MappingRF{SciMLExpectations.var"#2#5", Base.BottomRF{typeof(Base.add_sum)}}, init::Base.InitialValue, itr::Base.Iterators.Zip{Tuple{Tuple{Normal{Float64}, Normal{Float64}, Normal{Float64}, Int64, Int64, Int64}, StaticArraysCore.MVector{6, Float64}}})
@ Base .\reduce.jl:62
[4] foldl_impl
@ .\reduce.jl:48 [inlined]
[5] mapfoldl_impl
@ .\reduce.jl:44 [inlined]
[6] #mapfoldl#259
@ .\reduce.jl:170 [inlined]
[7] mapfoldl
@ .\reduce.jl:170 [inlined]
[8] #mapreduce#263
@ .\reduce.jl:302 [inlined]
[9] mapreduce
@ .\reduce.jl:302 [inlined]
[10] #sum#266
@ .\reduce.jl:528 [inlined]
[11] sum
@ .\reduce.jl:528 [inlined]
[12] #sum#267
@ .\reduce.jl:557 [inlined]
[13] sum
@ .\reduce.jl:557 [inlined]
[14] pdf_func
@ C:\Users\lakshay.julia\packages\SciMLExpectations\DWytJ\src\distribution_utils.jl:17 [inlined]
[15] pdf
@ C:\Users\lakshay.julia\packages\SciMLExpectations\DWytJ\src\distribution_utils.jl:25 [inlined]
[16] (::SciMLExpectations.var"#14#15"{GenericDistribution{SciMLExpectations.var"#pdf_func#4"{Tuple{Normal{Float64}, Normal{Float64}, Normal{Float64}, Int64, Int64, Int64}}, SciMLExpectations.var"#rand_func#6"{Tuple{Normal{Float64}, Normal{Float64}, Normal{Float64}, Int64, Int64, Int64}}, StaticArraysCore.SVector{6, Float64}, StaticArraysCore.SVector{6, Float64}}, typeof(h), typeof(loss), SystemMap{ODEProblem{true,Vector{Float64},Tuple{Float64, Float64},…}, Tuple{Tsit5{Static.False,…}}, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}})(x::StaticArraysCore.MVector{6, Float64}, p::ArrayPartition{Float64, Tuple{Vector{Float64}, Vector{Float64}}})
@ SciMLExpectations C:\Users\lakshay.julia\packages\SciMLExpectations\DWytJ\src\expectation.jl:48
[17] substitute_f_vector(t::StaticArraysCore.SVector{6, Float64}, p::ArrayPartition{Float64, Tuple{Vector{Float64}, Vector{Float64}}}, f::SciMLExpectations.var"#14#15"{GenericDistribution{SciMLExpectations.var"#pdf_func#4"{Tuple{Normal{Float64}, Normal{Float64}, Normal{Float64}, Int64, Int64, Int64}}, SciMLExpectations.var"#rand_func#6"{Tuple{Normal{Float64}, Normal{Float64}, Normal{Float64}, Int64, Int64, Int64}}, StaticArraysCore.SVector{6, Float64}, StaticArraysCore.SVector{6, Float64}}, typeof(h), typeof(loss), SystemMap{ODEProblem{true,Vector{Float64},Tuple{Float64, Float64},…}, Tuple{Tsit5{Static.False,…}}, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}}, lb::StaticArraysCore.SVector{6, Float64}, ub::StaticArraysCore.SVector{6, Float64})
@ Integrals C:\Users\lakshay.julia\packages\Integrals\9qNWp\src\infinity_handling.jl:50
[18] #24
@ C:\Users\lakshay.julia\packages\Integrals\9qNWp\src\infinity_handling.jl:64 [inlined]
[19] #41
@ C:\Users\lakshay.julia\packages\Integrals\9qNWp\src\Integrals.jl:149 [inlined]
[20] (::HCubature.GenzMalik{6, Float64})(f::Integrals.var"#41#43"{IntegralProblem{false, ArrayPartition{Float64, Tuple{Vector{Float64}, Vector{Float64}}}, Integrals.var"#24#26"{SciMLExpectations.var"#14#15"{GenericDistribution{SciMLExpectations.var"#pdf_func#4"{Tuple{Normal{Float64}, Normal{Float64}, Normal{Float64}, Int64, Int64, Int64}}, SciMLExpectations.var"#rand_func#6"{Tuple{Normal{Float64}, Normal{Float64}, Normal{Float64}, Int64, Int64, Int64}}, StaticArraysCore.SVector{6, Float64}, StaticArraysCore.SVector{6, Float64}}, typeof(h), typeof(loss), SystemMap{ODEProblem{true,Vector{Float64},Tuple{Float64, Float64},…}, Tuple{Tsit5{Static.False,…}}, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}}, StaticArraysCore.SVector{6, Float64}, StaticArraysCore.SVector{6, Float64}}, StaticArraysCore.SVector{6, Float64}, StaticArraysCore.SVector{6, Float64}, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}}, a::StaticArraysCore.SVector{6, Float64}, b::StaticArraysCore.SVector{6, Float64}, norm::typeof(LinearAlgebra.norm))
@ HCubature C:\Users\lakshay.julia\packages\HCubature\QvyJW\src\genz-malik.jl:121
[21] hcubature(f::Integrals.var"#41#43"{IntegralProblem{false, ArrayPartition{Float64, Tuple{Vector{Float64}, Vector{Float64}}}, Integrals.var"#24#26"{SciMLExpectations.var"#14#15"{GenericDistribution{SciMLExpectations.var"#pdf_func#4"{Tuple{Normal{Float64}, Normal{Float64}, Normal{Float64}, Int64, Int64, Int64}}, SciMLExpectations.var"#rand_func#6"{Tuple{Normal{Float64}, Normal{Float64}, Normal{Float64}, Int64, Int64, Int64}}, StaticArraysCore.SVector{6, Float64}, StaticArraysCore.SVector{6, Float64}}, typeof(h), typeof(loss), SystemMap{ODEProblem{true,Vector{Float64},Tuple{Float64, Float64},…}, Tuple{Tsit5{Static.False,…}}, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}}, StaticArraysCore.SVector{6, Float64}, StaticArraysCore.SVector{6, Float64}}, StaticArraysCore.SVector{6, Float64}, StaticArraysCore.SVector{6, Float64}, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}}, a::StaticArraysCore.SVector{6, Float64}, b::StaticArraysCore.SVector{6, Float64}, norm::typeof(LinearAlgebra.norm), rtol::Float64, atol::Float64, maxevals::Int64, initdiv::Int64)
@ HCubature C:\Users\lakshay.julia\packages\HCubature\QvyJW\src\HCubature.jl:61
[22] #hcubature#3
@ C:\Users\lakshay.julia\packages\HCubature\QvyJW\src\HCubature.jl:179 [inlined]
[23] __solvebp_call(prob::IntegralProblem{false, ArrayPartition{Float64, Tuple{Vector{Float64}, Vector{Float64}}}, Integrals.var"#24#26"{SciMLExpectations.var"#14#15"{GenericDistribution{SciMLExpectations.var"#pdf_func#4"{Tuple{Normal{Float64}, Normal{Float64}, Normal{Float64}, Int64, Int64, Int64}}, SciMLExpectations.var"#rand_func#6"{Tuple{Normal{Float64}, Normal{Float64}, Normal{Float64}, Int64, Int64, Int64}}, StaticArraysCore.SVector{6, Float64}, StaticArraysCore.SVector{6, Float64}}, typeof(h), typeof(loss), SystemMap{ODEProblem{true,Vector{Float64},Tuple{Float64, Float64},…}, Tuple{Tsit5{Static.False,…}}, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}}, StaticArraysCore.SVector{6, Float64}, StaticArraysCore.SVector{6, Float64}}, StaticArraysCore.SVector{6, Float64}, StaticArraysCore.SVector{6, Float64}, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}, alg::HCubatureJL{typeof(LinearAlgebra.norm)}, sensealg::Integrals.ReCallVJP{Integrals.ZygoteVJP}, lb::StaticArraysCore.SVector{6, Float64}, ub::StaticArraysCore.SVector{6, Float64}, p::ArrayPartition{Float64, Tuple{Vector{Float64}, Vector{Float64}}}; reltol::Float64, abstol::Float64, maxiters::Int64)
@ Integrals C:\Users\lakshay.julia\packages\Integrals\9qNWp\src\Integrals.jl:158
[24] #__solvebp#35
@ C:\Users\lakshay.julia\packages\Integrals\9qNWp\src\Integrals.jl:123 [inlined]
[25] solve(prob::IntegralProblem{false, ArrayPartition{Float64, Tuple{Vector{Float64}, Vector{Float64}}}, SciMLExpectations.var"#14#15"{GenericDistribution{SciMLExpectations.var"#pdf_func#4"{Tuple{Normal{Float64}, Normal{Float64}, Normal{Float64}, Int64, Int64, Int64}}, SciMLExpectations.var"#rand_func#6"{Tuple{Normal{Float64}, Normal{Float64}, Normal{Float64}, Int64, Int64, Int64}}, StaticArraysCore.SVector{6, Float64}, StaticArraysCore.SVector{6, Float64}}, typeof(h), typeof(loss), SystemMap{ODEProblem{true,Vector{Float64},Tuple{Float64, Float64},…}, Tuple{Tsit5{Static.False,…}}, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}}, StaticArraysCore.SVector{6, Float64}, StaticArraysCore.SVector{6, Float64}, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}, alg::HCubatureJL{typeof(LinearAlgebra.norm)}; sensealg::Integrals.ReCallVJP{Integrals.ZygoteVJP}, do_inf_transformation::Nothing, kwargs::Base.Pairs{Symbol, Real, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:reltol, :abstol, :maxiters), Tuple{Float64, Float64, Int64}}})
@ Integrals C:\Users\lakshay.julia\packages\Integrals\9qNWp\src\Integrals.jl:108
[26] integrate(quadalg::HCubatureJL{typeof(LinearAlgebra.norm)}, adalg::NonfusedAD, f::Function, lb::StaticArraysCore.SVector{6, Float64}, ub::StaticArraysCore.SVector{6, Float64}, p::ArrayPartition{Float64, Tuple{Vector{Float64}, Vector{Float64}}}; nout::Int64, batch::Int64, kwargs::Base.Pairs{Symbol, Real, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:reltol, :abstol, :maxiters), Tuple{Float64, Float64, Int64}}})
@ SciMLExpectations C:\Users\lakshay.julia\packages\SciMLExpectations\DWytJ\src\expectation.jl:148
[27] solve(::ExpectationProblem{SystemMap{ODEProblem{true,Vector{Float64},Tuple{Float64, Float64},…}, Tuple{Tsit5{Static.False,…}}, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}, typeof(loss), typeof(h), GenericDistribution{SciMLExpectations.var"#pdf_func#4"{Tuple{Normal{Float64}, Normal{Float64}, Normal{Float64}, Int64, Int64, Int64}}, SciMLExpectations.var"#rand_func#6"{Tuple{Normal{Float64}, Normal{Float64}, Normal{Float64}, Int64, Int64, Int64}}, StaticArraysCore.SVector{6, Float64}, StaticArraysCore.SVector{6, Float64}}, ArrayPartition{Float64, Tuple{Vector{Float64}, Vector{Float64}}}}, ::Koopman{NonfusedAD}; maxiters::Int64, batch::Int64, quadalg::HCubatureJL{typeof(LinearAlgebra.norm)}, ireltol::Float64, iabstol::Float64, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ SciMLExpectations C:\Users\lakshay.julia\packages\SciMLExpectations\DWytJ\src\expectation.jl:134
[28] solve(::ExpectationProblem{SystemMap{ODEProblem{true,Vector{Float64},Tuple{Float64, Float64},…}, Tuple{Tsit5{Static.False,…}}, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}, typeof(loss), typeof(h), GenericDistribution{SciMLExpectations.var"#pdf_func#4"{Tuple{Normal{Float64}, Normal{Float64}, Normal{Float64}, Int64, Int64, Int64}}, SciMLExpectations.var"#rand_func#6"{Tuple{Normal{Float64}, Normal{Float64}, Normal{Float64}, Int64, Int64, Int64}}, StaticArraysCore.SVector{6, Float64}, StaticArraysCore.SVector{6, Float64}}, ArrayPartition{Float64, Tuple{Vector{Float64}, Vector{Float64}}}}, ::Koopman{NonfusedAD})
@ SciMLExpectations C:\Users\lakshay.julia\packages\SciMLExpectations\DWytJ\src\expectation.jl:125
[29] top-level scope
@ In[57]:1

​
Can you please help me with this ? I dont know if I am missing something

Here is the code I am trying to execute:

using SciMLExpectations

u0s_dis = [cor_dist_1,cor_dist_2,cor_dist_3,0,0,0]

A Genereic function containing all the aspects of the dynamical system you are uncertain about, in this example this is the initial condition.

gd = GenericDistribution(u0s_dis...)

#Covariance matrix

The function h which maps a realization of the uncertainty space to the initial conditions and parameters of the ODEProblem

h(x, u, p) = u, p

prob = ODEProblem(hcw!, u0, tspan, p)

sm = SystemMap(prob, solver)

exprob = ExpectationProblem(sm, loss, h, gd; nout = 1)

sol = solve(exprob,Koopman())
sol.u

Here, problem is the forced dynamics of a system

Regards

Koopman() now supports infinite integration domains

According to the SciML tutorial,

https://github.com/SciML/SciMLTutorials.jl/blob/e8c8c3e51703fd08b874abbbc7b52ed895d0cb6f/tutorials/DiffEqUncertainty/01-expectation_introduction.jmd#L111

However, when I actually tried this, I got the right answer:

u0_dist = [Normal(3.0,2.0)]
expectation(g, prob, u0_dist, p, Koopman(), Tsit5())

u: 1-element Vector{Float64}:
 0.9035426476099575

Compared with the analytical solution:

exp(p[1]*4.0)*mean(u0_dist[1])

0.9035826357366064

Add Codespace support

Compute resid

provide methods to compute resid for expectation(MonteCarlo()) and centralmoment

Definition of g

From the docstring:
𝕌 = Initial condition space, ℙ = model parameter space
g: 𝕌 × ℙ → ℝⁿᵒᵘᵗ
But this cannot be true since you can calculate expectation at multiple or even infinite time-points.
It is more like:
g: 𝕌 × 𝕋 × ℙ → ℝⁿᵒᵘᵗ ,
where 𝕋 is the time-space.
𝕌 × 𝕋 is then the solution space, which is how g is implemented in practice.

and possibly also better to call 𝕌 state space rather than initial condition space.