sciml / stochasticdiffeq.jl

Solvers for stochastic differential equations which connect with the scientific machine learning (SciML) ecosystem

StochasticDiffEq.jl

StochasticDiffEq.jl is a component package in the DifferentialEquations ecosystem. It holds the stochastic differential equations solvers and utilities. While completely independent and usable on its own, users interested in using this functionality should check out DifferentialEquations.jl.

API

StochasticDiffEq.jl is part of the JuliaDiffEq common interface, but can be used independently of DifferentialEquations.jl. The only requirement is that the user passes an StochasticDiffEq.jl algorithm to solve. For example, we can solve the SDE tutorial from the docs using the SRIW1() algorithm:

using StochasticDiffEq
α=1
β=1
u₀=1/2
f(u,p,t) = α*u
g(u,p,t) = β*u
dt = 1//2^(4)
tspan = (0.0,1.0)
prob = SDEProblem(f,g,u₀,(0.0,1.0))
sol =solve(prob,SRIW1())

The options for solve are defined in the common solver options page and are thoroughly explained in the ODE tutorial.

That example uses the out-of-place syntax f(u,p,t), while the inplace syntax (more efficient for systems of equations) is shown in the Lorenz example:

function lorenz(du,u,p,t)
 du[1] = 10.0(u[2]-u[1])
 du[2] = u[1]*(28.0-u[3]) - u[2]
 du[3] = u[1]*u[2] - (8/3)*u[3]
end

function σ_lorenz(du,u,p,t)
 du[1] = 3.0
 du[2] = 3.0
 du[3] = 3.0
end

prob_sde_lorenz = SDEProblem(lorenz,σ_lorenz,[1.0,0.0,0.0],(0.0,10.0))
sol = solve(prob_sde_lorenz)
plot(sol,vars=(1,2,3))

The problems default to diagonal noise. Non-diagonal noise can be added by setting the noise_prototype:

f = (du,u,p,t) -> du.=1.01u
g = function (du,u,p,t)
  du[1,1] = 0.3u[1]
  du[1,2] = 0.6u[1]
  du[1,3] = 0.9u[1]
  du[1,4] = 0.12u[2]
  du[2,1] = 1.2u[1]
  du[2,2] = 0.2u[2]
  du[2,3] = 0.3u[2]
  du[2,4] = 1.8u[2]
end
prob = SDEProblem(f,g,ones(2),(0.0,1.0),noise_rate_prototype=zeros(2,4))

Colored noise can be set using an AbstractNoiseProcess. For example, we can set the underlying noise process to a GeometricBrownianMotionProcess via:

μ = 1.0
σ = 2.0
W = GeometricBrownianMotionProcess(μ,σ,0.0,1.0,1.0)
# ...
# Define f,g,u0,tspan for a SDEProblem
# ...
prob = SDEProblem(f,g,u0,tspan,noise=W)

StochasticDiffEq.jl also handles solving random ordinary differential equations. This is shown in the RODE tutorial.

using StochasticDiffEq
function f(u,p,t,W)
  2u*sin(W)
end
u0 = 1.00
tspan = (0.0,5.0)
prob = RODEProblem(f,u0,tspan)
sol = solve(prob,RandomEM(),dt=1/100)

Available Solvers

For the list of available solvers, please refer to the DifferentialEquations.jl SDE Solvers page and the RODE Solvers page.

stochasticdiffeq.jl's Issues

Low truncation error Stratonovich method

http://www.sciencedirect.com/science/article/pii/S0377042717305010

Stochastic Interpolations

Just start with linear interpolations (justified by Wong-Zukai)

PI adaptive timestepping

Set StochasticDiffEq.jl with the Integrator Interface

Strong Order 1.0 general Stratanovich integrators

http://www.sciencedirect.com/science/article/pii/S0377042707006012

Adams-Type methods for non-commutative SDE

https://link.springer.com/article/10.1023/A:1022363612387

Adaptive time stepping heuristics for low order methods

Going to put together a new algorithm based on

https://www.sciencedirect.com/science/article/pii/S0377042703007155
https://www.sciencedirect.com/science/article/pii/037704279400007N

much like the SPICE estimator of the Trapezoid method.

Methods for solving for moment equations

Instead of solving the SDE, it would be nice to be able to get the evolution of the moments. Here's some references for special cases, but I don't see the general algorithm in some form yet. We may need to choose a special form, like additive, and just implement it on that:

http://ieeexplore.ieee.org/document/7798960/
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.332.4621&rep=rep1&type=pdf
https://link.springer.com/article/10.1007/BF01543805
http://aip.scitation.org/doi/abs/10.1063/1.4929837?crawler=true&mimetype=application%2Fpdf&journalCode=jcp

Add `tdir` compatibility

Split Step Explicit Methods

These are a lot like the methods that @onoderat implemented, but they are fully explicit and don't require derivative calculations. They are essentially the Runge-Kutta forms of that. They aren't too difficult to implement (but could be extremely impactful), so for now I'll leave the Milstein methods as a possible outside contribution for others.

https://www.sciencedirect.com/science/article/pii/S0377042709007419

Error tagging new release

The tag name "2.0.0" is not of the appropriate SemVer form (vX.Y.Z).
cc: @ChrisRackauckas

Precompilation failure

Precompilation of StochasticDiffEq.jl fails both on the latest tagged version and on master

julia> Pkg.update()
...
julia> using StochasticDiffEq
INFO: Precompiling module StochasticDiffEq.
ERROR: LoadError: LoadError: UndefVarError: @compat not defined
 in macro expansion; at ./none:2 [inlined]
 in anonymous at ./<missing>:?
while loading /home/blegat/.julia/v0.5/StochasticDiffEq/src/algorithms.jl, in expression starting on line 5
while loading /home/blegat/.julia/v0.5/StochasticDiffEq/src/StochasticDiffEq.jl, in expression starting on line 21
ERROR: Failed to precompile StochasticDiffEq to /home/blegat/.julia/lib/v0.5/StochasticDiffEq.ji.
 in eval_user_input(::Any, ::Base.REPL.REPLBackend) at ./REPL.jl:64
 in macro expansion at ./REPL.jl:95 [inlined]
 in (::Base.REPL.##3#4{Base.REPL.REPLBackend})() at ./event.jl:68

julia> Pkg.checkout("StochasticDiffEq")
INFO: Checking out StochasticDiffEq master...
INFO: Pulling StochasticDiffEq latest master...
INFO: No packages to install, update or remove

julia> using StochasticDiffEq
INFO: Precompiling module StochasticDiffEq.
ERROR: LoadError: LoadError: UndefVarError: @compat not defined
 in macro expansion; at ./none:2 [inlined]
 in anonymous at ./<missing>:?
while loading /home/blegat/.julia/v0.5/StochasticDiffEq/src/algorithms.jl, in expression starting on line 5
while loading /home/blegat/.julia/v0.5/StochasticDiffEq/src/StochasticDiffEq.jl, in expression starting on line 21
ERROR: Failed to precompile StochasticDiffEq to /home/blegat/.julia/lib/v0.5/StochasticDiffEq.ji.
 in eval_user_input(::Any, ::Base.REPL.REPLBackend) at ./REPL.jl:64
 in macro expansion at ./REPL.jl:95 [inlined]
 in (::Base.REPL.##3#4{Base.REPL.REPLBackend})() at ./event.jl:68

Fiddling with the adaptive defaults

I tend to think that beta1 and beta2 are off. I think that beta1=0.4 and beta2=0.1 may be better defaults than what we have right now, but this would need extensive testing. That's a very high gain but right now we rarely ever reject, and are usually 10-100x below the error estimate we are aiming for.

SROCK Methods

• SROCK2 https://hal.archives-ouvertes.fr/hal-00739754v1/document
• SROCK1 https://epubs.siam.org/doi/abs/10.1137/070679375
• SROCK1 Ito https://projecteuclid.org/download/pdf_1/euclid.cms/1229619673
• SKSROCK https://arxiv.org/abs/1708.08145
• KomBurSROCK2 https://www.sciencedirect.com/science/article/pii/S0377042712000441
• TangXiaoSROCK2 https://link.springer.com/article/10.1007/s11075-018-0615-y
• SROCKC2 https://ijnao.um.ac.ir/index.php/math/article/view/69454

Truncated implicit Milstein

https://www.sciencedirect.com/science/article/pii/S0377042718300359

Adaptive weak noise multistep

Based off of:

https://www.sciencedirect.com/science/article/pii/S0377042706004195
https://www.math.hu-berlin.de/~winkler/papers/smssiam.pdf

Higher order solvers for non-diagonal noise

Automatic ggprime handling

#53

@onoderat 's Pr introduces ggprime which is g times the Jacobian of g. This is defined in the PCE paper (see SciML/DifferentialEquations.jl#247 )

In-place noise updates

• Make an in-place next! for ChunkedArrays
• Make ChunkedArrays re-update the buffer in-place?
• Add a way for option number of arguments for ChunkedArrays. Send W for fractional Brownian motion

Split Step Balanced Milstein

https://www.sciencedirect.com/science/article/pii/S0168927408001207

Extrapolations for implicit methods

Instead of just using the linear extrapolant, it should take into account the noise to do a better prediction of the first value.

Try new RNGs from RNG.jl

Add Weak Error Calculations

Error tagging new release

The tag name "4.2.1" is not of the appropriate SemVer form (vX.Y.Z).
cc: @ChrisRackauckas

Improvements to save_idxs

similar to SciML/OrdinaryDiffEq.jl#180

Split-Step Implicit Euler

https://arxiv.org/pdf/1204.6620.pdf

DoubleImplicitMilstein

I think there should just be an option to enable the double implicitness, and have it true by default.

https://arxiv.org/pdf/1204.1647.pdf

RODE RDE Random Ordinary Differential Equations thesis

http://publikationen.ub.uni-frankfurt.de/frontdoor/index/index/docId/40146

Basic Implicit Methods

• Implicit Euler Maruyama
• Implicit Trapezoid Maruyama
• Implicit Euler Milstein
• Implicit Trapezoid Milstein

This paper shows that the trapezoid versions will be more exact: https://arxiv.org/pdf/1310.0392.pdf .

Improved Milstein for stiff

https://advancesindifferenceequations.springeropen.com/track/pdf/10.1186/s13662-015-0699-9?site=advancesindifferenceequations.springeropen.com

Make all methods write from `uprev` to `u`

Matching OrdinaryDiffEq.jl. This is part of making the Integrator interface work.

SDEProblem(..., noise=NoiseWrapper(OrnsteinUhlenbeckProcess!(...))) does not work with adaptive method

I thought I'd use NoiseWrapper(OrnsteinUhlenbeckProcess!(...)) until the bridge distribution is implemented for Ornstein-Uhlenbeck Process to use adaptive method. But it didn't work:

oup = OrnsteinUhlenbeckProcess!(1.0, 0.0, 1.0, 0.0, [0.0], [0.0])
noise = NoiseWrapper(oup)
prob = SDEProblem((t, u) -> -u, (t, u) -> 1.0, [0.0], (0.0, 1.0), noise=noise)
sol = solve(prob)

Note that solve(prob, dt=1e-3, adaptive=false) works. Above code failed with DimensionMismatch error:

ERROR: DimensionMismatch("Cannot left-divide transposed vector by matrix")
Stacktrace:
 [1] perform_step! at ~/.julia/v0.6/StochasticDiffEq/src/integrators/sri.jl:293 [inlined]
 [2] perform_step! at ~/.julia/v0.6/StochasticDiffEq/src/integrators/sri.jl:262 [inlined]
 [3] solve!(::StochasticDiffEq.SDEIntegrator{StochasticDiffEq.SRIW1,Array{Float64,1},Float64,Float64,Float64,Float64,DiffEqNoiseProcess.NoiseWrapper{Float64,2,Float64,Array{Float64,1},Array{Float64,1},DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Float64,Float64,Float64},Void,true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},Array{Array{Float64,1},1},true},Array{Float64,1},DiffEqBase.RODESolution{Float64,2,Array{Array{Float64,1},1},Void,Void,Array{Float64,1},DiffEqNoiseProcess.NoiseWrapper{Float64,2,Float64,Array{Float64,1},Array{Float64,1},DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Float64,Float64,Float64},Void,true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},Array{Array{Float64,1},1},true},DiffEqBase.SDEProblem{Array{Float64,1},Float64,false,DiffEqNoiseProcess.NoiseWrapper{Float64,2,Float64,Array{Float64,1},Array{Float64,1},DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Float64,Float64,Float64},Void,true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},Array{Array{Float64,1},1},true},##109#111,##110#112,Void,UniformScaling{Int64},Void},StochasticDiffEq.SRIW1,StochasticDiffEq.LinearInterpolationData{Array{Array{Float64,1},1},Array{Float64,1}}},StochasticDiffEq.SRIW1ConstantCache,Void,##109#111,##110#112,StochasticDiffEq.SDEOptions{Float64,Float64,DiffEqBase.#ODE_DEFAULT_NORM,DiffEqBase.CallbackSet{Tuple{},Tuple{}},DiffEqBase.#ODE_DEFAULT_ISOUTOFDOMAIN,DiffEqBase.#ODE_DEFAULT_PROG_MESSAGE,DiffEqBase.#ODE_DEFAULT_UNSTABLE_CHECK,DataStructures.BinaryHeap{Float64,DataStructures.LessThan},Void,Void,Float64,Float64,Float64,Float64},DiffEqNoiseProcess.NoiseWrapper{Float64,2,Float64,Array{Float64,1},Array{Float64,1},DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Float64,Float64,Float64},Void,true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},Array{Array{Float64,1},1},true}}) at ~/.julia/v0.6/StochasticDiffEq/src/solve.jl:391
 [4] #solve#58(::Array{Any,1}, ::Function, ::DiffEqBase.SDEProblem{Array{Float64,1},Float64,false,DiffEqNoiseProcess.NoiseWrapper{Float64,2,Float64,Array{Float64,1},Array{Float64,1},DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Float64,Float64,Float64},Void,true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},Array{Array{Float64,1},1},true},##109#111,##110#112,Void,UniformScaling{Int64},Void}, ::StochasticDiffEq.SRIW1, ::Array{Any,1}, ::Array{Any,1}, ::Array{Any,1}, ::Type{Val{true}}) at ~/.julia/v0.6/StochasticDiffEq/src/solve.jl:7
 [5] (::DiffEqBase.#kw##solve)(::Array{Any,1}, ::DiffEqBase.#solve, ::DiffEqBase.SDEProblem{Array{Float64,1},Float64,false,DiffEqNoiseProcess.NoiseWrapper{Float64,2,Float64,Array{Float64,1},Array{Float64,1},DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Float64,Float64,Float64},Void,true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},Array{Array{Float64,1},1},true},##109#111,##110#112,Void,UniformScaling{Int64},Void}, ::StochasticDiffEq.SRIW1, ::Array{Any,1}, ::Array{Any,1}, ::Array{Any,1}, ::Type{Val{true}}) at ./<missing>:0 (repeats 2 times)
 [6] #solve#2(::Bool, ::Array{Any,1}, ::Function, ::DiffEqBase.SDEProblem{Array{Float64,1},Float64,false,DiffEqNoiseProcess.NoiseWrapper{Float64,2,Float64,Array{Float64,1},Array{Float64,1},DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Float64,Float64,Float64},Void,true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},Array{Array{Float64,1},1},true},##109#111,##110#112,Void,UniformScaling{Int64},Void}, ::Void) at ~/.julia/v0.6/DifferentialEquations/src/default_solve.jl:14
 [7] (::DiffEqBase.#kw##solve)(::Array{Any,1}, ::DiffEqBase.#solve, ::DiffEqBase.SDEProblem{Array{Float64,1},Float64,false,DiffEqNoiseProcess.NoiseWrapper{Float64,2,Float64,Array{Float64,1},Array{Float64,1},DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Float64,Float64,Float64},Void,true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},Array{Array{Float64,1},1},true},##109#111,##110#112,Void,UniformScaling{Int64},Void}, ::Void) at ./<missing>:0
 [8] #solve#1(::Bool, ::Array{Any,1}, ::Function, ::DiffEqBase.SDEProblem{Array{Float64,1},Float64,false,DiffEqNoiseProcess.NoiseWrapper{Float64,2,Float64,Array{Float64,1},Array{Float64,1},DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Float64,Float64,Float64},Void,true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},Array{Array{Float64,1},1},true},##109#111,##110#112,Void,UniformScaling{Int64},Void}) at ~/.julia/v0.6/DifferentialEquations/src/default_solve.jl:5
 [9] solve(::DiffEqBase.SDEProblem{Array{Float64,1},Float64,false,DiffEqNoiseProcess.NoiseWrapper{Float64,2,Float64,Array{Float64,1},Array{Float64,1},DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Float64,Float64,Float64},Void,true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},Array{Array{Float64,1},1},true},##109#111,##110#112,Void,UniformScaling{Int64},Void}) at ~/.julia/v0.6/DifferentialEquations/src/default_solve.jl:2
 [10] macro expansion at ./REPL.jl:97 [inlined]
 [11] (::Base.REPL.##1#2{Base.REPL.REPLBackend})() at ./event.jl:73

Exact simulation of SDEs: has lots of analytical solutions

https://link.springer.com/chapter/10.1007%2F978-3-642-13694-8_2

Multiple dispatch on special structural properties (commutative, diagonal, scalar etc.)

The source code in the repo currently has a few if statements to deal with the special structural properties of SDEs. For instance in the perform_step of EM, the following code is found

  if !is_diagonal_noise(integrator.sol.prob) || typeof(W.dW) <: Number
    noise = integrator.g(uprev,p,t)*W.dW
  else
    noise = integrator.g(uprev,p,t).*W.dW
  end

These if statements can be removed if we multiple dispatch on a structure type.

This structure type also has other implications for generating the wiener integrals. For instance if this structure is commutative, a generate noise process (to be written) can avoid the Wiktorsson approximations for I_{j1, j1}.

In the long term, this should help remove multiple solver methods structs like RKMilCommute vs RKMil using parametrized types of this structure type.

Currently the ideas here are very vague. Please let me know how you think it can be polished.

Fixed seed should fix noise

I think there is at least one bug. But maybe there is one more bug.

Bug 1

I'm not perfectly sure if using NoiseWrapper(OrnsteinUhlenbeckProcess!(...)) this way is OK, but I believe crashing randomly is not OK. Here is the code to reproduce the bug. Solving a SDE problem succeeds sometimes and fails sometimes. Following code try to solve it until there are 5 successes and 5 failures.

using DifferentialEquations

num_err = 0
for i = 1:1000
    if num_err > 5 && i - num_err > 5
        break
    end
    tspan = (0.0, 1.0)
    oup = OrnsteinUhlenbeckProcess!([1.0], 0.0, [1.0], 0.0, [0.0], [0.0])
    sol_noise = solve(NoiseProblem(oup, tspan), dt=0.001, seed=1)
    noise = NoiseWrapper(oup)
    prob = SDEProblem((t, u, du) -> du.=-u,
                      (t, u, g) -> g.=u,
                      [1.0], tspan, noise=noise)
    try
        sol = solve(prob, seed=1)
    catch
        num_err += 1
        print("!")
        continue
    end
    print(".")
end

I guess passing seed=1 to the second solve() (the on inside try) is no-op, since, if I understand correctly, the seed fed into the RNG is the one used when solving NoiseProblem. I was just passing it since I wans't 100% sure.

!!!!WARNING: dt <= dtmin. Aborting. If you would like to force continuation with dt=dtmin, set force_dtmin=true
.!!!!!!!!!!!!!!!!!WARNING: dt <= dtmin. Aborting. If you would like to force continuation with dt=dtmin, set force_dtmin=true
.!WARNING: dt <= dtmin. Aborting. If you would like to force continuation with dt=dtmin, set force_dtmin=true
..!!!!WARNING: dt <= dtmin. Aborting. If you would like to force continuation with dt=dtmin, set force_dtmin=true
.

(Side note: The OK case . usually gets the warning but not always; in the above example, there are four WARNINGs and five .)

Bug 2 (?)

When solving the above SDE problem fails, it complains that the bridge is not defined (see below for the full traceback). I thought NoiseWrapper is there for avoiding generating new samples. Or should I use NoiseGrid for this purpose? But then why the above code works sometimes? I was lucky so that all the adaptive time steps are on the pre-drawn time points?

ERROR: LoadError: MethodError: objects of type Void are not callable
Stacktrace:
 [1] interpolate! at /home/takafumi/.julia/v0.6/DiffEqNoiseProcess/src/noise_interfaces/noise_process_interface.jl:458 [inlined]
 [2] (::DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Array{Float64,1},Float64,Array{Float64,1}},Void,true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus})(::Array{Float64,1}, ::Array{Float64,1}, ::Float64) at /home/takafumi/.julia/v0.6/DiffEqNoiseProcess/src/types.jl:31
 [3] calculate_step!(::DiffEqNoiseProcess.NoiseWrapper{Float64,2,Float64,Array{Float64,1},Array{Float64,1},DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Array{Float64,1},Float64,Array{Float64,1}},Void,true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},Array{Array{Float64,1},1},true}, ::Float64) at /home/takafumi/.julia/v0.6/DiffEqNoiseProcess/src/noise_interfaces/noise_wrapper_interface.jl:15
 [4] loopheader! at /home/takafumi/.julia/v0.6/StochasticDiffEq/src/integrators/integrator_utils.jl:17 [inlined]
 [5] solve!(::StochasticDiffEq.SDEIntegrator{StochasticDiffEq.SRIW1,Array{Float64,1},Float64,Float64,Float64,Float64,DiffEqNoiseProcess.NoiseWrapper{Float64,2,Float64,Array{Float64,1},Array{Float64,1},DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Array{Float64,1},Float64,Array{Float64,1}},Void,true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},Array{Array{Float64,1},1},true},Array{Float64,1},DiffEqBase.RODESolution{Float64,2,Array{Array{Float64,1},1},Void,Void,Array{Float64,1},DiffEqNoiseProcess.NoiseWrapper{Float64,2,Float64,Array{Float64,1},Array{Float64,1},DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Array{Float64,1},Float64,Array{Float64,1}},Void,true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},Array{Array{Float64,1},1},true},DiffEqBase.SDEProblem{Array{Float64,1},Float64,true,DiffEqNoiseProcess.NoiseWrapper{Float64,2,Float64,Array{Float64,1},Array{Float64,1},DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Array{Float64,1},Float64,Array{Float64,1}},Void,true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},Array{Array{Float64,1},1},true},##389#391,##390#392,Void,UniformScaling{Int64},Void},StochasticDiffEq.SRIW1,StochasticDiffEq.LinearInterpolationData{Array{Array{Float64,1},1},Array{Float64,1}}},StochasticDiffEq.SRIW1Cache{Array{Float64,1},Array{Float64,1},Array{Float64,1}},Void,##389#391,##390#392,StochasticDiffEq.SDEOptions{Float64,Float64,DiffEqBase.#ODE_DEFAULT_NORM,DiffEqBase.CallbackSet{Tuple{},Tuple{}},DiffEqBase.#ODE_DEFAULT_ISOUTOFDOMAIN,DiffEqBase.#ODE_DEFAULT_PROG_MESSAGE,DiffEqBase.#ODE_DEFAULT_UNSTABLE_CHECK,DataStructures.BinaryHeap{Float64,DataStructures.LessThan},Void,Void,Float64,Float64,Float64,Float64},DiffEqNoiseProcess.NoiseWrapper{Float64,2,Float64,Array{Float64,1},Array{Float64,1},DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Array{Float64,1},Float64,Array{Float64,1}},Void,true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},Array{Array{Float64,1},1},true}}) at /home/takafumi/.julia/v0.6/StochasticDiffEq/src/solve.jl:389
 [6] #solve#58(::Array{Any,1}, ::Function, ::DiffEqBase.SDEProblem{Array{Float64,1},Float64,true,DiffEqNoiseProcess.NoiseWrapper{Float64,2,Float64,Array{Float64,1},Array{Float64,1},DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Array{Float64,1},Float64,Array{Float64,1}},Void,true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},Array{Array{Float64,1},1},true},##389#391,##390#392,Void,UniformScaling{Int64},Void}, ::StochasticDiffEq.SRIW1, ::Array{Any,1}, ::Array{Any,1}, ::Array{Any,1}, ::Type{Val{true}}) at /home/takafumi/.julia/v0.6/StochasticDiffEq/src/solve.jl:7
 [7] (::DiffEqBase.#kw##solve)(::Array{Any,1}, ::DiffEqBase.#solve, ::DiffEqBase.SDEProblem{Array{Float64,1},Float64,true,DiffEqNoiseProcess.NoiseWrapper{Float64,2,Float64,Array{Float64,1},Array{Float64,1},DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Array{Float64,1},Float64,Array{Float64,1}},Void,true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},Array{Array{Float64,1},1},true},##389#391,##390#392,Void,UniformScaling{Int64},Void}, ::StochasticDiffEq.SRIW1, ::Array{Any,1}, ::Array{Any,1}, ::Array{Any,1}, ::Type{Val{true}}) at ./<missing>:0 (repeats 2 times)
 [8] #solve#2(::Bool, ::Array{Any,1}, ::Function, ::DiffEqBase.SDEProblem{Array{Float64,1},Float64,true,DiffEqNoiseProcess.NoiseWrapper{Float64,2,Float64,Array{Float64,1},Array{Float64,1},DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Array{Float64,1},Float64,Array{Float64,1}},Void,true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},Array{Array{Float64,1},1},true},##389#391,##390#392,Void,UniformScaling{Int64},Void}, ::Void) at /home/takafumi/.julia/v0.6/DifferentialEquations/src/default_solve.jl:14
 [9] (::DiffEqBase.#kw##solve)(::Array{Any,1}, ::DiffEqBase.#solve, ::DiffEqBase.SDEProblem{Array{Float64,1},Float64,true,DiffEqNoiseProcess.NoiseWrapper{Float64,2,Float64,Array{Float64,1},Array{Float64,1},DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Array{Float64,1},Float64,Array{Float64,1}},Void,true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},Array{Array{Float64,1},1},true},##389#391,##390#392,Void,UniformScaling{Int64},Void}, ::Void) at ./<missing>:0
 [10] #solve#1(::Bool, ::Array{Any,1}, ::Function, ::DiffEqBase.SDEProblem{Array{Float64,1},Float64,true,DiffEqNoiseProcess.NoiseWrapper{Float64,2,Float64,Array{Float64,1},Array{Float64,1},DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Array{Float64,1},Float64,Array{Float64,1}},Void,true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},Array{Array{Float64,1},1},true},##389#391,##390#392,Void,UniformScaling{Int64},Void}) at /home/takafumi/.julia/v0.6/DifferentialEquations/src/default_solve.jl:5
 [11] (::DiffEqBase.#kw##solve)(::Array{Any,1}, ::DiffEqBase.#solve, ::DiffEqBase.SDEProblem{Array{Float64,1},Float64,true,DiffEqNoiseProcess.NoiseWrapper{Float64,2,Float64,Array{Float64,1},Array{Float64,1},DiffEqNoiseProcess.NoiseProcess{Float64,2,Float64,Array{Float64,1},Array{Float64,1},Array{Array{Float64,1},1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Array{Float64,1},Float64,Array{Float64,1}},Void,true,DataStructures.Stack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},ResettableStacks.ResettableStack{Tuple{Float64,Array{Float64,1},Array{Float64,1}}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},Array{Array{Float64,1},1},true},##389#391,##390#392,Void,UniformScaling{Int64},Void}) at ./<missing>:0
 [12] macro expansion at /home/takafumi/junk/2017/08/26-235344.jl:15 [inlined]
 [13] anonymous at ./<missing>:?
 [14] include_from_node1(::String) at ./loading.jl:569
 [15] macro expansion at ./REPL.jl:97 [inlined]
 [16] (::Base.REPL.##1#2{Base.REPL.REPLBackend})() at ./event.jl:73

Implement tstops

Multiple trajectory problems

In many cases one may want to solve the same SDE multiple times together. This can be different from Monte Carlo since coupling the trajectories can allow for faster convergence. Thus we can handle a MultipleTrajectorySDEProblem by generating a nested u0 of copies, putting a wrapper over a user-given f to calculate the larger f in pieces, handle the noise couplings, and put that directly into the solvers. This would allow building coupled versions of each solver, and incorporate algorithms like:

http://epubs.siam.org/doi/10.1137/15M1040098

Callback for Stochastic Jump Equations

This is done by implementing a Poisson distributed tstops along with the Jump affect!s. This will actually be added to DiffEqCallbacks. Then respond here:

https://discourse.julialang.org/t/code-for-simulating-jump-diffusion/1154

This requires the integrator interface + callbacks.

RKMil Stratonovich Interpretation

The Milstein strong order 1 derivative-free method has a Stratonovich version, which I am implementing via RKMil(interpretation=:Stratonovich). However, the algorithm doesn't seem to be converging correctly right now.

Resources:
https://arxiv.org/pdf/1102.4401.pdf
https://www.doria.fi/bitstream/handle/10024/93801/new_methods.pdf?sequence=3
https://infoscience.epfl.ch/record/143450/files/sde_tutorial.pdf
http://www.sciencedirect.com/science/article/pii/S0377042706004195

For now, it won't be documented. It's commented out test is:

https://github.com/JuliaDiffEq/StochasticDiffEq.jl/blob/master/test/stratonovich_convergence_tests.jl#L9

And the step is defined at:

https://github.com/JuliaDiffEq/StochasticDiffEq.jl/blob/master/src/integrators/low_order.jl#L61

If anyone wants this, just take a crack at it and find out how to change that step.

Scalar noise in SRI Methods

The SRI methods can actually handle the scalar noise case where a single noise term is repeated to all of the other entries. It would be nice to be able to set noise_prototype as a <:Number and then interpret that as a scalar noise problem.

More Measurements in MonteCarloSimulation

Integrator Change: Performance Tracking

The integrator change is almost complete, but there is an issue with performance tracking. For the test problem I am using

using StochasticDiffEq, DiffEqProblemLibrary
srand(200)
prob = oval2ModelExample(largeFluctuations=true,useBigs=false)
quick_prob = deepcopy(prob)
quick_prob.tspan = (0.0,1.0)

using BenchmarkTools
@benchmark begin
  srand(100)
  sol = solve(quick_prob,SRIW1(),dt=(1/2)^(18),progress_steps=Int(1e5),
        adaptivealg=:RSwM3,progress=false,qmax=4,save_timeseries=false,
        timeseries_steps=1000,abstol=1e-5,reltol=1e-3)
end

To run these you need to be on bench2 for DiffEqBase.

I thoroughly checked the accuracy of the calculations. Except in early commits, everything calculates the exact same trajectory. For most commits, note that the branch needs to have a fix. There are two locations where sqrt needs to be updated: both in the rejections. See f3b0979 for an example.

So at any point, a branch can be made off a commit and benchmarks can occur. Here's what happens over time.

The "Base" branch is bench_pi: it's before the most recent changes, only uses the macros, and is the fastest.

  # Bench PI

  BenchmarkTools.Trial:
  memory estimate:  61.67 mb
  allocs estimate:  487561
  --------------
  minimum time:     45.840 ms (9.37% GC)
  median time:      48.309 ms (13.53% GC)
  mean time:        48.174 ms (13.15% GC)
  maximum time:     52.718 ms (7.41% GC)
  --------------
  samples:          104
  evals/sample:     1
  time tolerance:   5.00%
  memory tolerance: 1.00%

  BenchmarkTools.Trial:
  memory estimate:  61.67 mb
  allocs estimate:  487562
  --------------
  minimum time:     44.596 ms (8.16% GC)
  median time:      46.701 ms (12.55% GC)
  mean time:        46.558 ms (12.04% GC)
  maximum time:     52.336 ms (6.89% GC)
  --------------
  samples:          108
  evals/sample:     1
  time tolerance:   5.00%
  memory tolerance: 1.00%

Next we have bench_mid: it's most of the changes, but still lots of the @def macros

# Bench Mid

BenchmarkTools.Trial: 
  memory estimate:  61.26 mb
  allocs estimate:  461027
  --------------
  minimum time:     46.949 ms (8.61% GC)
  median time:      49.106 ms (12.10% GC)
  mean time:        48.942 ms (11.64% GC)
  maximum time:     54.735 ms (6.63% GC)
  --------------
  samples:          103
  evals/sample:     1
  time tolerance:   5.00%
  memory tolerance: 1.00%

BenchmarkTools.Trial: 
  memory estimate:  61.26 mb
  allocs estimate:  461028
  --------------
  minimum time:     47.503 ms (8.61% GC)
  median time:      49.853 ms (12.14% GC)
  mean time:        49.618 ms (11.65% GC)
  maximum time:     55.661 ms (6.90% GC)
  --------------
  samples:          101
  evals/sample:     1
  time tolerance:   5.00%
  memory tolerance: 1.00%

It's acceptably close to bench PI, though I would like to find out why it's slower if possible. After that it's bench_single_solve, where most of the macros are gone:

  # Bench Single Solve

BenchmarkTools.Trial: 
  memory estimate:  61.73 mb
  allocs estimate:  476536
  --------------
  minimum time:     48.410 ms (8.36% GC)
  median time:      50.787 ms (12.60% GC)
  mean time:        51.230 ms (12.67% GC)
  maximum time:     75.402 ms (16.45% GC)
  --------------
  samples:          98
  evals/sample:     1
  time tolerance:   5.00%
  memory tolerance: 1.00%

BenchmarkTools.Trial: 
  memory estimate:  61.73 mb
  allocs estimate:  476537
  --------------
  minimum time:     47.717 ms (8.66% GC)
  median time:      50.336 ms (12.84% GC)
  mean time:        50.223 ms (12.43% GC)
  maximum time:     55.817 ms (6.87% GC)
  --------------
  samples:          100
  evals/sample:     1
  time tolerance:   5.00%
  memory tolerance: 1.00%

Again, small performance loss, no idea why. Right after that is bench_accept_header, where now all of the solvers use the same solve! command and there's a new loop header. Here's where it gets super interesting. If I comment out one like, I get:

BenchmarkTools.Trial: 
  memory estimate:  61.73 mb
  allocs estimate:  476535
  --------------
  minimum time:     49.142 ms (8.27% GC)
  median time:      51.910 ms (12.56% GC)
  mean time:        54.284 ms (13.12% GC)
  maximum time:     99.443 ms (13.03% GC)
  --------------
  samples:          93
  evals/sample:     1
  time tolerance:   5.00%
  memory tolerance: 1.00%

BenchmarkTools.Trial: 
  memory estimate:  61.73 mb
  allocs estimate:  476535
  --------------
  minimum time:     49.518 ms (7.91% GC)
  median time:      57.708 ms (13.50% GC)
  mean time:        57.122 ms (14.73% GC)
  maximum time:     85.435 ms (32.41% GC)
  --------------
  samples:          88
  evals/sample:     1
  time tolerance:   5.00%
  memory tolerance: 1.00%

and then if I uncomment it out (the branch bench_accept_header_uncomment) then I get:

BenchmarkTools.Trial: 
  memory estimate:  62.67 mb
  allocs estimate:  538423
  --------------
  minimum time:     118.871 ms (3.78% GC)
  median time:      122.657 ms (5.41% GC)
  mean time:        122.573 ms (5.20% GC)
  maximum time:     126.492 ms (3.40% GC)
  --------------
  samples:          41
  evals/sample:     1
  time tolerance:   5.00%
  memory tolerance: 1.00%

BenchmarkTools.Trial: 
  memory estimate:  62.67 mb
  allocs estimate:  538423
  --------------
  minimum time:     114.601 ms (3.86% GC)
  median time:      120.697 ms (5.60% GC)
  mean time:        120.593 ms (5.35% GC)
  maximum time:     126.930 ms (3.28% GC)
  --------------
  samples:          42
  evals/sample:     1
  time tolerance:   5.00%
  memory tolerance: 1.00%

Line is just a check: if isempty(tstops). Then some features are added to get to the master branch. It benchmarks at:

BenchmarkTools.Trial: 
  memory estimate:  122.52 mb
  allocs estimate:  4460374
  --------------
  minimum time:     285.688 ms (3.92% GC)
  median time:      288.482 ms (4.54% GC)
  mean time:        288.774 ms (4.40% GC)
  maximum time:     293.832 ms (3.65% GC)
  --------------
  samples:          18
  evals/sample:     1
  time tolerance:   5.00%
  memory tolerance: 1.00%

BenchmarkTools.Trial: 
  memory estimate:  122.52 mb
  allocs estimate:  4460374
  --------------
  minimum time:     281.921 ms (3.75% GC)
  median time:      284.899 ms (4.44% GC)
  mean time:        284.920 ms (4.26% GC)
  maximum time:     289.296 ms (3.51% GC)
  --------------
  samples:          18
  evals/sample:     1
  time tolerance:   5.00%
  memory tolerance: 1.00%

Thus the performance regression is huge! I then go in an comment out some if isempty(tstops) conditionals and get the branch fast_master which gives:

BenchmarkTools.Trial: 
  memory estimate:  120.87 mb
  allocs estimate:  4352465
  --------------
  minimum time:     160.632 ms (6.72% GC)
  median time:      163.469 ms (7.91% GC)
  mean time:        163.188 ms (7.85% GC)
  maximum time:     169.087 ms (6.31% GC)
  --------------
  samples:          31
  evals/sample:     1
  time tolerance:   5.00%
  memory tolerance: 1.00%

BenchmarkTools.Trial: 
  memory estimate:  120.87 mb
  allocs estimate:  4352465
  --------------
  minimum time:     158.549 ms (6.58% GC)
  median time:      161.241 ms (7.50% GC)
  mean time:        161.133 ms (7.59% GC)
  maximum time:     167.428 ms (6.21% GC)
  --------------
  samples:          32
  evals/sample:     1
  time tolerance:   5.00%
  memory tolerance: 1.00%

The last branch is features which adds some features to master, but isn't worth looking at until it's found out why master is so slow and how to get around it.

Stochastic Rosenbrock

https://www.sciencedirect.com/science/article/pii/S0168927416302355

Wiktorsson approximations

https://www.sciencedirect.com/science/article/pii/S0377042706004195 is a good reference.

Error tagging new release

The tag name "2.18.1" is not of the appropriate SemVer form (vX.Y.Z).
cc: @ChrisRackauckas

Non-Diagonal Additive Noise

The SRA algorithms can actually handle this, it's just not implemented. It would be nice to extend the implementation to handle this case.

Inplace Noise

Master does not use inplace noise functions (randn vs randn!). Current master timings need:

DataStructures.jl (my fork) PR-Heaps
ChunkedArrays.jl faster
DiffEqBase master
StochasticDiffEq master

I get something like:

BenchmarkTools.Trial: 
  memory estimate:  54.31 mb
  allocs estimate:  362451
  --------------
  minimum time:     50.256 ms (6.15% GC)
  median time:      50.370 ms (6.21% GC)
  mean time:        50.375 ms (6.20% GC)
  maximum time:     50.465 ms (6.21% GC)
  --------------
  samples:          12
  evals/sample:     1
  time tolerance:   0.01%
  memory tolerance: 1.00%

Inplace noise is with

DiffEqBase inplace
StochasticDiffEq inplace

and I get

BenchmarkTools.Trial: 
  memory estimate:  59.97 mb
  allocs estimate:  428872
  --------------
  minimum time:     63.457 ms (5.45% GC)
  median time:      63.591 ms (5.41% GC)
  mean time:        63.641 ms (5.42% GC)
  maximum time:     64.002 ms (5.50% GC)
  --------------
  samples:          11
  evals/sample:     1
  time tolerance:   0.01%
  memory tolerance: 1.00%

on the test problem

using StochasticDiffEq, DiffEqProblemLibrary
srand(200)
prob = oval2ModelExample(largeFluctuations=true,useBigs=false)
quick_prob = deepcopy(prob)
quick_prob.tspan = (0.0,1.0)

using BenchmarkTools
BenchmarkTools.DEFAULT_PARAMETERS.gcsample = true
BenchmarkTools.DEFAULT_PARAMETERS.time_tolerance = 0.0001
BenchmarkTools.DEFAULT_PARAMETERS.samples = 100
@benchmark begin
  srand(100)
  sol = solve(quick_prob,SRIW1(),dt=(1/2)^(18),progress_steps=Int(1e5),
        adaptivealg=:RSwM3,progress=false,qmax=4,save_timeseries=false,
        timeseries_steps=1000,abstol=1e-5,reltol=1e-3)
end

As a double check to see they are solving the same problem (same randoms), I see

srand(100)
@time sol = solve(quick_prob,SRIW1(),dt=(1/2)^(18),progress_steps=Int(1e5),
      adaptivealg=:RSwM3,progress=false,qmax=4,save_timeseries=false,
      timeseries_steps=1000,abstol=1e-5,reltol=1e-3)

println(sol.u[end])

and get

[0.011503,0.939809,0.00312214,0.00155873,0.0172325,0.0577331,0.237757,0.00134921,0.000238022,4.19916e-5,7.40824e-6,1.307e-6,0.0621091,1.24463,0.0483949,199.901,137.457,0.0177237,0.132583]

for both.

ImplicitRKMilCommute

Just requires doing the same noise term handling in the SDIRK algorithm.

Monte Carlo error

While trying to set up a Monte Carlo calculation based on the examples in the documentation, I encountered the following error:

`julia> using StochasticDiffEq

julia> u=0.0
0.0

julia> f(t,u) = 0.0
f (generic function with 1 method)

julia> g(t,u) = 1.0
g (generic function with 1 method)

julia> prob = SDEProblem(f,g,u,(0.0,1.0))
DiffEqBase.SDEProblem with uType Float64 and tType Float64. In-place: false
timespan: (0.0, 1.0)
u0: 0.0

julia> monte_prob = MonteCarloProblem(prob)
DiffEqBase.MonteCarloProblem with problem DiffEqBase.SDEProblem

julia> sol = solve(monte_prob,num_monte=1000,paralle_type=:threads)
ERROR: MethodError: no method matching solve(::DiffEqBase.MonteCarloProblem{DiffEqBase.SDEProblem{Float64,Float64,false,Void,#f,#g,Void,UniformScaling{Int64},Void},DiffEqBase.##205#211,DiffEqBase.##204#210,DiffEqBase.##206#212,Array{Any,1}}; num_monte=1000, paralle_type=:threads)
Closest candidates are:
solve(::DiffEqBase.AbstractODEProblem{uType,tType,isinplace}, ::DiffEqBase.InternalEuler.FwdEulerAlg; dt, tstops, kwargs...) where {uType, tType, isinplace} at /Users/snirgaz/.julia/v0.6/DiffEqBase/src/internal_euler.jl:21
solve(::DiffEqBase.AbstractODEProblem{uType,tType,isinplace}, ::DiffEqBase.InternalEuler.BwdEulerAlg; dt, tstops, tol, maxiter, kwargs...) where {uType, tType, isinplace} at /Users/snirgaz/.julia/v0.6/DiffEqBase/src/internal_euler.jl:51
solve(::DiffEqBase.AbstractNoiseProblem, ::Void, ::Any...; dt, kwargs...) at /Users/snirgaz/.julia/v0.6/DiffEqNoiseProcess/src/solve.jl:1
...

`
I must be doing something wrong... but it seems like "solve" does not take an MC type.

Any suggestions?

SDEProblem with OrnsteinUhlenbeckProcess! does not work for scalar state

This works:

noise = OrnsteinUhlenbeckProcess!(1.0, 0.0, 1.0, 0.0, [0.0], [0.0])
prob = SDEProblem((t, u) -> -u, (t, u) -> 1.0, [0.0], (0.0, 1.0), noise=noise)
sol = solve(prob, dt=1e-3, adaptive=false)

But this doesn't (notice W0, Z0 and u0 are scalar this time):

noise = OrnsteinUhlenbeckProcess!(1.0, 0.0, 1.0, 0.0, 0.0, 0.0)
prob = SDEProblem((t, u) -> -u, (t, u) -> 1.0, 0.0, (0.0, 1.0), noise=noise)
sol = solve(prob, dt=1e-3, adaptive=false)

Here is the whole stack trace:

ERROR: MethodError: no method matching randn!(::RandomNumbers.Xorshifts.Xoroshiro128Plus, ::Float64)
Closest candidates are:
  randn!(::AbstractRNG, ::SA<:StaticArrays.StaticArray) where SA<:StaticArrays.StaticArray at ~/.julia/v0.6/StaticArrays/src/arraymath.jl:133
  randn!(::AbstractRNG, ::AbstractArray{T,N} where N) where T at random.jl:1283
Stacktrace:
 [1] (::DiffEqNoiseProcess.OrnsteinUhlenbeck!{Float64,Float64,Float64})(::Float64, ::DiffEqNoiseProcess.NoiseProcess{Float64,1,Float64,Float64,Float64,Array{Float64,1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Float64,Float64,Float64},Void,true,DataStructures.Stack{Tuple{Float64,Float64,Float64}},ResettableStacks.ResettableStack{Tuple{Float64,Float64,Float64}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus}, ::Float64, ::RandomNumbers.Xorshifts.Xoroshiro128Plus) at ~/.julia/v0.6/DiffEqNoiseProcess/src/ornstein_uhlenbeck.jl:45
 [2] setup_next_step! at ~/.julia/v0.6/DiffEqNoiseProcess/src/noise_interfaces/noise_process_interface.jl:154 [inlined]
 [3] #init#59(::Float64, ::Int64, ::Bool, ::Array{Float64,1}, ::Array{Float64,1}, ::Array{Float64,1}, ::Void, ::Bool, ::Void, ::Bool, ::Bool, ::Bool, ::Bool, ::Rational{Int64}, ::Float64, ::Float64, ::Rational{Int64}, ::Rational{Int64}, ::Int64, ::Rational{Int64}, ::Bool, ::Rational{Int64}, ::Rational{Int64}, ::Rational{Int64}, ::Float64, ::Float64, ::Float64, ::DiffEqBase.#ODE_DEFAULT_NORM, ::DiffEqBase.#ODE_DEFAULT_UNSTABLE_CHECK, ::DiffEqBase.#ODE_DEFAULT_ISOUTOFDOMAIN, ::Bool, ::Bool, ::Bool, ::Bool, ::Int64, ::Bool, ::DiffEqBase.#ODE_DEFAULT_PROG_MESSAGE, ::String, ::Void, ::Void, ::Bool, ::Bool, ::Bool, ::Array{Any,1}, ::DiffEqBase.#init, ::DiffEqBase.SDEProblem{Float64,Float64,false,DiffEqNoiseProcess.NoiseProcess{Float64,1,Float64,Float64,Float64,Array{Float64,1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Float64,Float64,Float64},Void,true,DataStructures.Stack{Tuple{Float64,Float64,Float64}},ResettableStacks.ResettableStack{Tuple{Float64,Float64,Float64}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},##89#91,##90#92,Void,UniformScaling{Int64},Void}, ::StochasticDiffEq.SRIW1, ::Array{Any,1}, ::Array{Any,1}, ::Array{Any,1}, ::Type{Val{true}}) at ~/.julia/v0.6/StochasticDiffEq/src/solve.jl:377
 [4] (::DiffEqBase.#kw##init)(::Array{Any,1}, ::DiffEqBase.#init, ::DiffEqBase.SDEProblem{Float64,Float64,false,DiffEqNoiseProcess.NoiseProcess{Float64,1,Float64,Float64,Float64,Array{Float64,1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Float64,Float64,Float64},Void,true,DataStructures.Stack{Tuple{Float64,Float64,Float64}},ResettableStacks.ResettableStack{Tuple{Float64,Float64,Float64}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},##89#91,##90#92,Void,UniformScaling{Int64},Void}, ::StochasticDiffEq.SRIW1, ::Array{Any,1}, ::Array{Any,1}, ::Array{Any,1}, ::Type{Val{true}}) at ./<missing>:0
 [5] #solve#58(::Array{Any,1}, ::Function, ::DiffEqBase.SDEProblem{Float64,Float64,false,DiffEqNoiseProcess.NoiseProcess{Float64,1,Float64,Float64,Float64,Array{Float64,1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Float64,Float64,Float64},Void,true,DataStructures.Stack{Tuple{Float64,Float64,Float64}},ResettableStacks.ResettableStack{Tuple{Float64,Float64,Float64}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},##89#91,##90#92,Void,UniformScaling{Int64},Void}, ::StochasticDiffEq.SRIW1, ::Array{Any,1}, ::Array{Any,1}, ::Array{Any,1}, ::Type{Val{true}}) at ~/.julia/v0.6/StochasticDiffEq/src/solve.jl:6
 [6] (::DiffEqBase.#kw##solve)(::Array{Any,1}, ::DiffEqBase.#solve, ::DiffEqBase.SDEProblem{Float64,Float64,false,DiffEqNoiseProcess.NoiseProcess{Float64,1,Float64,Float64,Float64,Array{Float64,1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Float64,Float64,Float64},Void,true,DataStructures.Stack{Tuple{Float64,Float64,Float64}},ResettableStacks.ResettableStack{Tuple{Float64,Float64,Float64}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},##89#91,##90#92,Void,UniformScaling{Int64},Void}, ::StochasticDiffEq.SRIW1, ::Array{Any,1}, ::Array{Any,1}, ::Array{Any,1}, ::Type{Val{true}}) at ./<missing>:0 (repeats 2 times)
 [7] #solve#2(::Bool, ::Array{Any,1}, ::Function, ::DiffEqBase.SDEProblem{Float64,Float64,false,DiffEqNoiseProcess.NoiseProcess{Float64,1,Float64,Float64,Float64,Array{Float64,1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Float64,Float64,Float64},Void,true,DataStructures.Stack{Tuple{Float64,Float64,Float64}},ResettableStacks.ResettableStack{Tuple{Float64,Float64,Float64}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},##89#91,##90#92,Void,UniformScaling{Int64},Void}, ::Void) at ~/.julia/v0.6/DifferentialEquations/src/default_solve.jl:14
 [8] (::DiffEqBase.#kw##solve)(::Array{Any,1}, ::DiffEqBase.#solve, ::DiffEqBase.SDEProblem{Float64,Float64,false,DiffEqNoiseProcess.NoiseProcess{Float64,1,Float64,Float64,Float64,Array{Float64,1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Float64,Float64,Float64},Void,true,DataStructures.Stack{Tuple{Float64,Float64,Float64}},ResettableStacks.ResettableStack{Tuple{Float64,Float64,Float64}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},##89#91,##90#92,Void,UniformScaling{Int64},Void}, ::Void) at ./<missing>:0
 [9] #solve#1(::Bool, ::Array{Any,1}, ::Function, ::DiffEqBase.SDEProblem{Float64,Float64,false,DiffEqNoiseProcess.NoiseProcess{Float64,1,Float64,Float64,Float64,Array{Float64,1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Float64,Float64,Float64},Void,true,DataStructures.Stack{Tuple{Float64,Float64,Float64}},ResettableStacks.ResettableStack{Tuple{Float64,Float64,Float64}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},##89#91,##90#92,Void,UniformScaling{Int64},Void}) at ~/.julia/v0.6/DifferentialEquations/src/default_solve.jl:5
 [10] (::DiffEqBase.#kw##solve)(::Array{Any,1}, ::DiffEqBase.#solve, ::DiffEqBase.SDEProblem{Float64,Float64,false,DiffEqNoiseProcess.NoiseProcess{Float64,1,Float64,Float64,Float64,Array{Float64,1},DiffEqNoiseProcess.OrnsteinUhlenbeck!{Float64,Float64,Float64},Void,true,DataStructures.Stack{Tuple{Float64,Float64,Float64}},ResettableStacks.ResettableStack{Tuple{Float64,Float64,Float64}},DiffEqNoiseProcess.RSWM{:RSwM3,Float64},RandomNumbers.Xorshifts.Xoroshiro128Plus},##89#91,##90#92,Void,UniformScaling{Int64},Void}) at ./<missing>:0
 [11] macro expansion at ./REPL.jl:97 [inlined]
 [12] (::Base.REPL.##1#2{Base.REPL.REPLBackend})() at ./event.jl:73

Tested against current master (d424fcd). DiffEqNoiseProcess.jl is v0.5.0.

I found the issue 29 but not sure if it is related.