sciml / delaydiffeq.jl

Delay differential equation (DDE) solvers in Julia for the SciML scientific machine learning ecosystem. Covers neutral and retarded delay differential equations, and differential-algebraic equations.

DelayDiffEq.jl

DelayDiffEq.jl is a component package in the DifferentialEquations ecosystem. It holds the delay differential equation solvers and utilities. It is built on top of OrdinaryDiffEq to extend those solvers for delay differential equations. While completely independent and usable on its own, users interested in using this functionality should check out DifferentialEquations.jl.

API

DelayDiffEq.jl is part of the JuliaDiffEq common interface, but can be used independently of DifferentialEquations.jl. The only requirement is that the user passes a DelayDiffEq.jl algorithm to solve. For example, we can solve the DDE tutorial from the documentation using the MethodOfSteps(Tsit5()) algorithm:

using DelayDiffEq
const p0 = 0.2; const q0 = 0.3; const v0 = 1; const d0 = 5
const p1 = 0.2; const q1 = 0.3; const v1 = 1; const d1 = 1
const d2 = 1; const beta0 = 1; const beta1 = 1; const tau = 1
function bc_model(du,u,h,p,t)
  du[1] = (v0/(1+beta0*(h(p, t-tau)[3]^2))) * (p0 - q0)*u[1] - d0*u[1]
  du[2] = (v0/(1+beta0*(h(p, t-tau)[3]^2))) * (1 - p0 + q0)*u[1] +
          (v1/(1+beta1*(h(p, t-tau)[3]^2))) * (p1 - q1)*u[2] - d1*u[2]
  du[3] = (v1/(1+beta1*(h(p, t-tau)[3]^2))) * (1 - p1 + q1)*u[2] - d2*u[3]
end
lags = [tau]
h(p, t) = ones(3)
tspan = (0.0,10.0)
u0 = [1.0,1.0,1.0]
prob = DDEProblem(bc_model,u0,h,tspan,constant_lags = lags)
alg = MethodOfSteps(Tsit5())
sol = solve(prob,alg)
using Plots; plot(sol)

Both constant and state-dependent lags are supported. Interfacing with OrdinaryDiffEq.jl for implicit methods for stiff equations is also supported.

Available Solvers

For the list of available solvers, please refer to the DifferentialEquations.jl DDE Solvers page. For options for the solve command, see the common solver options page.

Citing

If you use DelayDiffEq.jl in your work, please cite the following:

@article{DifferentialEquations.jl-2017,
 author = {Rackauckas, Christopher and Nie, Qing},
 doi = {10.5334/jors.151},
 journal = {The Journal of Open Research Software},
 keywords = {Applied Mathematics},
 note = {Exported from https://app.dimensions.ai on 2019/05/05},
 number = {1},
 pages = {},
 title = {DifferentialEquations.jl – A Performant and Feature-Rich Ecosystem for Solving Differential Equations in Julia},
 url = {https://app.dimensions.ai/details/publication/pub.1085583166 and http://openresearchsoftware.metajnl.com/articles/10.5334/jors.151/galley/245/download/},
 volume = {5},
 year = {2017}
}

@article{widmann2022delaydiffeq,
  title={DelayDiffEq: Generating Delay Differential Equation Solvers via Recursive Embedding of Ordinary Differential Equation Solvers},
  author={Widmann, David and Rackauckas, Chris},
  journal={arXiv preprint arXiv:2208.12879},
  year={2022}
}

delaydiffeq.jl's Issues

reeval_internals should fix `uprev` and `uprev2` as well

https://github.com/JuliaDiffEq/DelayDiffEq.jl/blob/master/src/callbacks.jl#L27

No refactorization of Jacobian option

Stiff methods need to solve a linear system. Rosenbrock23 would be a great method for delay diffeqs, but currently it refactorizes the Jacobian with each step. This is only a problem when doing the Picard iterations (max dt too large). But it's simple to fix because the linear solvers already have a flag for whether to change the Jacobian: it just needs to be flipped.

Pass full integrator instead of parameters

As discussed in SciML/DiffEqProblemLibrary.jl#39, especially for the history function it seems reasonable to pass the full integrator as argument instead of only the parameters, i.e., having h(integrator, t) instead of h(p, t) and also f(u, h, integrator, t) instead of f(u, h, p, t). This would enable the user to write generic history functions with correct output types (see the discussion in the PR) and hopefully allow to simplify the implementation in DelayDiffEq.

According to @ChrisRackauckas

we should have a common arg for using integrator instead of p, and then we just need to make every package handle that well.

I think we should approach this issue slightly differently. A user has to decide whether to pass around the integrator or only the parameters already when implementing f (or h), i.e., it is a property that does not depend on the numerical algorithm but rather of the differential equation function. Hence I guess it would make sense to handle this issue by modifying DiffEqFunctions instead of different algorithms. We could replace

abstract type AbstractDiffEqFunction{iip} <: Function end

with

abstract type AbstractDiffEqFunction{iip,unpackparams} <: Function end

and then define, e.g.,

(f::ODEFunction{true,unpackparams})(du, u, integrator, t) where unpackparams = unpackparams ? f.f(du, u, get_p(integrator), t) : f.f(du, u, integrator, t).

In that way, we just have to implement get_p for every integrator (which would be integrator.p by default) and could always pass integrator in every package.

Anderson acceleration

Instead of using Picard iterations, Anderson acceleration should be used for speed and robustness. Thankfully, NLsolve is getting some of that goodness:

JuliaNLSolvers/NLsolve.jl#101

When that goes through, we can swap the solvers over.

Strange autodifferentiation errors

I did some parameter estimation, similar to the test problem in DiffEqParamEstim (https://github.com/JuliaDiffEq/DiffEqParamEstim.jl/blob/master/test/dde_tests.jl):

julia> using DelayDiffEq, RecursiveArrayTools, DiffEqParamEstim, NLopt
julia> function f_lotka(t, u, h, p, du)
           du[1] = 0.5*u[1] - p[1]*u[1]*u[2]
           du[2] = p[2]*h(t-0.5)[1]*u[2] - 0.5*u[2]
       end
julia> h = (t) -> [0.5, 0.5]
julia> u0 = [0.5, 0.5]
julia> tspan = (0., 10.)
julia> f = DDEParameterizedFunction(f_lotka, [0.5, 1.])
julia> prob = DDEProblem(f, h, u0, tspan, [0.5])

As expected

julia> sol = solve(prob, MethodOfSteps(Rosenbrock23()));

successfully computes a solution. However, the parameter estimation problem

julia> t = collect(linspace(0,10,30))
julia> randomized = VectorOfArray([(sol(t[i]) + .03randn(2)) for i in 1:length(t)])
julia> data = convert(Array, randomized)
julia> cost_function = build_loss_objective(prob, MethodOfSteps(Rosenbrock23()),L2Loss(t,data),maxiter=10000)
julia> opt = Opt(:LD_LBFGS, 2)
julia> min_objective!(opt, cost_function.cost_function2)
julia> lower_bounds!(opt, [0.0, 1.])
julia> upper_bounds!(opt, [1.0, 1.])
julia> maxeval!(opt, 10000)
julia> (minf,minx,ret) = NLopt.optimize(opt, [0.2, 1.0]);

fails with the error message

ERROR: The provided configuration (of type ForwardDiff.JacobianConfig{ForwardDiff.Tag{OrdinaryDiffEq.UJacobianWrapper{OrdinaryDiffEq.VectorFReturn{DelayDiffEq.##37#46{DiffEqBase.DDEProblem{Array{Float64,1},Float64,Array{Float64,1},Void,true,DiffEqBase.##285#287{DiffEqBase.DDEProblem{Array{Float64,1},Float64,Array{Float64,1},Void,true,DiffEqBase.DDEParameterizedFunction{true,#f_lotka,Array{Float64,1}},##1#2,Void,UniformScaling{Int64}},Array{Float64,1}},##1#2,Void,UniformScaling{Int64}}},Tuple{Int64}},Float64,Array{Float64,1}},0xb046287d533b082d},Float64,2,Tuple{Array{ForwardDiff.Dual{ForwardDiff.Tag{OrdinaryDiffEq.UJacobianWrapper{OrdinaryDiffEq.VectorFReturn{DelayDiffEq.##37#46{DiffEqBase.DDEProblem{Array{Float64,1},Float64,Array{Float64,1},Void,true,DiffEqBase.##285#287{DiffEqBase.DDEProblem{Array{Float64,1},Float64,Array{Float64,1},Void,true,DiffEqBase.DDEParameterizedFunction{true,#f_lotka,Array{Float64,1}},##1#2,Void,UniformScaling{Int64}},Array{Float64,1}},##1#2,Void,UniformScaling{Int64}}},Tuple{Int64}},Float64,Array{Float64,1}},0xb046287d533b082d},Float64,2},1},Array{ForwardDiff.Dual{ForwardDiff.Tag{OrdinaryDiffEq.UJacobianWrapper{OrdinaryDiffEq.VectorFReturn{DelayDiffEq.##37#46{DiffEqBase.DDEProblem{Array{Float64,1},Float64,Array{Float64,1},Void,true,DiffEqBase.##285#287{DiffEqBase.DDEProblem{Array{Float64,1},Float64,Array{Float64,1},Void,true,DiffEqBase.DDEParameterizedFunction{true,#f_lotka,Array{Float64,1}},##1#2,Void,UniformScaling{Int64}},Array{Float64,1}},##1#2,Void,UniformScaling{Int64}}},Tuple{Int64}},Float64,Array{Float64,1}},0xb046287d533b082d},Float64,2},1}}}) was constructed for a function other than the current target function. ForwardDiff cannot safely perform differentiation in this context; see the following issue for details: https://github.com/JuliaDiff/ForwardDiff.jl/issues/83. You can resolve this problem by constructing and using a configuration with the appropriate target function, e.g. `ForwardDiff.GradientConfig(OrdinaryDiffEq.UJacobianWrapper, x)` 

when trying to perform a step of the Rosenbrock integrator.

I experienced the same problem also on other occasions, but don't have another example at hand right now. Initially I thought it might be caused by building the linked cache in lines
https://github.com/JuliaDiffEq/DelayDiffEq.jl/blob/master/src/utils.jl#L64-L75
but maybe it rather is a problem with the problem generator function or the implementation in DiffEqParamEstim... However, as far as I remember, at least one of the other examples was not related to parameter estimation.

Slowdown from discontinuity declaration

https://discourse.julialang.org/t/declaring-state-dependent-delay-with-ddeproblem-simulation-gets-slower/26048

It would be good to profile this a bit.

CompositeAlgorithm error

Specifying a CompositeAlgorithm currently results in the error:

ERROR: type FPFunctionalCache has no field caches

MWE:

using DelayDiffEq
prob = DDEProblem((du,u,h,p,t)->du[1]=-h(p,t-1)[1],[1.0],(p,t)->[1.0],(0.0,5.0))
choice_function(integrator) = 1
alg = CompositeAlgorithm((Tsit5(),RK4()),choice_function)
solve(prob,MethodOfSteps(alg))

Approximate equality in not-in-place stepping

This commit d3c5bae turned exact checks to approximate checks because the inplace vs outofplace versions now only step within floating point error. This is a minor point would could be corrected again in the future since on v0.6 it was completely error free.

Different solutions for in-place and not in-place functions with unconstrained algorithm

Hi!

As already mentioned in #14, the unconstrained algorithm computes different solutions for problems with in-place and not in-place functions. A small example:

julia> using DelayDiffEq, DiffEqBase, OrdinaryDiffEq, DiffEqProblemLibrary
julia> alg = MethodOfSteps(BS3(); constrained=false)
julia> u₀ = 1.0
julia> prob = prob_dde_1delay_scalar_notinplace(u₀)
julia> sol = solve(prob, alg)
julia> prob2 = prob_dde_1delay(u₀)
julia> sol2 = solve(prob2, alg)
julia> sol.t == sol2.t
false

julia> sol.u == sol2[1, :]
false

julia> sol.errors
Dict{Symbol,Float64} with 3 entries:
  :l∞    => 3.33289e-5
  :final => 1.86351e-5
  :l2    => 1.40652e-5

julia> sol2.errors
Dict{Symbol,Float64} with 3 entries:
  :l∞    => 3.31446e-5
  :final => 1.86005e-5
  :l2    => 1.401e-5

I assume the problem originates from the DDE integrator and the ODE integrator.integrator pointing, among others, both to the same arrays u, uprev, and k in the in-place case. So for some assignments such as https://github.com/JuliaDiffEq/DelayDiffEq.jl/blob/master/src/integrator_interface.jl#L96, which is executed in the first iteration, the value is only updated once for not in-place integrators but immediately in every step for in-place integrators. Moreover, even during calculations in https://github.com/JuliaDiffEq/DelayDiffEq.jl/blob/master/src/integrator_interface.jl#L69 any change of u or k is immediately propagated to integrator.integrator, and thus might also change the interpolation which is not desired, I guess. I was able to reduce the difference of the errors in the example above by completely separating integrator.u and integrator.k also for in-place functions, i.e. by creating a copy of both arrays during initialization and always using recursivecopy! instead of assignments such as integrator.integrator.u = integrator.u, but I got still a tiny difference.

However, I'm still confused by the current implementation of perform_step!. For step sizes below the minimal delay the next step can be directly calculated by the correspond method in OrdinaryDiffEq.jl (https://github.com/JuliaDiffEq/DelayDiffEq.jl/blob/master/src/integrator_interface.jl#L114), whereas for larger step sizes one calculates a first approximation of the next step (also via OrdinaryDiffEq.jl, https://github.com/JuliaDiffEq/DelayDiffEq.jl/blob/master/src/integrator_interface.jl#L69), which is then used for interpolation during the next iteration since integrator.f depends on a HistoryFunction that contains the initial history function, the current solution of integrator.integrator, and the current state of integrator.integrator (which is relevant for interpolation). If this is correct, I have some questions/remarks:

• Can the line https://github.com/JuliaDiffEq/DelayDiffEq.jl/blob/master/src/integrator_interface.jl#L41 removed and https://github.com/JuliaDiffEq/DelayDiffEq.jl/blob/master/src/integrator_interface.jl#L45 changed to integrator.integrator.tprev = integrator.t? To enable extrapolation in the first iteration only integrator.integrator is relevant, so this should be sufficient? And we can remove the redundant line https://github.com/JuliaDiffEq/DelayDiffEq.jl/blob/master/src/integrator_interface.jl#L94, can't we?
• Should integrator.uprev and integrator.integrator.uprev not always equal the same value since also in subsequent iterations the value at time t of the integrator, that was reached in the last step, stays the same? So why do we need https://github.com/JuliaDiffEq/DelayDiffEq.jl/blob/master/src/integrator_interface.jl#L96?
• Why do we need to cache integrator.t, integrator.tprev, and integrator.uprev, even though they are not changed by perform_step! in OrdinaryDiffEq.jl? Can't we remove lines https://github.com/JuliaDiffEq/DelayDiffEq.jl/blob/master/src/integrator_interface.jl#L54-L56 and https://github.com/JuliaDiffEq/DelayDiffEq.jl/blob/master/src/integrator_interface.jl#L107-L109?
• Why are the values such as u, k, fsallast, and EEst that are changed by perform_step! in integrator not copied to integrator.integrator at the end of the method?

Change interface of analytical solutions

As discussed in SciML/DiffEqProblemLibrary.jl#39:

The interface f_analytic(u0, p, t) doesn't make much sense, the analytical solution should depend on the history function as well. I propose using f_analytic(u0, h, p, t), similar to the interface for f(u0, h, p, t).

This change requires PRs to DiffEqBase, DiffEqProblemLibrary, and DelayDiffEq.

Test failures of lazy interpolants

Currently some of the tests for Vern6 and Vern9 fail sometimes (see https://travis-ci.org/JuliaDiffEq/DelayDiffEq.jl/builds). However, the test failures do not depend on the Julia version (they both fail and succeed on Julia 1.0.3 and the nightlies from time to time).

Closer inspection reveals that tests only error when tested on Intel(R) Xeon(R) CPU @ 2.30GHz with libLLVM-6.0.1 (ORCJIT, haswell) but run without problems on Intel(R) Xeon(R) CPU @ 2.50GHz with libLLVM-6.0.0 (ORCJIT, ivybridge) and Intel(R) Xeon(R) CPU @ 2.60GHz with libLLVM-6.0.0 (ORCJIT, sandybridge). So it seems most likely the problems are caused by some floating point differences in the in-place and not in-place implementations of Vern6 and Vern9 that become apparent on specific architectures and maybe are related to #73.

Register

@JuliaRegistrator register()

forward differentiation using `optimize` for DDE

Hello,

I am working on an optimization problem for delay differential equation, and I wanted to use forward differentiation using optimize(...; autodif:=forward), but it doesn't work.
This is the error we get:
MethodError: Cannotconvert` an object of type Discontinuity{ForwardDiff.Dual{ForwardDiff.Tag{getfield(Main, Symbol("#residuals#13")){Int64,Array{Float64,2},Array{Float64,2},Array{Float64,1},Array{Float64,1}},Float64},Float64,6}} to an object of type Discontinuity{Float64}
Closest candidates are:
convert(::Type{S}, !Matched::T<:(Union{CategoricalString{R}, CategoricalValue{T,R} where T} where R)) where {S, T<:(Union{CategoricalString{R}, CategoricalValue{T,R} where T} where R)} at /home/asm/.julia/packages/CategoricalArrays/xjesC/src/value.jl:91
convert(::Type{T}, !Matched::T) where T at essentials.jl:154
Discontinuity{Float64}(::Any, !Matched::Any) where tType at /home/asm/.julia/packages/DelayDiffEq/7Ng3E/src/discontinuity_type.jl:8

Stacktrace:
[1] fill!(::SubArray{Discontinuity{Float64},1,Array{Discontinuity{Float64},1},Tuple{UnitRange{Int64}},true}, ::Discontinuity{ForwardDiff.Dual{ForwardDiff.Tag{getfield(Main, Symbol("#residuals#13")){Int64,Array{Float64,2},Array{Float64,2},Array{Float64,1},Array{Float64,1}},Float64},Float64,6}}) at ./multidimensional.jl:837
[2] __cat(::Array{Discontinuity{Float64},1}, ::Tuple{Int64}, ::Tuple{Bool}, ::Array{Discontinuity{Float64},1}, ::Vararg{Any,N} where N) at ./abstractarray.jl:1412
[3] _cat_t(::Val{1}, ::Type, ::Array{Discontinuity{Float64},1}, ::Vararg{Any,N} where N) at ./abstractarray.jl:1392
[4] #cat_t#103(::Val{1}, ::Function, ::Type{Discontinuity{Float64}}, ::Array{Discontinuity{Float64},1}, ::Vararg{Any,N} where N) at ./abstractarray.jl:1384
[5] (::getfield(Base, Symbol("#kw##cat_t")))(::NamedTuple{(:dims,),Tuple{Val{1}}}, ::typeof(Base.cat_t), ::Type{Discontinuity{Float64}}, ::Array{Discontinuity{Float64},1}, ::Vararg{Any,N} where N) at ./none:0
[6] typed_vcat(::Type, ::Array{Discontinuity{Float64},1}, ::Discontinuity{ForwardDiff.Dual{ForwardDiff.Tag{getfield(Main, Symbol("#residuals#13")){Int64,Array{Float64,2},Array{Float64,2},Array{Float64,1},Array{Float64,1}},Float64},Float64,6}}, ::Discontinuity{ForwardDiff.Dual{ForwardDiff.Tag{getfield(Main, Symbol("#residuals#13")){Int64,Array{Float64,2},Array{Float64,2},Array{Float64,1},Array{Float64,1}},Float64},Float64,6}}) at ./abstractarray.jl:1493
continues...`

Here is our notebook.
Converting the tspan and u0 also did not work, it throws the same error. I would appreciate your advice on this. @ChrisRackauckas
Thank you!

Test errors on Windows 32bit

Hi!

As already mentioned in #17 (comment) and #19 (comment), some tests fail only on Windows 32bit. The problem does not only occur for these PRs but also for the master branch.

I tried to track down this problem but I'm not sure what's exactly the underlying cause. What I found out so far:

• The problem was introduced when I added additional unconstrained tests in
  https://github.com/JuliaDiffEq/DelayDiffEq.jl/blob/master/test/unconstrained.jl#L3-L57
  during the refactoring of the DDE problems (which were moved to DiffEqProblemLibrary). After that change the tests on Windows 32bit failed for the first time: https://ci.appveyor.com/project/ChrisRackauckas/delaydiffeq-jl/build/1.0.77/job/re955uo86yqud8cq
• The additional constrained tests pass without errors
• My first guess was some inaccuracy in the fixed-point iteration (which seemed like the only major difference between these tests), so I replaced the calculation of the residuals with some simple stupid assignments (https://github.com/devmotion/DelayDiffEq.jl/commit/221ff7ae5e901f2cb127e20b77e369e4b9bbc6e5). However, the tests still passed locally but failed on Windows 32bit
• I printed sol.t, sol.u, and sol.errors for these additional constrained and unconstrained tests (https://github.com/devmotion/DelayDiffEq.jl/commit/ceefa53821e8ebefc885c0658f35d1a32e47c705), and ran the tests again, both locally and on Windows 32bit. I got the following output on my computer:

julia> Pkg.test("DelayDiffEq")
INFO: Computing test dependencies for DelayDiffEq...
INFO: No packages to install, update or remove                                                                                     
INFO: Testing DelayDiffEq                                                                                                          
Test Summary:           | Pass  Total
Discontinuity Tree Test |    1      1
  4.993588 seconds (550.96 k allocations: 30.189 MiB, 0.30% gc time)
sol.t = [0.0, 0.1, 0.369144, 0.574108, 0.725764, 0.833981, 0.907633, 0.954618, 0.981867, 0.995353, 1.0, 1.00139, 1.0153, 1.15436, 2.0, 3.0, 3.28148, 3.35698, 3.52603, 3.65314, 3.81, 3.95676, 4.10746, 4.25055, 4.38703, 4.50295, 4.6409, 4.79308, 4.97254, 5.20421, 5.45049, 5.65683, 5.86996, 6.06915, 6.26313, 6.44645, 6.61885, 6.74856, 6.86535, 7.00722, 7.16207, 7.34754, 7.59843, 7.82791, 8.04062, 8.24993, 8.44994, 8.64243, 8.82501, 8.99568, 9.14872, 9.28259, 9.44973, 9.62997, 9.8607, 10.0]
sol.u = [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.998609, 0.984703, 0.845636, 1.11022e-16, -0.5, -0.464102, -0.443863, -0.385907, -0.33314, -0.260523, -0.188274, -0.11314, -0.0438535, 0.0181178, 0.0662641, 0.116931, 0.16321, 0.203556, 0.231992, 0.234213, 0.216656, 0.183803, 0.143529, 0.0992303, 0.0559762, 0.0165551, -0.011077, -0.0338119, -0.058078, -0.0797574, -0.0985111, -0.111028, -0.110233, -0.100506, -0.0843359, -0.0646734, -0.0436413, -0.0232667, -0.0049751, 0.0100674, 0.0217497, 0.0339833, 0.0438881, 0.0513028, 0.0529556]
sol.errors = Dict(:l∞=>2.96934e-5,:final=>1.78371e-5,:l2=>1.35508e-5)
sol.t = [0.0, 0.1, 0.149934, 0.179314, 0.194233, 0.2, 0.201641, 0.218051, 0.273502, 0.306588, 0.324284, 0.331958, 0.333333, 0.333862, 0.339152, 0.39205, 0.4, 0.409802, 0.507823, 0.533333, 0.6, 0.666667, 0.712008, 0.733333, 0.783429, 0.81763, 0.865753, 0.866667, 0.875806, 0.885145, 0.899284, 0.914972, 0.932586, 0.949364, 0.972833, 0.998694, 1.0]
sol.u = [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.998359, 0.981949, 0.926498, 0.893412, 0.875716, 0.868042, 0.866667, 0.865609, 0.855029, 0.749233, 0.733329, 0.713773, 0.523496, 0.475552, 0.357774, 0.25328, 0.190706, 0.164046, 0.108254, 0.07548, 0.0363202, 0.0356527, 0.0291302, 0.0227476, 0.0136143, 0.00420783, -0.00547466, -0.0138657, -0.024304, -0.0341455, -0.0345982]
sol.errors = Dict(:l∞=>4.03291e-6,:final=>1.46663e-6,:l2=>2.22825e-6)
Test Summary:        | Pass  Total
Constrained Timestep |   10     10
 27.103416 seconds (14.49 M allocations: 1.939 GiB, 2.65% gc time)
sol.t = [0.0, 1.0e-6, 1.1e-5, 0.000111, 0.001111, 0.011111, 0.111111, 0.377703, 0.580532, 0.730432, 0.837233, 0.909777, 0.955924, 0.982569, 0.995653, 1.0, 1.00132, 1.01453, 1.14666, 2.0, 3.0, 3.26511, 3.33927, 3.50374, 3.62911, 3.78416, 3.93023, 4.08103, 4.22455, 4.36281, 4.48381, 4.6155, 4.76682, 4.9402, 5.15894, 5.43584, 5.63522, 5.85524, 6.05396, 6.25046, 6.43464, 6.60866, 6.74208, 6.86279, 7.00486, 7.16185, 7.34824, 7.60133, 7.83111, 8.04468, 8.2543, 8.45465, 8.64726, 8.82985, 9.00035, 9.15292, 9.28823, 9.45597, 9.63774, 9.87158, 10.0]
sol.u = [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.998679, 0.985466, 0.853337, 1.11022e-16, -0.5, -0.467964, -0.448955, -0.394426, -0.343608, -0.272909, -0.201494, -0.126241, -0.0561746, 0.00749744, 0.0586436, 0.108196, 0.155992, 0.197464, 0.228516, 0.234825, 0.219217, 0.18646, 0.146831, 0.102202, 0.0587451, 0.018811, -0.0097538, -0.0333402, -0.0577081, -0.0797298, -0.0985655, -0.111087, -0.110144, -0.100245, -0.0839407, -0.0641743, -0.0431001, -0.0227317, -0.00449233, 0.0104579, 0.0222076, 0.0343852, 0.0442345, 0.0515062, 0.0529548]
sol.errors = Dict(:l∞=>3.33279e-5,:final=>1.8635e-5,:l2=>1.40649e-5)
sol.t = [0.0, 1.0e-6, 1.1e-5, 0.000111, 0.001111, 0.011111, 0.111111, 0.156734, 0.183003, 0.195837, 0.2, 0.201278, 0.214062, 0.271031, 0.305181, 0.323593, 0.331714, 0.333333, 0.333933, 0.339929, 0.399889, 0.4, 0.401112, 0.412228, 0.523388, 0.533333, 0.546797, 0.6, 0.666667, 0.712008, 0.733333, 0.783169, 0.817419, 0.865489, 0.866667, 0.87844, 0.888516, 0.904293, 0.920763, 0.938668, 0.958127, 0.982948, 1.0]
sol.u = [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.998722, 0.985938, 0.928969, 0.894819, 0.876407, 0.868286, 0.866667, 0.865467, 0.853475, 0.733556, 0.733333, 0.731111, 0.708953, 0.494169, 0.475554, 0.450694, 0.357776, 0.253282, 0.190708, 0.164048, 0.108521, 0.0756704, 0.0365139, 0.0356535, 0.0273022, 0.0205131, 0.0105298, 0.000924303, -0.00860783, -0.0179362, -0.0283543, -0.0345979]
sol.errors = Dict(:l∞=>1.85086e-6,:final=>1.15435e-6,:l2=>1.04234e-6)
Standard tests complete. Onto idxs tests
  7.321181 seconds (13.41 M allocations: 479.789 MiB, 1.99% gc time)
  5.287681 seconds (9.22 M allocations: 408.107 MiB, 2.27% gc time)
Test Summary:          | Pass  Broken  Total
Unconstrained Timestep |   14       2     16
 40.970641 seconds (48.47 M allocations: 3.004 GiB, 2.72% gc time)
Test Summary: | Pass  Total
Events        |    4      4
  8.789850 seconds (4.54 M allocations: 597.395 MiB, 2.74% gc time)
Test Summary: | Pass  Total
Units         |    6      6
 25.956555 seconds (55.17 M allocations: 1.900 GiB, 3.36% gc time)
Test Summary: | Pass  Total
Unique Times  |    2      2
  4.135222 seconds (2.09 M allocations: 285.094 MiB, 2.50% gc time)
INFO: DelayDiffEq tests passed
INFO: No packages to install, update or remove

The same tests on Windows 32bit yield (among others)

sol.t = [0.0, 0.1, 0.369144, 0.574108, 0.725764, 0.833981, 0.907633, 0.954618, 0.981867, 0.995353, 1.0, 1.00139, 1.0153, 1.15436, 2.0, 3.0, 3.28148, 3.35698, 3.52603, 3.65314, 3.81, 3.95676, 4.10746, 4.25055, 4.38703, 4.50295, 4.6409, 4.79308, 4.97254, 5.20421, 5.45049, 5.65683, 5.86996, 6.06915, 6.26313, 6.44645, 6.61885, 6.74856, 6.86535, 7.00722, 7.16207, 7.34754, 7.59843, 7.82791, 8.04062, 8.24993, 8.44994, 8.64243, 8.82501, 8.99568, 9.14872, 9.28259, 9.44973, 9.62997, 9.8607, 10.0]
sol.u = [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.998609, 0.984703, 0.845636, 1.11022e-16, -0.5, -0.464102, -0.443863, -0.385907, -0.33314, -0.260523, -0.188274, -0.11314, -0.0438535, 0.0181178, 0.0662641, 0.116931, 0.16321, 0.203556, 0.231992, 0.234213, 0.216656, 0.183803, 0.143529, 0.0992303, 0.0559762, 0.0165551, -0.011077, -0.0338119, -0.058078, -0.0797574, -0.0985111, -0.111028, -0.110233, -0.100506, -0.0843359, -0.0646734, -0.0436413, -0.0232667, -0.0049751, 0.0100674, 0.0217497, 0.0339833, 0.0438881, 0.0513028, 0.0529556]
sol.errors = Dict(:final=>1.78371e-5,:l2=>1.35508e-5,:l∞=>2.96934e-5)
sol.t = [0.0, 0.1, 0.149934, 0.179314, 0.194233, 0.2, 0.201641, 0.218051, 0.273502, 0.306588, 0.324284, 0.331958, 0.333333, 0.333862, 0.339152, 0.39205, 0.4, 0.409802, 0.507823, 0.533333, 0.6, 0.666667, 0.712008, 0.733333, 0.783429, 0.81763, 0.865753, 0.866667, 0.875806, 0.885145, 0.899284, 0.914972, 0.932586, 0.949364, 0.972833, 0.998694, 1.0]
sol.u = [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.998359, 0.981949, 0.926498, 0.893412, 0.875716, 0.868042, 0.866667, 0.865609, 0.855029, 0.749233, 0.733329, 0.713773, 0.523496, 0.475552, 0.357774, 0.25328, 0.190706, 0.164046, 0.108254, 0.07548, 0.0363202, 0.0356527, 0.0291302, 0.0227476, 0.0136143, 0.00420783, -0.00547466, -0.0138657, -0.024304, -0.0341455, -0.0345982]
sol.errors = Dict(:final=>1.46663e-6,:l2=>2.22825e-6,:l∞=>4.03291e-6)
Test Summary:        | Pass  Total
Constrained Timestep |   10     10
 92.728570 seconds (14.67 M allocations: 1023.284 MiB, 0.70% gc time)
sol.t = [0.0, 0.0001, 0.0011, 0.0111, 0.1111, 0.377695, 0.580526, 0.730427, 0.83723, 0.909775, 0.955922, 0.982568, 0.995653, 1.0, 1.00132, 1.01453, 1.14667, 2.0, 3.0, 3.26511, 3.33927, 3.50374, 3.62911, 3.78416, 3.93023, 4.08103, 4.22455, 4.36281, 4.48381, 4.6155, 4.76682, 4.9402, 5.15894, 5.43584, 5.63522, 5.85524, 6.05396, 6.25046, 6.43464, 6.60866, 6.74208, 6.86279, 7.00486, 7.16185, 7.34824, 7.60133, 7.83111, 8.04468, 8.2543, 8.45465, 8.64726, 8.82985, 9.00035, 9.15292, 9.28823, 9.45597, 9.63774, 9.87158, 10.0]
sol.u = [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.998679, 0.985465, 0.853329, 1.11022e-16, -0.5, -0.467964, -0.448955, -0.394426, -0.343608, -0.272909, -0.201494, -0.126241, -0.0561746, 0.00749744, 0.0586436, 0.108196, 0.155992, 0.197464, 0.228516, 0.234825, 0.219217, 0.18646, 0.146831, 0.102202, 0.0587451, 0.018811, -0.0097538, -0.0333402, -0.0577081, -0.0797298, -0.0985655, -0.111087, -0.110144, -0.100245, -0.0839407, -0.0641743, -0.0431001, -0.0227317, -0.00449233, 0.0104579, 0.0222076, 0.0343852, 0.0442345, 0.0515062, 0.0529548]
sol.errors = Dict(:final=>1.8635e-5,:l2=>1.43013e-5,:l∞=>3.33279e-5)
sol.t = [0.0, 0.0001, 0.0011, 0.0111, 0.1111, 0.156727, 0.182999, 0.195835, 0.2, 0.201279, 0.214065, 0.271034, 0.305183, 0.323594, 0.331715, 0.333333, 0.333933, 0.339928, 0.399881, 0.4, 0.401189, 0.413074, 0.531929, 0.533333, 0.538473, 0.58987, 0.6, 0.666667, 0.711969, 0.733333, 0.783166, 0.817407, 0.865494, 0.866667, 0.878392, 0.888456, 0.904206, 0.920664, 0.938568, 0.957985, 0.982786, 1.0]
sol.u = [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.998721, 0.985935, 0.928966, 0.894817, 0.876406, 0.868285, 0.866667, 0.865468, 0.853477, 0.733571, 0.733333, 0.730957, 0.707271, 0.478178, 0.475555, 0.466, 0.374815, 0.357777, 0.253283, 0.190759, 0.164048, 0.108524, 0.0756807, 0.0365097, 0.0356528, 0.0273341, 0.0205515, 0.0105816, 0.000978592, -0.00855772, -0.0178725, -0.0282924, -0.0345987]
sol.errors = Dict(:final=>2.0367e-6,:l2=>1.25787e-6,:l∞=>2.3578e-6)
Unconstrained Timestep: Test Failed
  Expression: sol.errors[:l∞] < 2.0e-6
   Evaluated: 2.3578041580193942e-6 < 2.0e-6
Stacktrace:
 [1] include_from_node1(::String) at .\loading.jl:569
 [2] include at .\sysimg.jl:14
 [3] macro expansion at C:\Users\appveyor\.julia\v0.6\DelayDiffEq\test\runtests.jl:6 [inlined]
 [4] macro expansion at .\test.jl:860 [inlined]
 [5] macro expansion at .\util.jl:237 [inlined]
 [6] anonymous at .\<missing>:?
Unconstrained Timestep: Test Failed
  Expression: sol.errors[:final] < 2.0e-6
   Evaluated: 2.0367022636677556e-6 < 2.0e-6
Stacktrace:
 [1] include_from_node1(::String) at .\loading.jl:569
 [2] include at .\sysimg.jl:14
 [3] macro expansion at C:\Users\appveyor\.julia\v0.6\DelayDiffEq\test\runtests.jl:6 [inlined]
 [4] macro expansion at .\test.jl:860 [inlined]
 [5] macro expansion at .\util.jl:237 [inlined]
 [6] anonymous at .\<missing>:?
Standard tests complete. Onto idxs tests
  5.337208 seconds (13.41 M allocations: 319.968 MiB, 3.03% gc time)
  4.102716 seconds (9.22 M allocations: 252.775 MiB, 5.94% gc time)
Test Summary:          | Pass  Fail  Broken  Total
Unconstrained Timestep |   12     2       2     16

The complete log file can be found here. Interestingly, the constrained tests seem to generate the same sol.t, sol.u, and sol.errors; however, in both unconstrained tests already the first time step differs (1.0e-6 vs. 0.0001).

Thus my guess so far is that on Windows 32bit somehow a different initial time step is calculated, which causes all other discrepancies in the solution. However, I do not know why a different time step is calculated in the first place.

Add other DDEFunction features

Just like #137, Wfact and all of that can get added.

saveat doubles the end point

function delay_lotka_volterra(du,u,h,p,t)
  x, y = u
  α, β, δ, γ = p
  du[1] = dx = (α - β*y)*h(p,t-0.1)[1]
  du[2] = dy = (δ*x - γ)*y
end
h(p,t) = ones(eltype(p),2)
prob = DDEProblem(delay_lotka_volterra,[1.0,1.0],h,(0.0,10.0),[2.2, 1.0, 2.0, 0.4],constant_lags=[0.1])
sol = solve(prob,MethodOfSteps(Tsit5()),saveat=0.1)
length(sol)
length(0:0.1:10.0)

Register

@JuliaRegistrator register()

Register

@JuliaRegistrator register()

Generic State-dependent Delay Handling

http://www.radford.edu/~thompson/RP/dklag6.pdf

This method uses event handling to track the delay functions and find t - ai(t,u) hits zero for delay function ai. We can implement this by creating a helper function on solve(::DDEProblem,...) that looks for Number in the lag vector. If all values are Number, we have a constant-lags and we know how to deal with that. If some (or all) of the values are not Number, then we assume that they are functions (to allow callable types) of the form ai(t,u). Then we create the t - ai(t,u) callbacks which don't have an affect! but just cause the solver to walk back to those points.

By doing this, we will no longer need to specialize on ConstantLagDDEProblem because that will be implicit in the lags structure. This setup will actually work for neutral DDEs as well. However, we should find a way of flagging non_neutral. When non_neutral, we can put a counter in the callbacks so we only have to propagate them alg_order many times, making it more efficient.

Lastly, it should have an option to turn derivative tracking off.

This will make all solves handle state-dependent delays though, which will be nice in a different way than the residual control methods.

Test failures on master

Rerunning the tests detected failures on master. Looking into it.

TimeDerivativeWrapper not defined on 1.6.0

julia> using DelayDiffEq
INFO: Precompiling module DelayDiffEq.
ERROR: LoadError: LoadError: UndefVarError: TimeDerivativeWrapper not defined
Stacktrace:
 [1] include_from_node1(::String) at ./loading.jl:576
 [2] include(::String) at ./sysimg.jl:14
 [3] include_from_node1(::String) at ./loading.jl:576
 [4] include(::String) at ./sysimg.jl:14
 [5] anonymous at ./<missing>:2
while loading /Users/solver/.julia/v0.6/DelayDiffEq/src/utils.jl, in expression starting on line 47
while loading /Users/solver/.julia/v0.6/DelayDiffEq/src/DelayDiffEq.jl, in expression starting on line 33
ERROR: Failed to precompile DelayDiffEq to /Users/solver/.julia/lib/v0.6/DelayDiffEq.ji.
Stacktrace:
 [1] compilecache(::String) at ./loading.jl:710
 [2] _require(::Symbol) at ./loading.jl:497
 [3] require(::Symbol) at ./loading.jl:405

Here is my Pkg.status():

julia> Pkg.status()
17 required packages:
 - Atom                          0.6.5
 - BenchmarkTools                0.2.2
 - DifferentialEquations         3.0.2
 - Documenter                    0.12.3
 - GLVisualize                   0.6.1
 - GR                            0.24.0
 - IJulia                        1.6.2
 - Interact                      0.6.2
 - OrdinaryDiffEq                2.29.0
 - PkgDev                        0.1.6
 - Plotly                        0.1.1
 - Plots                         0.13.1
 - Polynomials                   0.1.6
 - QuadGK                        0.1.3
 - QuartzImageIO                 0.3.1
 - RandomMatrices                0.3.0              master
 - SingularIntegralEquations     0.3.0+             development
190 additional packages:
 - ASTInterpreter2               0.1.0
 - AbstractFFTs                  0.2.0
 - AlgebraicDiffEq               0.1.0
 - ApproxFun                     0.7.0+             development
 - AxisAlgorithms                0.2.0
 - AxisArrays                    0.2.0
 - BandedMatrices                0.3.1+             master1
 - BinDeps                       0.7.0
 - Blink                         0.5.4
 - BlockArrays                   0.2.0+             master
 - BlockBandedMatrices           0.0.0-             master (unregistered)
 - BoundaryValueDiffEq           0.3.0
 - BufferedStreams               0.3.3
 - Calculus                      0.2.2
 - CatIndices                    0.1.0
 - ChunkedArrays                 0.1.1
 - CodeTools                     0.4.7
 - Codecs                        0.4.0
 - ColorTypes                    0.6.6
 - ColorVectorSpace              0.5.2
 - Colors                        0.8.2
 - Combinatorics                 0.4.1
 - CommonSubexpressions          0.0.1
 - Compat                        0.35.0
 - ComplexPhasePortrait          0.0.0-             master (unregistered)
 - ComputationalResources        0.2.0
 - Conda                         0.7.0
 - Contour                       0.4.0
 - CoordinateTransformations     0.4.1
 - CustomUnitRanges              0.1.0
 - DataArrays                    0.6.2
 - DataFrames                    0.10.1
 - DataStructures                0.7.2
 - DebuggerFramework             0.1.1
 - DelayDiffEq                   1.6.0
 - DiffBase                      0.3.2
 - DiffEqBase                    2.5.0
 - DiffEqBiological              0.3.0
 - DiffEqCallbacks               0.5.0
 - DiffEqDevTools                0.9.5
 - DiffEqDiffTools               0.2.0
 - DiffEqFinancial               0.3.0
 - DiffEqJump                    0.5.1
 - DiffEqMonteCarlo              0.9.1
 - DiffEqNoiseProcess            0.6.0
 - DiffEqPDEBase                 0.3.5
 - DiffEqParamEstim              0.7.1
 - DiffEqPhysics                 0.0.1
 - DiffEqSensitivity             0.3.0
 - DiffEqUncertainty             0.0.1
 - DiffResults                   0.0.2
 - DiffRules                     0.0.1
 - DimensionalPlotRecipes        0.0.2
 - Distances                     0.5.0
 - DistributedArrays             0.4.0
 - Distributions                 0.15.0
 - DocSeeker                     0.1.0
 - DocStringExtensions           0.4.1
 - Domains                       0.0.0-             approxfun-support (unregistered)
 - DualNumbers                   0.3.0
 - EllipsisNotation              0.3.0
 - FFTViews                      0.1.0
 - FFTW                          0.0.4
 - FastGaussQuadrature           0.3.0+             master
 - FastPolynomialRoots           0.0.0-             master (unregistered)
 - FastTransforms                0.2.2+             master
 - FileIO                        0.5.2
 - FillArrays                    0.0.0-             master (unregistered)
 - FiniteElementDiffEq           0.4.0
 - FixedPointNumbers             0.4.3
 - ForwardDiff                   0.7.0
 - FreeType                      1.3.0
 - FreeTypeAbstraction           0.1.0
 - FunctionWrappers              0.1.0
 - GLAbstraction                 0.5.1
 - GLFW                          1.4.1
 - GLWindow                      0.7.0
 - GSL                           0.3.6
 - GZip                          0.3.0
 - GenericSVD                    0.1.0
 - GeometryTypes                 0.4.3
 - Graphics                      0.2.0
 - Hiccup                        0.1.1
 - HierarchicalMatrices          0.0.2
 - Homebrew                      0.6.1
 - HttpCommon                    0.3.0
 - HttpParser                    0.3.0
 - HttpServer                    0.2.0
 - IdentityRanges                0.1.0
 - ImageAxes                     0.4.0
 - ImageCore                     0.5.0
 - ImageFiltering                0.2.3
 - ImageMetadata                 0.4.0
 - ImageMorphology               0.0.2
 - ImageTransformations          0.4.1
 - Images                        0.11.5
 - IndirectArrays                0.2.0
 - Interpolations                0.7.2
 - IntervalSets                  0.1.1
 - IterTools                     0.1.0
 - IterativeSolvers              0.4.1
 - JSON                          0.16.0
 - Juno                          0.3.2
 - LNR                           0.0.2
 - LaTeXStrings                  0.3.0
 - Lazy                          0.12.0
 - LearnBase                     0.1.6
 - Libz                          0.2.4
 - LineSearches                  3.2.0
 - LinearMaps                    1.0.3
 - LossFunctions                 0.2.0
 - LowRankApprox                 0.1.0
 - LsqFit                        0.3.0
 - MacroTools                    0.4.0
 - MappedArrays                  0.0.7
 - MbedTLS                       0.5.1
 - Measures                      0.1.0
 - Media                         0.3.0
 - MeshIO                        0.1.0
 - ModernGL                      0.2.1
 - MuladdMacro                   0.0.2
 - MultiScaleArrays              0.4.0
 - MultivariateOrthogonalPolynomials 0.0.0-             master (unregistered)
 - Mustache                      0.3.0
 - Mux                           0.2.3
 - NLSolversBase                 3.1.0
 - NLsolve                       0.12.1
 - NaNMath                       0.2.6
 - OffsetArrays                  0.4.2
 - Optim                         0.10.1
 - OptimBase                     0.1.0
 - OscillatoryIntegrals          0.0.0-             master (unregistered)
 - PDMats                        0.7.1
 - Packing                       0.1.0
 - PaddedViews                   0.2.0
 - ParameterizedFunctions        2.3.0
 - Parameters                    0.8.0
 - PenaltyFunctions              0.0.2
 - PlotThemes                    0.1.4
 - PlotUtils                     0.4.4
 - PlotlyJS                      0.7.1
 - PositiveFactorizations        0.1.0
 - Primes                        0.2.0
 - ProgressMeter                 0.5.1
 - Quaternions                   0.3.0
 - RandomNumbers                 0.1.1
 - RangeArrays                   0.2.0
 - RatFun                        0.0.0-             master (unregistered)
 - Ratios                        0.2.0
 - Reactive                      0.6.0
 - RecipesBase                   0.2.3
 - RecursiveArrayTools           0.12.4
 - Reexport                      0.0.3
 - Requests                      0.5.1
 - Requires                      0.4.3
 - ResettableStacks              0.2.0
 - RiemannHilbert                0.0.0-             master (unregistered)
 - Rmath                         0.3.0
 - Roots                         0.4.1
 - Rotations                     0.6.1
 - SHA                           0.5.2
 - SIUnits                       0.1.0
 - SO                            0.0.0-             master (unregistered, dirty)
 - ShowItLikeYouBuildIt          0.2.0
 - Showoff                       0.1.1
 - SignedDistanceFields          0.2.0
 - SimpleTraits                  0.5.1
 - SortingAlgorithms             0.2.0
 - SpecialFunctions              0.3.4
 - SpectralMeasures              0.0.0-             master (unregistered)
 - StaticArrays                  0.6.6
 - StatsBase                     0.19.1
 - StatsFuns                     0.5.0
 - SteadyStateDiffEq             0.1.0
 - StochasticDiffEq              2.15.2
 - StringDistances               0.2.0
 - SugarBLAS                     0.0.4
 - Sundials                      0.14.3
 - SymEngine                     0.2.0
 - TexExtensions                 0.1.0
 - TiledIteration                0.1.0
 - ToeplitzMatrices              0.3.0+             master
 - Tokenize                      0.4.2
 - URIParser                     0.2.0
 - UnicodeFun                    0.1.0
 - UnicodePlots                  0.2.5
 - VectorizedRoutines            0.0.2
 - WebSockets                    0.3.0
 - WoodburyMatrices              0.2.2
 - ZMQ                           0.5.1

Residual Control Methods for State-Dependent Delays

Repost: SciML/OrdinaryDiffEq.jl#26

Shampine's paper on this: http://faculty.smu.edu/shampine/residuals.pdf

That one just uses a residual control method. We can add this residual error estimator to RK4() to make it adaptive and then it can solve state-dependent delay equations without any further modification. This is probably the one to start with.

Register v5.3.0

@JuliaRegistrator register()

CompositeAlgorithm and switching compatibility

This library is almost compatible with the CompositeAlgorithm. The issue is just the return. The solution of the integrator.integrator is appropriately the CompositeSolution, but the integrator's solution is just an ODESolution. The former is required because it has the alg_choice field which makes the interpolant match the chosen algorithm.

Estimations of the interpolation error

I would like to try some neat heuristic that's used in RADAR5 and that I assume could be helpful for non-smooth solutions such as the artificial example in RADAR5 due to Hairer and Guglielmi, which is mentioned as third example in https://www.sciencedirect.com/science/article/pii/S147466701736929X. I'm not able to solve this problem properly - I can compute a solution with TRBDF2, SDIRK2, or KenCarp4 that converges to the correct steady state but instead of the expected oscillating non-smooth behaviour DelayDiffEq just picks one branch. As Guglielmi writes, "given the non smoothness of the solution, the control of the error in the dense output is crucial". I think we might not perform well since we do not control the interpolation well enough.

I think, one limitation of the algorithm right now is that we (mainly) check the convergence of the fixed-point iteration and the local error estimates at times t + dt for accepting/rejecting the steps, but we would like to have an estimate for the error of the continuous interpolation as well. The main idea that's used in RADAR5 is to take two interpolations and to check how close they are. It's just a heuristic (if I remember correctly) but still it might help RADAR5 to outperform DelayDiffEq on this class of problems.

As far as I know, currently there's no easy way for using different interpolation methods in the same time interval. I assume that addressing this issue would (mainly) require upstream changes in OrdinaryDiffEq and DiffEqBase that introduce proper interpolation types that somehow can be used in a modular and exchangeable way.

State dependent problem fails

The following example fails with the error: ERROR: UndefVarError: find_first_continuous_callback not defined

using DelayDiffEq
f(du,u,h,p,t) = du .= h(p,t-delay)
delay = 1.0
problem = DDEProblem(f,[1.0],(p,t)->[1.0],(0.0,1.0),dependent_lags = delay)
sol = solve(problem,MethodOfSteps(RK4()))

It works fine with constant_lags.

Test function also gives a similar error:

constant lag function: Error During Test at /Users/h/.julia/packages/DelayDiffEq/M0fUp/test/dependent_delays.jl:13
  Got exception outside of a @test
  UndefVarError: find_first_continuous_callback not defined

Register

@JuliaRegistrator register()

Test Rosenbrock methods on RADAR5_waltman

Ref: SciML/OrdinaryDiffEq.jl#941

Event Handling Test Failure

https://travis-ci.org/github/SciML/DelayDiffEq.jl/builds/684495227?utm_source=github_status&utm_medium=notification

Package compatibility caps

Ref: https://discourse.julialang.org/t/package-compatibility-caps/15301

ForwardDiff on DelayDiffEq

Found by DiffEqFlux tests:

using Flux, DiffEqFlux, DelayDiffEq, Test

## Setup DDE to optimize
function delay_lotka_volterra(du,u,h,p,t)
  x, y = u
  α, β, δ, γ = p
  du[1] = dx = (α - β*y)*h(p,t-0.1)[1]
  du[2] = dy = (δ*x - γ)*y
end
h(p,t) = ones(eltype(p),2)
prob = DDEProblem(delay_lotka_volterra,[1.0,1.0],h,(0.0,10.0),constant_lags=[0.1])
p = param([2.2, 1.0, 2.0, 0.4])
function predict_fd_dde()
  diffeq_fd(p,sol->sol[1,:],101,prob,MethodOfSteps(Tsit5()),saveat=0.0:0.1:10.0)
end
loss_fd_dde() = sum(abs2,x-1 for x in predict_fd_dde())
@test_broken loss_fd_dde()
@test_broken Flux.back!(loss_fd_dde())

Slow compile times

While benchmarking some DDE algorithms I noticed almost unbearable compile times (see SciML/DiffEqDevTools.jl#42).

I just copied the example from that issue to a file "benchmark.jl"

using DelayDiffEq, DiffEqDevTools, DiffEqProblemLibrary.DDEProblemLibrary
DDEProblemLibrary.importddeproblems()

const prob = DDEProblemLibrary.prob_dde_mackey
const sol = solve(prob, MethodOfSteps(Vern9()); dtmax = 0.1, reltol = 1e-14, abstol = 1e-14)
const test_sol = TestSolution(sol)

function test(alg, abstols, reltols)
    @time WorkPrecisionSet(prob, abstols, reltols, [Dict(:alg => MethodOfSteps(alg))];
                           appxsol = test_sol, maxiters = 100_000, error_estimate = :final)

    @time WorkPrecisionSet(prob, abstols, reltols, [Dict(:alg => MethodOfSteps(alg))];
                           appxsol = test_sol, maxiters = 100_000, error_estimate = :final)

    nothing
end

test(BS3(), [1e-4, 1e-5], [1e-1, 1e-2])

and ran

julia --project=. --startup-file=no -e "include(\"benchmark.jl\")"

for different versions of DelayDiffEq, OrdinaryDiffEq, DiffEqBase, DiffEqDevTools, and DiffEqProblemLibrary (using Julia 1.1.1).

With versions

• DelayDiffEq 5.2.0
• OrdinaryDiffEq 5.6.0
• DiffEqBase 5.7.0
• DiffEqDevTools 2.8.0
• DiffEqProblemLibrary 4.1.0
• DiffEqMonteCarlo 0.14.0

(basically, the versions from beginning of May that were use in the Mackey and Glass benchmarks http://juliadiffeq.org/DiffEqBenchmarks.jl/html/NonStiffDDE/Mackey_Glass_wpd.html), I get

 31.279454 seconds (210.32 M allocations: 20.587 GiB, 11.69% gc time)
 12.358761 seconds (153.44 M allocations: 2.817 GiB, 15.91% gc time)

However, with

• DelayDiffEq master
• OrdinaryDiffEq master
• DiffEqBase master
• DiffEqDevTools master
• DiffEqProblemLibrary master

the timings are

167.584735 seconds (244.37 M allocations: 93.351 GiB, 9.82% gc time)
  0.419306 seconds (1.06 M allocations: 21.572 MiB, 3.71% gc time)

Fixing for NLsolve update

SciML/OrdinaryDiffEq.jl#217 seems like it caused an issue. I worked through some of it but couldn't find out how to handle uhold changes:

SDIRK Integrators: Error During Test
  Got an exception of type LoadError outside of a @test
  LoadError: MethodError: no method matching reshape_f(::OrdinaryDiffEq.ImplicitRHS_Scalar{DelayDiffEq.##38#49{DiffEqBase.DDEProblem{Float64,Float64,Array{Float64,1},Void,false,DiffEqProblemLibrary.#f_1delay_notinplace,DiffEqProblemLibrary.##28#29{Float64},Void,UniformScaling{Int64}}},Float64,Float64}, ::Float64)
  Closest candidates are:
    reshape_f(::Any, !Matched::AbstractArray) at C:\Users\Chris\.julia\v0.6\NLsolve\src\utils.jl:106
  Stacktrace:
   [1] autodiff_setup(::Function, ::Float64, ::Type{Val{1}}) at C:\Users\Chris\.julia\v0.6\OrdinaryDiffEq\src\misc_utils.jl:31
   [2] (::OrdinaryDiffEq.NLSOLVEJL_SETUP{0,true})(::Type{Val{:init}}, ::Function, ::Float64) at C:\Users\Chris\.julia\v0.6\OrdinaryDiffEq\src\misc_utils.jl:57
   [3] build_linked_cache(::OrdinaryDiffEq.GenericImplicitEulerConstantCache{Array{Float64,1},OrdinaryDiffEq.ImplicitRHS_Scalar{DiffEqProblemLibrary.#f_1delay_notinplace,Float64,Float64},NLsolve.DifferentiableMultivariateFunction{NLsolve.#fvec!#27{OrdinaryDiffEq.ImplicitRHS_Scalar{DiffEqProblemLibrary.#f_1delay_notinplace,Float64,Float64},Array{Float64,1}},OrdinaryDiffEq.##2#5{OrdinaryDiffEq.##1#4{NLsolve.#fvec!#27{OrdinaryDiffEq.ImplicitRHS_Scalar{DiffEqProblemLibrary.#f_1delay_notinplace,Float64,Float64},Array{Float64,1}}},Array{Float64,1},ForwardDiff.JacobianConfig{ForwardDiff.Tag{Void,0xb046287d533b082d},Float64,1,Tuple{Array{ForwardDiff.Dual{ForwardDiff.Tag{Void,0xb046287d533b082d},Float64,1},1},Array{ForwardDiff.Dual{ForwardDiff.Tag{Void,0xb046287d533b082d},Float64,1},1}}}},OrdinaryDiffEq.##3#6{OrdinaryDiffEq.##1#4{NLsolve.#fvec!#27{OrdinaryDiffEq.ImplicitRHS_Scalar{DiffEqProblemLibrary.#f_1delay_notinplace,Float64,Float64},Array{Float64,1}}},Array{Float64,1},ForwardDiff.JacobianConfig{ForwardDiff.Tag{Void,0xb046287d533b082d},Float64,1,Tuple{Array{ForwardDiff.Dual{ForwardDiff.Tag{Void,0xb046287d533b082d},Float64,1},1},Array{ForwardDiff.Dual{ForwardDiff.Tag{Void,0xb046287d533b082d},Float64,1},1}}}}}}, ::OrdinaryDiffEq.GenericImplicitEuler{OrdinaryDiffEq.NLSOLVEJL_SETUP{0,true}}, ::Float64, ::Float64, ::Float64, ::DelayDiffEq.##38#49{DiffEqBase.DDEProblem{Float64,Float64,Array{Float64,1},Void,false,DiffEqProblemLibrary.#f_1delay_notinplace,DiffEqProblemLibrary.##28#29{Float64},Void,UniformScaling{Int64}}}, ::Float64, ::Float64) at C:\Users\Chris\.julia\v0.6\DelayDiffEq\src\utils.jl:23
   [4] #init#31(::Array{DelayDiffEq.Discontinuity{Float64},1}, ::Float64, ::Float64, ::Array{Float64,1}, ::Void, ::Bool, ::Bool, ::Bool, ::Bool, ::Bool, ::Int64, ::Float64, ::Int64, ::Int64, ::Bool, ::Void, ::Array{Any,1}, ::DiffEqBase.#init, ::DiffEqBase.DDEProblem{Float64,Float64,Array{Float64,1},Void,false,DiffEqProblemLibrary.#f_1delay_notinplace,DiffEqProblemLibrary.##28#29{Float64},Void,UniformScaling{Int64}}, ::DelayDiffEq.MethodOfSteps{OrdinaryDiffEq.GenericImplicitEuler{OrdinaryDiffEq.NLSOLVEJL_SETUP{0,true}},Void,Void,Void,false}, ::Array{Float64,1}, ::Array{Float64,1}, ::Array{Any,1}) at C:\Users\Chris\.julia\v0.6\DelayDiffEq\src\solve.jl:218
   [5] init(::DiffEqBase.DDEProblem{Float64,Float64,Array{Float64,1},Void,false,DiffEqProblemLibrary.#f_1delay_notinplace,DiffEqProblemLibrary.##28#29{Float64},Void,UniformScaling{Int64}}, ::DelayDiffEq.MethodOfSteps{OrdinaryDiffEq.GenericImplicitEuler{OrdinaryDiffEq.NLSOLVEJL_SETUP{0,true}},Void,Void,Void,false}, ::Array{Float64,1}, ::Array{Float64,1}, ::Array{Any,1}) at C:\Users\Chris\.julia\v0.6\DelayDiffEq\src\solve.jl:17
   [6] #solve#58(::Array{Any,1}, ::Function, ::DiffEqBase.DDEProblem{Float64,Float64,Array{Float64,1},Void,false,DiffEqProblemLibrary.#f_1delay_notinplace,DiffEqProblemLibrary.##28#29{Float64},Void,UniformScaling{Int64}}, ::DelayDiffEq.MethodOfSteps{OrdinaryDiffEq.GenericImplicitEuler{OrdinaryDiffEq.NLSOLVEJL_SETUP{0,true}},Void,Void,Void,false}, ::Array{Float64,1}, ::Array{Float64,1}, ::Array{Any,1}) at C:\Users\Chris\.julia\v0.6\DelayDiffEq\src\solve.jl:429
   [7] solve(::DiffEqBase.DDEProblem{Float64,Float64,Array{Float64,1},Void,false,DiffEqProblemLibrary.#f_1delay_notinplace,DiffEqProblemLibrary.##28#29{Float64},Void,UniformScaling{Int64}}, ::DelayDiffEq.MethodOfSteps{OrdinaryDiffEq.GenericImplicitEuler{OrdinaryDiffEq.NLSOLVEJL_SETUP{0,true}},Void,Void,Void,false}) at C:\Users\Chris\.julia\v0.6\DelayDiffEq\src\solve.jl:429
   [8] macro expansion at C:\Users\Chris\.julia\v0.6\DelayDiffEq\test\sdirk_integrators.jl:28 [inlined]
   [9] anonymous at .\<missing>:?
   [10] include_from_node1(::String) at .\loading.jl:576
   [11] include(::String) at .\sysimg.jl:14
   [12] macro expansion at C:\Users\Chris\.julia\v0.6\DelayDiffEq\test\runtests.jl:15 [inlined]
   [13] macro expansion at .\test.jl:860 [inlined]
   [14] macro expansion at .\util.jl:237 [inlined]
   [15] anonymous at .\<missing>:?
   [16] include_from_node1(::String) at .\loading.jl:576
   [17] include(::String) at .\sysimg.jl:14
   [18] process_options(::Base.JLOptions) at .\client.jl:305
   [19] _start() at .\client.jl:371
  while loading C:\Users\Chris\.julia\v0.6\DelayDiffEq\test\sdirk_integrators.jl, in expression starting on line 21
Test Summary:     | Error  Total
SDIRK Integrators |     1      1

Addition of higher order discontinuities to mesh required

I'm just writing an introduction to numerical algorithms for DDE solving for my master's thesis, and therefore I'm studying Bellen and Zennaro, Numerical Methods for Delay Differential Equations. In my opinion we have to include additional discontinuities in the mesh in order to guarantee global order min{p,q+1} of the resulting DDE method where p is the order of the underlying ODE method and q the order of continuous extension, i.e. interpolant.

In DelayDiffEq a discontinuity of order k is defined as jump discontinuity in the kth derivative of the solution to the DDE, following Shampine and Thompson, Numerical Solution of Delay Differential Equations. According to Definition 2.1.3 by Bellen and Zennaro

A discontinuity point ξ is said to be of order k if y^{(s)}(ξ) exists for s = 0,...,k and y^{(k)}(ξ) is Lipschitz continuous at ξ

which is, in my opinion, not well-defined; but it is clarified in Bellen et al., Recent trends in numerical solution of retarded functional differential equations, Acta numerica 2009:

Every point where some derivative y^{(s)} jumps will be called discontinuity point or breaking point. We also say that a breaking point ξ has order k if the solution is C^k-continuous at ξ. In particular, by k = -1 we mean that the solution is discontinuous at ξ.

As stressed in both publications by Bellen,

for a method to be of order p, we usually ask the solution to be at least C^{p+1}-continuous on [t_n, t_{n+1}]. Therefore, for the success of the method, it is crucial to include among the mesh points all discontinuities of y^{(s)} at least for s = 0,1,..., p+1.

Indeed, proofs of convergence results for all algorithms (for constant delays, time dependent delays, and state dependent delays) require the solution between two adjacent mesh points to be at least C^{p+1}-continuous, assuming the applied ODE method is of order p. However, because of the different definition of order of discontinuities, the algorithms and theorems by Bellen et al. require "only" the addition of discontinuities up to order p to the mesh - although with our and Shampine's definition one has to include discontinuities up to order p+1.

Thus I guess, the current implementation of discontinuity tracking needs to be fixed and we also have to add discontinuities in the p+1th derivative to the mesh. I added a fix in https://github.com/devmotion/DelayDiffEq.jl/tree/additional_discontinuities, which also needs some tests to be adjusted.

Calculation of dt after initialization

I guess we should move the calculation of dt after the initialization of callbacks, just as in SciML/OrdinaryDiffEq.jl@3cadf5b.

However, since (also in OrdinaryDiffEq!) there are some assignments based on dt (such as dtcache and dtproposed) prior to this calculation - should we update them? Also the cache and the linked cache in DelayDiffEq are created with the "old" dt value - most of the caches do not actually use dt but in
https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl/blob/master/src/caches/linear_nonlinear_caches.jl
expA = expm(A, dt) has to be recalculated I guess? So these considerations not only affect DelayDiffEq but also OrdinaryDiffEq.

Mass matrix

The latest mass matrix changes in DiffEqBase break DelayDiffEq.

Instability when integrating backwards

The following bails with an instability warning:

using DelayDiffEq
f(du,u,h,p,t) = du .= h(p,t+1)
sol = solve(DDEProblem(f,[1.0],(p,t)->[1.0],(1.0,0.0)),MethodOfSteps(RK4()))

The solution expected here is the straight line u(t) = t.

DDE callbacks are not initialized

Please see SciML/JumpProcesses.jl#71 (comment).

Explicitly specify Jacobian for DDE problems

The explicit specification of Jacobian for DDE problems is not yet supported?
e.g., for ODEs we can provide function
f(::Type{Val{:jac}},J,u,p,t)
but there is no way to specify Jacobian dependent on history function h, like
f(::Type{Val{:jac}},J,u,h,p,t)

Some ideas

In my opinion, DelayDiffEq is a lot more complicated than it should be, which also makes analysis and fixes more difficult. Hence I started to think about how the current situation could be improved.

Status quo

Currently, when initializing a new DDEIntegrator an ODEIntegrator is created (but not initialized) whose function, however, can never be evaluated since it has the wrong number of arguments. Hence in a quite confusing way it is wrapped, together with its solution, in a HistoryFunction, which itself is wrapped in a new differential equation of the form needed for OrdinaryDiffEq. Based on this, a DDEIntegrator is defined that duplicates all fields of an ODEIntegrator and uses this closure for function evaluations; hence OrdinaryDiffEq can be used for integration; in a quite advanced way the initial ODEIntegrator is updated to ensure a correct history function. Moreover, to reduce memory the caches of this ODEIntegrator are transferred to the DDEIntegrator, and the solution of the integrator is shortened already during integration if possible.

Ideas

• Create one fully functional ODEIntegrator: This is currently not possible since the DDE function depends on argument h as well which itself depends on the interpolation of the ODEIntegrator that does not exist yet; possible solution:
  • Wrap h in a HistoryFunction that can be called as (::HistoryFunction)(integrator, t, ::Val{deriv}; idxs = nothing) (and so on):
    1. does not require the non-existing integrator at creation time
    2. requires the possibility to pass integrator instead of parameters p during integration as it was discussed already when the breaking changes were introduced in January
    3. passing integrator to the wrapper would still allow to only pass integrator.p to the wrapped initial function h
  • Wrap f and this HistoryFunction (probably together with analytic solution, gradient, Jacobian etc.) in a DDEFunction <: AbstractDiffEqFunction
    1. allows to define (::DDEFunction)(du, u, integrator, t) (and so on), as required by OrdinaryDiffEq
    2. requires to pass integrator instead of parameters p
    3. would still allow to pass only integrator.p to f
    4. would group all different functions such as f and h that belong together (e.g. usually the analytic solution depends on both)
• Wrap it inside a DDEIntegrator which contains no duplicate fields:
  • currently almost every update of ODEIntegrator breaks DDEIntegrator
• Use ODEIntegrator for integration and DDEIntegrator mostly for its control, i.e. for resets and fixed-point iteration:
  • from previous experiences, the most difficult part would be to implement intra-/extrapolations and fixed-point iterations correctly
• Add possibility to only save dense output of certain indices:
  • currently a very complicated algorithm exists to reduce the solution in special cases
  • based on the implementation of RADAR5, one could (alternatively) save dense output only for certain indices (but still save solution at all or given time points for all or given indices)
  • would be simpler probably, and would also work for state-dependent delays

Bizzare error with DDE system

I don't know if this the right venue to report this. If there is a better place please let me know and close the issue. Here is my problem:

I have the system of delay-differential equations implemented here, more precisely in the function all_model.
When I try to run this code it fails with

ERROR: BoundsError: attempt to access 1-element Array{Float64,1} at index [2]

If it is of any interest here is also the stacktrace.

The error appears only when h0(p,t)[1]=0.

an issue for time-depent lags dde of a common model in laser

Dear All,

I'm involved to simulate a result from a dynamic model in laser to match with measured experimental results and I fail to obtain expected result in Matlab (using dde23, ddesd). Therefore, I transfer to use Julia and face a problem as well (cannot even let laser lasing...)

The model describes an optical feedback effect back in laser and the model (rate euqations) are described here, on page 581 for this paper:
https://www.osapublishing.org/aop/abstract.cfm?URI=aop-7-3-570
and this is a screenshot for the rate equation:

The three differential equations above the the photon density, carrier density and phase changing with time.

$$\tau_{ext}$$ is the time-dependent lag in a harmonic motion: $$2L/c + 2lambda/c * sin(2\pi f t)$$

below is the code I write in Julia and refer to official example on documentation:
`

using DifferentialEquations
using Plots
const c = 3e+8;
L_ext_0 = 0.8;
freq_oscillation = 1 / 1e-6; # 1000 ns
lambda = 850e-9; # 850 nm
A = lambda;
 
# time dependent lag
tau(u, p, t) = 2 * L_ext_0 / c + 2 * A / c * sin(2 * pi * freq_oscillation * t)
 
# const
const eta_i = 0.8; I = 1.5 * 30e-3; const q = 1.602e-19; const V = 0.9e-16;
const Gamma = 0.25; const V = 0.9e-16; const tau_n = 1.4e-9;
const n_g = 3.6; const L = 290e-6; const R1 = ((1 - n_g) / (1 + n_g))^2; const R2 = R1;
const alpha_i = 1500; const alpha_m = 1 / (2 * L) * log(1 / (R1 * R2));
const v_g = c / n_g;
const tau_p = 1 / (v_g * (alpha_i + alpha_m));

const beta_sp = 2e-4; const tau_sp = 2.3e-9;

Attenuation = -40; # dB
epsilon_loss = 10^(Attenuation / 10);
R = 0.7;
kappa = epsilon_loss * (1 - R2) * sqrt( R / R2 );
tau_in = (2 * L) / v_g;
kappa_tide = kappa / tau_in;

freq_1 = c / lambda;
omega_th_1 = 2 * pi * freq_1;
freq_2 = freq_1 + c / (2 * n_g * L);
omega_th_2 = 2 * pi * freq_2;

const alpha_Henry = 4.6;
const N_tr = 2e+24;
const a = 0.75e-20;

# define rre function
function rre_multimode(du, u , h, p, t)
    G1 = v_g * a * (u[1] - N_tr)
    du[1] = (eta_i * I) / (q * V) - u[1] / tau_n - (G1 * u[2])
    du[2] = Gamma * G1 * u[2] - u[2] / tau_p + Gamma * beta_sp * u[1] / tau_sp + 2 * kappa_tide * sqrt(u[2] * h(p, t-tau(u, p, t))[2]) * cos(omega_th_1 * tau(u, p, t) + u[3] - h(p, t-tau(u, p, t))[3])
    du[3] = 0.5 * alpha_Henry * Gamma * G1 - 0.5 * alpha_Henry / tau_p - kappa_tide * sqrt(h(p, t-tau(u, p, t))[2] / u[2]) * sin(omega_th_1 * tau(u, p, t) + u[3] - h(p, t-tau(u, p, t))[3])
    #du[2] = Gamma * G1 * u[2] - u[2] / tau_p + Gamma * beta_sp * u[1] / tau_sp + 2 * kappa_tide * sqrt( complex(u[2] * h(p, t-tau(u, p, t)) )[2]) * cos(omega_th_1 * tau(u, p, t) + u[3] - h(p, t-tau(u, p, t))[3])
    #du[3] = 0.5 * alpha_Henry * Gamma * G1 - 0.5 * alpha_Henry / tau_p - kappa_tide * sqrt( complex(h(p, t-tau(u, p, t))[2] / u[2]) ) * sin(omega_th_1 * tau(u, p, t) + u[3] - h(p, t-tau(u, p, t))[3])
end

h(p, t) = [1, 1, pi, 1, pi]
tspan = (0.0, 1000e-9)
u0 = Float64[1, 1, pi, 1, pi]
d_lags = ((u,p,t) -> tau(u,p,t),)
#d_lags = (tau,)

prob = DDEProblem(rre_multimode, u0, h, tspan, dependent_lags = d_lags)
alg = MethodOfSteps(Tsit5())
sol = solve(prob,alg, adaptive = false, dt=1e-11, dtmax = 1e-11, reltol = 1e-8, abstol= 1e-9, force_dtmin = true)

plot(sol.t, sol[2,:])

but I cannot get the result even as matlab did from this dynamic model (rate euqation).

Below is the matlab simulation result (simulated by ddesd solver):

Even the above beautiful waveform is not the same as what I simulate from a steady-state equation (derived from rate equation above), which looks like this:

In experiment, we can observe the tilt and sharp fringes like the picture above and this is the expectation.

I appreciate guys who replied to my previous problem and specially for Chris who replied me in email.

I have make my effort during this vacation and try to avoid any trivial mistake and till now I cannot figure out what is the problem behind. I cannot understand why matlab gives different simulation results as well.

Any help is appreciated and for this issue, I think Julia DDE solver for this laser model doesn't work for some reasons.

Thank you very much!
Xiao

Robertson problem

I started playing around a bit more with stiff DDEs, and think www.unige.ch/~hairer/radar5-v2.1.tar and accompanying publications contain some nice examples (which should be added to DiffEqProblemLibrary as well, I guess).

However, my first tests show that there are some problems with stiff problems. The first problem is a modified Robertson problem, taken from "The numerical integration of stiff delay differential equations" by Guglielmi; I only changed the integration interval from [0.0, 1e9] to [0.0, 1e5] since otherwise maxiters has to be increased (maybe the number of iterations could be reduced in general by changing some defaults?):

using DelayDiffEq

function f_dde_rober1(du, u, h, p, t)
    # compute repeating terms
    x = 4e-2 * u[1]
    y = 1e4 * u[2] * u[3]
    z = 3e7 * (1 - h(p, t - 1; idxs = 2)) * u[2]^2

    # compute derivatives
    du[1] = y - x
    du[2] = x - y - z
    du[3] = z

    nothing
end
# simplified history function, hence `initial_order` has to be set
h_dde_rober(p, t; idxs = 2) = 0.0

prob_dde_rober1 = DDEProblem(f_dde_rober1, [1.0, 0.0, 0.0], h_dde_rober,
                             (0.0, 1e5);
                             constant_lags = [1])

Using Tsit5 integrations is stopped at around t = 627 since the maximal number of iterations is reached, and the solution is wrong as the following plot shows:

sol = solve(prob_dde_rober1, MethodOfSteps(Tsit5()); initial_order = 1)

using Plots
plotly()
plot(sol)

Using Rosenbrock23 a solution is computed on the whole integration interval:

sol = solve(prob_dde_rober1, MethodOfSteps(Rosenbrock23()); initial_order = 1)

plot(sol)

This solution shows the expected dynamics; moreover, it indicates that the stiff solvers work in principle and are quite performant since according to Guglielmi "RADAR5 is the only code able to correctly integrate the above very stiff problem" out of the solvers he considered.

Interestingly, both in Guglielmi and Hairer's "Implementing Radau IIA methods for stiff delay differential equations" and as example problem in the RADAR5 package a different Robertson problem is mentioned (here again on a reduced integration interval):

function f_dde_rober2(du, u, h, p, t)
    # compute repeating terms
    x = 4e-2 * u[1]
    y = 1e4 * h(p, t - 1e-2; idxs = 2) * u[3]
    z = 3e7 * u[2] * u[2]

    # compute derivatives
    du[1] = y - x
    du[2] = x - y - z
    du[3] = z

    nothing
end

prob_dde_rober2 = DDEProblem(f_dde_rober2, [1.0, 0.0, 0.0], h_dde_rober,
                             (0.0, 1e5);
                             constant_lags = [1e-2])

For this problem, however, Rosenbrock23 computes a solution only on [0.0, 6.14] since the time steps become too small:

sol = solve(prob_dde_rober2, MethodOfSteps(Rosenbrock23()); initial_order = 1)

plot(sol)

Similar dynamics can be observed with Rodas5, although in this case integration stops because of instability (i.e. NaN values):

sol = solve(prob_dde_rober2, MethodOfSteps(Rodas5()); initial_order = 1)

plot(sol)

Using absolute and relative tolerances that are mentioned in Guglielmi and Hairer's publication I can provoke the same explosions as before at a slightly later time point:

sol = solve(prob_dde_rober2, MethodOfSteps(Rodas5()); initial_order = 1,
                                           reltol = 1e-6, abstol = 1e-10)

plot(sol)

According to Guglielmi and Hairer, with a delay of 3e-2 the second component of this problem oscillates and then explodes at around t = 16.8.

I managed to compile and run RADAR5 and the contained Robertson example on my computer; RADAR5 successfully computed a solution on the interval [0.0, 1e10]. Even though I have no experience with Fortran I studied the implementation of the problem to check whether the implementation corresponds to the stated DDE; hereby I noticed that Guglielmi and Hairer apparently used slightly different default parameters in their implementation (tau = 1e-3, reltol = 1e-9, abstol = 1e-14, and an integration interval of [0.0, 1e11]) but I could still compute a correct solution after changing these values to the ones stated above.

I do not know why DelayDiffEq does not compute a correct solution for the second problem. I'm also surprised that Rodas5 with default settings returns NaN values; I assumed this could (and should) not happen since then the time step should be decreased... I also played around with the number of fixed point iterations; setting force_dtmin = true in the last example resulted in NaN values as well. Are there any settings I could tune? Do you have any ideas why RADAR5 can solve this problem whereas I could not compute a solution with DelayDiffEq?

MethodError at early times

I receive a MethodError when trying to solve the following DDE problem:
DDEProblem((t,u,h,du) -> du[1] = -h(t/2)[1], t -> [1.0], [1.0], (0.0, 10.0), [])

The error only occurs if tspan starts sufficiently early.
e.g. It works when tspan = (0.1,10.0), but not when tspan = (0.001,10.0).

Interpolation after an event is not correct in the proceeding interval

using DelayDiffEq
# Gut dosing model
function f(t,u,h,du)
 Depot,Central = u
 du[1] = -1.5*Depot
 du[2] =  1.5*Depot - (1/30)*Central - 0.1h(t-1,Val{0},2)
end

h(t,idxs) = 0.0
prob = DDEProblem(f,h,zeros(2),(0.0,11.0),[1])

θ = [
     1.5,  #Ka
     1.0,  #CL
     30.0 #V
     ]

condition(t,u,integrator) = t == 10
affect!(integrator) = integrator.u[1] += 100
cb = DiscreteCallback(condition,affect!)
sol = solve(prob,MethodOfSteps(Tsit5()),callback=cb,tstops=[10],dt=6)

sol.t

Float64[12]
0.00
1.00
2.00
3.00
4.00
5.00
6.00
10.0
10.0
10.6…
10.9…
11.0

sol(10.6)

Float64[2]
-59.3…
58.7…

sol(10.7)

Float64[2]
35.0…
64.1…

It might not be re-computing the derivative at the timepoint of discontinuity? That means it's not handling the FSAL reset too.

Units tests failure

PainterQubits/Unitful.jl#107

PainterQubits/Unitful.jl#108

Example on repo frontpage/documentation fails

After copy-and-pasting from the repo frontpage, upon trying to solve, I get the following error with a lot of very gnarly type signatures. This also happens if I do using DifferentialEquations and I ran into it on other toy models.

Main> sol = solve(prob,alg)
ERROR: TypeError: non-boolean (DataType) used in boolean context
Stacktrace:
 [1] ode_addsteps! at /home/curry/.julia/v0.6/OrdinaryDiffEq/src/dense/low_order_rk_addsteps.jl:540 [inlined]
 [2] advance_ode_integrator!(::DelayDiffEq.DDEIntegrator{OrdinaryDiffEq.Tsit5,Array{Float64,1},Float64,Void,Float64,Float64,Float64,Array{Float64,1},Float64,Float64,Array{Array{Float64,1},1},DiffEqBase.ODESolution{Float64,2,Array{Array{Float64,1},1},Void,Void,Array{Float64,1},Array{Array{Array{Float64,1},1},1},DiffEqBase.DDEProblem{Array{Float64,1},Float64,Array{Int64,1},Array{Any,1},true,Void,Void,#bc_model,#h,Void,UniformScaling{Int64}},OrdinaryDiffEq.Tsit5,OrdinaryDiffEq.InterpolationData{DelayDiffEq.##35#41{DiffEqBase.DDEProblem{Array{Float64,1},Float64,Array{Int64,1},Array{Any,1},true,Void,Void,#bc_model,#h,Void,UniformScaling{Int64}}},Array{Array{Float64,1},1},Array{Float64,1},Array{Array{Array{Float64,1},1},1},OrdinaryDiffEq.Tsit5Cache{Array{Float64,1},Array{Float64,1},Array{Float64,1},Array{Float64,1},OrdinaryDiffEq.Tsit5ConstantCache{Float64,Float64}}}},DelayDiffEq.##37#43{DiffEqBase.DDEProblem{Array{Float64,1},Float64,Array{Int64,1},Array{Any,1},true,Void,Void,#bc_model,#h,Void,UniformScaling{Int64}}},Void,OrdinaryDiffEq.Tsit5Cache{Array{Float64,1},Array{Float64,1},Array{Float64,1},Array{Float64,1},OrdinaryDiffEq.Tsit5ConstantCache{Float64,Float64}},OrdinaryDiffEq.ODEIntegrator{OrdinaryDiffEq.Tsit5,Array{Float64,1},Float64,Void,Float64,Float64,Float64,Array{Array{Float64,1},1},DiffEqBase.ODESolution{Float64,2,Array{Array{Float64,1},1},Void,Void,Array{Float64,1},Array{Array{Array{Float64,1},1},1},DiffEqBase.ODEProblem{Array{Float64,1},Float64,true,Void,#bc_model,Void,Void,UniformScaling{Int64},DiffEqBase.StandardODEProblem},OrdinaryDiffEq.Tsit5,OrdinaryDiffEq.InterpolationData{#bc_model,Array{Array{Float64,1},1},Array{Float64,1},Array{Array{Array{Float64,1},1},1},OrdinaryDiffEq.Tsit5Cache{Array{Float64,1},Array{Float64,1},Array{Float64,1},Array{Float64,1},OrdinaryDiffEq.Tsit5ConstantCache{Float64,Float64}}}},#bc_model,Void,OrdinaryDiffEq.Tsit5Cache{Array{Float64,1},Array{Float64,1},Array{Float64,1},Array{Float64,1},OrdinaryDiffEq.Tsit5ConstantCache{Float64,Float64}},OrdinaryDiffEq.DEOptions{Float64,Float64,Float64,Float64,DiffEqBase.#ODE_DEFAULT_NORM,DiffEqBase.CallbackSet{Tuple{},Tuple{}},DiffEqBase.#ODE_DEFAULT_ISOUTOFDOMAIN,DiffEqBase.#ODE_DEFAULT_PROG_MESSAGE,DiffEqBase.#ODE_DEFAULT_UNSTABLE_CHECK,DataStructures.BinaryHeap{Float64,DataStructures.LessThan},DataStructures.BinaryHeap{Float64,DataStructures.LessThan},Void,Void,Int64,Array{Float64,1},Array{Float64,1},Array{Float64,1}},Array{Float64,1}},DiffEqBase.#ODE_DEFAULT_NORM,OrdinaryDiffEq.DEOptions{Float64,Float64,Float64,Float64,DiffEqBase.#ODE_DEFAULT_NORM,DiffEqBase.CallbackSet{Tuple{},Tuple{}},DiffEqBase.#ODE_DEFAULT_ISOUTOFDOMAIN,DiffEqBase.#ODE_DEFAULT_PROG_MESSAGE,DiffEqBase.#ODE_DEFAULT_UNSTABLE_CHECK,DataStructures.BinaryHeap{Float64,DataStructures.LessThan},DataStructures.BinaryHeap{DelayDiffEq.Discontinuity{Float64},DataStructures.LessThan},Void,Void,Int64,Array{Float64,1},Array{Float64,1},Array{DelayDiffEq.Discontinuity{Float64},1}},Void,Array{Float64,1}}) at /home/curry/.julia/v0.6/DelayDiffEq/src/integrator_utils.jl:254
 [3] perform_step!(::DelayDiffEq.DDEIntegrator{OrdinaryDiffEq.Tsit5,Array{Float64,1},Float64,Void,Float64,Float64,Float64,Array{Float64,1},Float64,Float64,Array{Array{Float64,1},1},DiffEqBase.ODESolution{Float64,2,Array{Array{Float64,1},1},Void,Void,Array{Float64,1},Array{Array{Array{Float64,1},1},1},DiffEqBase.DDEProblem{Array{Float64,1},Float64,Array{Int64,1},Array{Any,1},true,Void,Void,#bc_model,#h,Void,UniformScaling{Int64}},OrdinaryDiffEq.Tsit5,OrdinaryDiffEq.InterpolationData{DelayDiffEq.##35#41{DiffEqBase.DDEProblem{Array{Float64,1},Float64,Array{Int64,1},Array{Any,1},true,Void,Void,#bc_model,#h,Void,UniformScaling{Int64}}},Array{Array{Float64,1},1},Array{Float64,1},Array{Array{Array{Float64,1},1},1},OrdinaryDiffEq.Tsit5Cache{Array{Float64,1},Array{Float64,1},Array{Float64,1},Array{Float64,1},OrdinaryDiffEq.Tsit5ConstantCache{Float64,Float64}}}},DelayDiffEq.##37#43{DiffEqBase.DDEProblem{Array{Float64,1},Float64,Array{Int64,1},Array{Any,1},true,Void,Void,#bc_model,#h,Void,UniformScaling{Int64}}},Void,OrdinaryDiffEq.Tsit5Cache{Array{Float64,1},Array{Float64,1},Array{Float64,1},Array{Float64,1},OrdinaryDiffEq.Tsit5ConstantCache{Float64,Float64}},OrdinaryDiffEq.ODEIntegrator{OrdinaryDiffEq.Tsit5,Array{Float64,1},Float64,Void,Float64,Float64,Float64,Array{Array{Float64,1},1},DiffEqBase.ODESolution{Float64,2,Array{Array{Float64,1},1},Void,Void,Array{Float64,1},Array{Array{Array{Float64,1},1},1},DiffEqBase.ODEProblem{Array{Float64,1},Float64,true,Void,#bc_model,Void,Void,UniformScaling{Int64},DiffEqBase.StandardODEProblem},OrdinaryDiffEq.Tsit5,OrdinaryDiffEq.InterpolationData{#bc_model,Array{Array{Float64,1},1},Array{Float64,1},Array{Array{Array{Float64,1},1},1},OrdinaryDiffEq.Tsit5Cache{Array{Float64,1},Array{Float64,1},Array{Float64,1},Array{Float64,1},OrdinaryDiffEq.Tsit5ConstantCache{Float64,Float64}}}},#bc_model,Void,OrdinaryDiffEq.Tsit5Cache{Array{Float64,1},Array{Float64,1},Array{Float64,1},Array{Float64,1},OrdinaryDiffEq.Tsit5ConstantCache{Float64,Float64}},OrdinaryDiffEq.DEOptions{Float64,Float64,Float64,Float64,DiffEqBase.#ODE_DEFAULT_NORM,DiffEqBase.CallbackSet{Tuple{},Tuple{}},DiffEqBase.#ODE_DEFAULT_ISOUTOFDOMAIN,DiffEqBase.#ODE_DEFAULT_PROG_MESSAGE,DiffEqBase.#ODE_DEFAULT_UNSTABLE_CHECK,DataStructures.BinaryHeap{Float64,DataStructures.LessThan},DataStructures.BinaryHeap{Float64,DataStructures.LessThan},Void,Void,Int64,Array{Float64,1},Array{Float64,1},Array{Float64,1}},Array{Float64,1}},DiffEqBase.#ODE_DEFAULT_NORM,OrdinaryDiffEq.DEOptions{Float64,Float64,Float64,Float64,DiffEqBase.#ODE_DEFAULT_NORM,DiffEqBase.CallbackSet{Tuple{},Tuple{}},DiffEqBase.#ODE_DEFAULT_ISOUTOFDOMAIN,DiffEqBase.#ODE_DEFAULT_PROG_MESSAGE,DiffEqBase.#ODE_DEFAULT_UNSTABLE_CHECK,DataStructures.BinaryHeap{Float64,DataStructures.LessThan},DataStructures.BinaryHeap{DelayDiffEq.Discontinuity{Float64},DataStructures.LessThan},Void,Void,Int64,Array{Float64,1},Array{Float64,1},Array{DelayDiffEq.Discontinuity{Float64},1}},Void,Array{Float64,1}}) at /home/curry/.julia/v0.6/DelayDiffEq/src/integrator_interface.jl:184
 [4] solve!(::DelayDiffEq.DDEIntegrator{OrdinaryDiffEq.Tsit5,Array{Float64,1},Float64,Void,Float64,Float64,Float64,Array{Float64,1},Float64,Float64,Array{Array{Float64,1},1},DiffEqBase.ODESolution{Float64,2,Array{Array{Float64,1},1},Void,Void,Array{Float64,1},Array{Array{Array{Float64,1},1},1},DiffEqBase.DDEProblem{Array{Float64,1},Float64,Array{Int64,1},Array{Any,1},true,Void,Void,#bc_model,#h,Void,UniformScaling{Int64}},OrdinaryDiffEq.Tsit5,OrdinaryDiffEq.InterpolationData{DelayDiffEq.##35#41{DiffEqBase.DDEProblem{Array{Float64,1},Float64,Array{Int64,1},Array{Any,1},true,Void,Void,#bc_model,#h,Void,UniformScaling{Int64}}},Array{Array{Float64,1},1},Array{Float64,1},Array{Array{Array{Float64,1},1},1},OrdinaryDiffEq.Tsit5Cache{Array{Float64,1},Array{Float64,1},Array{Float64,1},Array{Float64,1},OrdinaryDiffEq.Tsit5ConstantCache{Float64,Float64}}}},DelayDiffEq.##37#43{DiffEqBase.DDEProblem{Array{Float64,1},Float64,Array{Int64,1},Array{Any,1},true,Void,Void,#bc_model,#h,Void,UniformScaling{Int64}}},Void,OrdinaryDiffEq.Tsit5Cache{Array{Float64,1},Array{Float64,1},Array{Float64,1},Array{Float64,1},OrdinaryDiffEq.Tsit5ConstantCache{Float64,Float64}},OrdinaryDiffEq.ODEIntegrator{OrdinaryDiffEq.Tsit5,Array{Float64,1},Float64,Void,Float64,Float64,Float64,Array{Array{Float64,1},1},DiffEqBase.ODESolution{Float64,2,Array{Array{Float64,1},1},Void,Void,Array{Float64,1},Array{Array{Array{Float64,1},1},1},DiffEqBase.ODEProblem{Array{Float64,1},Float64,true,Void,#bc_model,Void,Void,UniformScaling{Int64},DiffEqBase.StandardODEProblem},OrdinaryDiffEq.Tsit5,OrdinaryDiffEq.InterpolationData{#bc_model,Array{Array{Float64,1},1},Array{Float64,1},Array{Array{Array{Float64,1},1},1},OrdinaryDiffEq.Tsit5Cache{Array{Float64,1},Array{Float64,1},Array{Float64,1},Array{Float64,1},OrdinaryDiffEq.Tsit5ConstantCache{Float64,Float64}}}},#bc_model,Void,OrdinaryDiffEq.Tsit5Cache{Array{Float64,1},Array{Float64,1},Array{Float64,1},Array{Float64,1},OrdinaryDiffEq.Tsit5ConstantCache{Float64,Float64}},OrdinaryDiffEq.DEOptions{Float64,Float64,Float64,Float64,DiffEqBase.#ODE_DEFAULT_NORM,DiffEqBase.CallbackSet{Tuple{},Tuple{}},DiffEqBase.#ODE_DEFAULT_ISOUTOFDOMAIN,DiffEqBase.#ODE_DEFAULT_PROG_MESSAGE,DiffEqBase.#ODE_DEFAULT_UNSTABLE_CHECK,DataStructures.BinaryHeap{Float64,DataStructures.LessThan},DataStructures.BinaryHeap{Float64,DataStructures.LessThan},Void,Void,Int64,Array{Float64,1},Array{Float64,1},Array{Float64,1}},Array{Float64,1}},DiffEqBase.#ODE_DEFAULT_NORM,OrdinaryDiffEq.DEOptions{Float64,Float64,Float64,Float64,DiffEqBase.#ODE_DEFAULT_NORM,DiffEqBase.CallbackSet{Tuple{},Tuple{}},DiffEqBase.#ODE_DEFAULT_ISOUTOFDOMAIN,DiffEqBase.#ODE_DEFAULT_PROG_MESSAGE,DiffEqBase.#ODE_DEFAULT_UNSTABLE_CHECK,DataStructures.BinaryHeap{Float64,DataStructures.LessThan},DataStructures.BinaryHeap{DelayDiffEq.Discontinuity{Float64},DataStructures.LessThan},Void,Void,Int64,Array{Float64,1},Array{Float64,1},Array{DelayDiffEq.Discontinuity{Float64},1}},Void,Array{Float64,1}}) at /home/curry/.julia/v0.6/DelayDiffEq/src/solve.jl:280
 [5] #solve#51(::Array{Any,1}, ::Function, ::DiffEqBase.DDEProblem{Array{Float64,1},Float64,Array{Int64,1},Array{Any,1},true,Void,Void,#bc_model,#h,Void,UniformScaling{Int64}}, ::DelayDiffEq.MethodOfSteps{OrdinaryDiffEq.Tsit5,Void,Void,Void,false}, ::Array{Array{Float64,1},1}, ::Array{Float64,1}, ::Array{Any,1}) at /home/curry/.julia/v0.6/DelayDiffEq/src/solve.jl:362
 [6] solve(::DiffEqBase.DDEProblem{Array{Float64,1},Float64,Array{Int64,1},Array{Any,1},true,Void,Void,#bc_model,#h,Void,UniformScaling{Int64}}, ::DelayDiffEq.MethodOfSteps{OrdinaryDiffEq.Tsit5,Void,Void,Void,false}) at /home/curry/.julia/v0.6/DelayDiffEq/src/solve.jl:361
 [7] eval(::Module, ::Any) at ./boot.jl:235

Stochastic Delay Differential Equations

StochasticDiffEq.jl has a very similar structure to OrdinaryDiffEq.jl. With some tweaks it should be possible to make DelayDiffEq.jl handle both of them, in which case we'll have the first stochastic delay differential equation setup which will be really really cool!

Methods with lazy interpolants will not work with MethodOfSteps

This means the Verner methods are excluded. It likely has to do something with the adding of the extra steps not happening correctly. It probably needs #15 (comment) .

State-dependent delay methods