juliasmoothoptimizers / solvertools.jl Goto Github PK

Create A. L. using TRON

The amount of required packages is too large.

Follow IPOPT example

The Windows tests keep failing because of AmplNLReader. We should eliminate that package from the tests and e.g., hard-code one problem and its derivatives instead.

If more solvers are expected to be implemented in SolverTools.jl, standard tests (allocations, multiple precision,...) should be implemented and run over sub-solvers of the same kind (TR and LS so far) as in JSOSolvers.jl.

Checklist for stable release

• Stable releases of
  • NLPModels (JuliaSmoothOptimizers/NLPModels.jl#37)
  • AmplNLReader (JuliaSmoothOptimizers/AmplNLReader.jl#19)
  • CUTEst (JuliaSmoothOptimizers/CUTEst.jl#43)
  • Krylov
• [ ] Good solvers (how good? how do we measure?)
  • [x] Unconstrained minimization
  • [x] Bound-contrained minimization (TRON)
  • [ ] Equality-constrained minimization (Augmented Lagrangian?)
  • [ ] Inequality-constrained minimization (Use SlackModel?)
  • [ ] Equality-constrained minimization with bounds (TRON + AugLagr?)
  • [ ] General constraints and bounds (SlackModel + TRON + AugLagr?)
• [ ] Increase coverage

cgtol should be proportional to the square root of the gradient norm and used as relative tolerance. Benchmark.

After #74, we can have try creating a way to get the information and print it correctly depending on the type. The type conversion will already be done.
This can be used to create some kind of Logger to be used inside the solvers or by solve_problems - notice that we can't use DataFrames this way because the printing is on real-time.

Concept:

function trunk(...; logger::SolverLogger)
  ...
  prepare!(logger, [Int, Float64, Float64, Float64], [:iter, :fx, :ngx, :time])
  ...
  k += 1
  fx = f(x)
  normgx = norm(gx)
  elapsed_time = time() - start_time
  with_logger(logger) do
    @info(iter=k, fx=fx, ngx=normgx, time=elapsed_time)
end

Example:

julia> model = CUTEstModel("BENNETT5LS")
Minimization problem BENNETT5LS
nvar = 3, ncon = 0 (0 linear)

julia> x = model.meta.x0
3-element Array{Float64,1}:
 -2000.0
    50.0
     0.8

julia> g = grad(model, x)
3-element Array{Float64,1}:
      37.195399822788076
    1518.022256935859
 -478331.78119128384

julia> x1 = x - g
3-element Array{Float64,1}:
  -2037.195399822788
  -1468.022256935859
 478332.5811912838

julia> obj(model, x1)
NaN

Solvers should be able to trap NaNs and return an appropriate status.

Instead of redefining a new "dummy solver" here https://github.com/JuliaSmoothOptimizers/SolverTools.jl/blob/main/test/dummy_solver.jl

This is, in particular, used in TRON

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Keyword argument τ1 of armijo-wolfe default value (0.9999) is rounded up to 1 with Float16. I believe this breaks theoretical convergence of the line-search algorithm.

Use logger as arguments.

Add stats and the information gathered in DataFrames

https://github.com/JuliaSmoothOptimizers/JSOSolvers.jl/blob/5c5294cc90036d606d6aa204d43f03654e61f3e5/src/tronls.jl#L8

This is the same as

With the creation of SolverBenchmark.jl, we can remove plot-related things from this package completely and only use SolverBenchmark.jl, remove the solvers and create specific packages for them, and have only:

• Tools: (folder auxiliary, linesearch, and trust-region);
• Structure: (folder stats, essentially ExecutionStats);
• Bmark: (folder bmark).

This should stay small. Then, we could change the name to something more descriptive (SolverTools.jl?).

So, for developing a new solver and creating dataframes with comparison: using SolverTools. For plotting or publishing these results: using SolverBenchmark.

What about the solvers? Should they have individual packages? Should they be bundled according to type. Should all "approved" solvers be in a special package?
We could have

• JSOSolvers.jl: Competitive solvers, with benchmarks against other solvers (IPOPT, Optim), with PkgBenchmark checking regressions;
• JSOTextbookSolvers.jl: Simple solvers, decent implementation.

Something like

Execution stats:
 - Status: ...
 - Dual measure: ...
 - Primal measure: ...

cf @joaoluisro

Homebrew.jl has become unreliable with Julia 0.5.

Perhaps trunk could also use TRON's slightly more elaborate trust-region.

I think it could be useful to have a line-search solver for Goldstein-Armijo conditions, which, contrary to the Wolfe condition, does not require to compute the gradient along the search direction.

Does anyone know an efficient algorithm for the Goldstein conditions?
I did a bit of research and found: https://optimization-online.org/wp-content/uploads/2022/11/OptLett_Nov_2023_final.pdf,
which looks to performed better than the standard Goldstein line-search algorithm.

Relevant discussion: https://discourse.julialang.org/t/trust-region-constrained-optimization/93077/6

From #43

eliminate BenchmarkProfiles.jl from the requirements and only export the benchmark functions if it's installed

Using requires
export profile_solvers

profile_solvers(stats :: Dict{Symbol, Array{Int,2}}; kwargs...) = error("BenchmarkProfiles is not installed")

@require BenchmarkProfiles begin
  function profile_solvers(stats :: Dict{Symbol, Array{Int,2}}; kwargs...)
    # our current function
  end
end

Even if I added finalize(problem) in solve_problems, CUTE complains:
ERROR: LoadError: CUTEst: call cutest_finalize on current model first

Short draft on the design of merit functions

Examples of merit functions for equality-constrained problems:

• l1 and l2: phi(x, eta) = f(x) + eta * |c(x)|_p, where p = 1 or p = 2.
• l2 squared (what is the name of this one again?): phi(x, eta) = f(x) + eta * |c(x)|_2^2 / 2.
• Fletcher: phi(x, eta) = f(x) - y(x)' * c(x) + eta * |c(x)|^2 / 2, where y(x) = argmin |g(x) + A(x)' * y|^2
• aug. Lagrangian: phi(x, y, eta) = f(x) - y' * c(x) + eta * |c(x)|^2 / 2.

Important features:

• value at given points
• directional derivative
• Memory efficient
• Save evaluations
• Option to update internal values or not (defaults to yes)

Suggested implementation:

• AbstractMerit
• L1Merit <: AbstractMerit, AugLagMerit <: AbstractMerit
• Assume fx, cx, gx and Ad are stored internally, where fx is the objective, cx is the constraints, and gx is the gradient at x and Ad is the Jacobian times the direction d.
• The user may decide to keep pointers or new copies. Test will be made to determine that's possible.
• obj(::AbstractMerit, x::Vector; update=true)
• directional(::AbstractMerit, x::Vector, d::Vector; update=true)

Discussion:

• Line search expects a LineModel, which has a lot more liberty and deals with an nlp directly.
  • For instance, a LineModel can compute grad! on an nlp.
• Trust region expects an nlp an also computes `grad!.
• Could we unify these?
• Should AbstractMerit be an NLPModel? (I think no)
• Should we disallow grad! on nlp from line search and trust region?

I'm doing this now.

instead of redefining a new nlpmodel in tests https://github.com/JuliaSmoothOptimizers/SolverTools.jl/blob/main/test/simple_model.jl

It would be great to maintain benchmarks to compare pull requests to develop, and to measure the improvements brought by a new revision.

BenchmarkTools.jl seems useful to benchmark solver building blocks.

I think we need a different tool to compare the performance of solvers across versions and across branches, e.g., using aggregate measures, performance profiles, etc.

@abelsiqueira

What is needed for a 1.0 release?

A segmentation error occurs when you run solve_problems(ipopt, problem). I'm using packages:

Status `~/.julia/environments/v1.1/Project.toml`
[9e28174c] BinDeps v0.8.10+ [`~/.julia/dev/BinDeps`]
[1b53aba6] CUTEst v0.6.0
[944b1d66] CodecZlib v0.5.2
[9aa1b823] FastClosures v0.2.1
[f6369f11] ForwardDiff v0.10.3
[7073ff75] IJulia v1.18.0
[b6b21f68] Ipopt v0.5.4
[7782544b] JSOSolvers v0.0.0 [`~/.julia/dev/JSOSolvers`]
[4076af6c] JuMP v0.19.0
[ba0b0d4f] Krylov v0.3.0
[5c8ed15e] LinearOperators v0.5.3
[a4795742] NLPModels v0.8.0+ [`~/.julia/dev/NLPModels`]
[bd118512] NLPModelsIpopt v0.1.0 [`~/.julia/dev/NLPModelsIpopt`]
[581a75fa] SolverBenchmark v0.1.0
[a513cd03] SolverTools v0.0.0 [`~/.julia/dev/SolverTools`]

Connected to JuliaSmoothOptimizers/JSOSolvers.jl#94 we could use the logger for armijo_wolfe.
This will create a new dependency to SolverCore.jl though.

Is it possible to insert empty values in log_row ?

The aim is that the empty entry would still be aligned with the other columns.
This is useful, for instance, when printing logs for an algorithm that computes two different steps.

Distinguish failures from

• unsuitable models (for instance with constraints for an unconstrained solver)
• erroneous models (for instance, f(x_0) fails)
• etc.

More generally, devise a sound output format including the solution, statistics of computation, status (property of the returned solution).

When we drop 0.6, we should drop MiniLogging and use the native logging system. That will remove one dependency (though I like MiniLogging!).

Hello, is it possible to consider the counter field as a NLPModels.Counters instead of a NLSModels.Counters ?