argmin-rs / argmin Goto Github PK
View Code? Open in Web Editor NEWNumerical optimization in pure Rust
Home Page: http://argmin-rs.org
License: Apache License 2.0
Numerical optimization in pure Rust
Home Page: http://argmin-rs.org
License: Apache License 2.0
Hello,
Unless I am mistaken, Brent method is the name for two algorithms: one for root-finding and one for minimization. They are respectively described under the names zero
and localmin
in chapters 4 and 5 of the book Algorithms for Minimization without Derivatives by Brent (1973). Although they share the same ideas, the first is not directly applicable to the second problem unless we know the derivative of the function to optimize.
It seems that the function implemented in argmin
is Brent's root-finding method (currently the only one described on Wikipedia). Hence I think that we are still missing Brent's minimization method. I would like to implement the minimization method, but my question is what to do of the root-finding method already present? Is root-finding one of the purposes of argmin
? In which case, maybe would it be appropriate to separate root-finding and optimization algorithms, since those are two different kinds of problems?
cc @xemwebe
Currently, running an Executor
returns an ArgminResult
. While this holds the result of the optimization, it may give the impression of being related to error handling.
In addition, it is unnecessarily difficult to extract the actual result of the optimization as one needs to access the state
field of type IterState
. None of this is properly documented.
ArgminResult
seems to be unnecessary, as one can just return IterState
instead. But maybe there is an even better option.
argmin/src/solver/particleswarm/mod.rs
Lines 46 to 51 in 1cf4523
If possible, set the required dependencies and features on a "per example basis" in Cargo.toml to better indicate to the user what is necessary in order to run the example. This is based on the suggestion here.
I think it would be good to remove the WIP tag from the project title.
I understand that things can (always) be improved, and some API refactoring can still be planned but I think that a 2 yo project with some amount of users is no longer "Work in progress". Just because it implies that this is somehow less stable than any other actively developed project.
Hello,
CMA-ES is a powerful evolutionary algorithm for black-box optimization problems. If there is interest in adding it to this optimization suite, I would be happy to give it a try. My plan is to start by following the purecma.py
implementation, a public domain pedagogical implementation by the author of the algorithm, and add more advanced features if necessary.
Currently the stopping criterion tolerance is often hard coded. For instance in LBFGS,
argmin/src/solver/quasinewton/lbfgs.rs
Lines 166 to 172 in 46f43cd
the stopping criterion for the gradient is std::f64::EPSILON.sqrt() ~ 1e-8
while in practical machine learning application 1e-4 could be sufficient, as performance matter more than perfect convergence .
So the question is what API would allow for these stopping conditions to be set by the user. For instance, it could be some combination of,
a) add tol
to the struct initialization in new
. That's probably not ideal, as for instance LBFGS would require 2 tolerances, and always providing them manually is not great.
b) add tol
to the struct but not as an argument in new
method
c) implement a Default
trait with current values.
d) some form of builder pattern or a method e.g. with_tol
.
For now I would likely tend to propose b) with d) using a with_tol
or just tol
method (which also would be backward compatible), and I'm looking for some feedback. I'm not sure what would best generalize to different solvers. Also ideally the solution should also be consistent with Executor.max_iters approach.
When compiling argmin with the nalgebral
feature, one currently gets many warnings of the kind:
warning: use of deprecated type alias `nalgebra::MatrixMN`: use SMatrix for a statically-sized matrix using integer dimensions, or OMatrix for an owned matrix using types as dimensions.
--> src/core/math/zero_nalgebra.rs:17:34
|
17 | impl<N, R, C> ArgminZeroLike for MatrixMN<N, R, C>
| ^^^^^^^^^^^^^^^^^
As someone who hasn't used nalgebra yet, I'm unsure which way to go. Is SMatrix
or OMatrix
a more appropriate replacement for MatrixMN
? Or should the math traits be implemented for both? Any help by nalgebra users is highly appreciated :)
Now that we have nalgebra support thanks to @Maher4Ever (#68), I think it is time to release a new version.
When I started this project, I wanted it to be as modular as possible, which is why I've decided to split the project into a "core" part which holds all the "infrastructure" and a part where all the solvers are implemented. The idea was that others, who wanted to implement solvers based on the principles introduced here, would not need to import the entire project. However, this has lead to an awkward development-experience: In order to develop locally, all repositories have to be pulled and argmin
's Cargo.toml
needs to be adapted to point to the local crates.
Does anyone have a suggestion how this could be solved better? Workspaces may be an option but I'm not sure if they sufficiently improve the situation. I'm tending towards just moving the core part into argmin
...
I would still keep testfunctions
, finitediff
, modcholesky
and pyargmin
separated, as they seem sufficiently self-contained.
There seems to be a bug in HagerZhangLineSearch (for instance, running the steepestdescent or nonlinear_cg examples with HagerZhangLineSearch will not converge towards the solution).
Thanks for this package!
It looks like currently Travis CI only build this crate without running tests. It would be nice if tests were also run.
Hello,
Coming from Python (scipy.optimize.minimize
) and MATLAB (fmincon
), I can't find a way to specify bounds (and other constraints), which to me are very essential parts of an optimization problem.
Python / Scipy:
res = minimize(opti_fun, x0, bounds=mybounds, constraints=myconstraints)
Matlab:
res = fmincon(opti_fun, x0, A, b, Aeq, beq, lowerbound, upperbound)
Looking through the documentation, I could not find that this is implemented. (With the exception of some manual bounds in the simulatedannealing.rs example).
Is this something that is being worked on? Or is it planned? Or is it too complicated / time consuming at the moment? Or did I just miss something here?
I would love to have constrained optimization (at least with bounds) in Rust!
Currently checkpoints can only be written to disk. It would be better if users could write their own checkpointing mechanisms. In order to do so, a checkpointing trait should be defined. This could be implemented similar to observers.
rand_xorshift::XorShiftRng
uses Xorshift128 (also called xor128
) which is from Marsaglia's original paper. It has significant weaknesses ([0][1]). It was replaced as rand
's default noncryptographic PRNG in 2018.
Stochastic optimization requires high quality random numbers and Xorshift128 does not provide that.
argmin could use a sensible default like rand
's current xoshiro256++
or even allow users to select one via traits (I'm willing to write a PR for that).
Hi! I am very excited to try out the ParticleSwarm optimizer. It seems however that it only provides 2 values to the objective function (presumably x/y coordinates). Would it be possible to use it for higher dimensional optimization?
Thanks for this very nice package!
On the general topic of benchmarks, I would be quite interested to know how the L-BFGS implementation in argmin would compare to,
There are Python wrappers for these, in PyLBFGS and scipy.optimize. So using pyargmin it should be relatively easy to compare them (albeit with the Python overhead).
L-BGFS is very commonly used in machine learning (also see the "Rust needs BFGS" blog by the author of the bfgs crate). I'm mostly interested in seeing of it's possible to get a comparable or faster implementation in Rust than the one in scipy.optimize, that would be more readable (and support both f64
and f32
).
Opening this issue mostly to report progress on it. If you have any suggestions (e.g. on the benchmark cases to use) I would be interested as well.
Hi,
I am a beginner to use this project, I have a rather trivial question, why fn gradient(&self, p: &Self::Param) -> Result<Self::Param, Error>
this function returns the type Param
.
Although in the Rosenbrock example, Param
is a vector type, so it doesn't matter. But for example, I have a scenario I know very well how many variables and how many parameters, so I use array instead of vector. For example I I use a one f64 number array [f64;1]
as parameter, and the function is x_1+x_2+p[0]
, (this is a stupid function, but only for example), so I know that the gradient should be [f64;2]
, so, this actually won't work.
in search of a solver in rust i stumbled over this very promising library. However, I missed implementations of popular solver I used quite a lot, e.g. Brent's method (for one dimensional problems) and the methods provided by the ancient minpack package (Levenberg-Marquardt).
I was wondering what's the reason for that. Is it just a matter of priorities or are the any particular reasons to exclude these algoritms?
Hey, thanks for creating and maintaining this crate :) I only glanced over the examples, but wondered why you don't use the builder pattern. It could simplify setting up a new solver a bit, otherwise the crate is looking really useful. Greetings from Aachen.
argmin should be tested in a WASM environment.
Current behavior of ArgminSlogLogger::file
is to truncate the log file on each run.
Sometimes I want to see the logs of a previous run, particularly if using checkpoints,
The ndarray < 0.15
requirement currently blocks rust-ml/linfa#124. This is currently blocked on https://github.com/rust-ndarray/ndarray-linalg which still has to be published their latest version to crates.io
Currently Serialization and Deserialization is implemented for all types which get serialized when creating a checkpoint. It would be convenient if this was optional by hiding it behind a feature gate.
I'm trying to build the master version of this crate and run test and am seeing some failures in examples so far,
$ cargo --version
cargo 1.35.0 (6f3e9c367 2019-04-04)
$ cargo test
cargo test [tags/v0.2.4]
Compiling argmin v0.2.4 (/home/rth/src/argmin)
error[E0277]: the trait bound `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>: argmin::prelude::ArgminMul<f64, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>` is not satisfied
--> examples/trustregion_nd.rs:66:15
|
66 | let res = Executor::new(cost, solver, init_param)
| ^^^^^^^^^^^^^ the trait `argmin::prelude::ArgminMul<f64, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>` is not implemented for `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>`
|
= note: required because of the requirements on the impl of `argmin::prelude::Solver<Rosenbrock>` for `argmin::solver::trustregion::TrustRegion<argmin::solver::trustregion::Dogleg>`
= note: required by `argmin::prelude::Executor::<O, S>::new`
[...]
error: aborting due to 21 previous errors
Some errors occurred: E0277, E0599.
For more information about an error, try `rustc --explain E0277`.
error: aborting due to 9 previous errors
Some errors occurred: E0277, E0599.
For more information about an error, try `rustc --explain E0277`.
error: Could not compile `argmin`.
warning: build failed, waiting for other jobs to finish...
error: Could not compile `argmin`.
warning: build failed, waiting for other jobs to finish...
error: build failed
see full output below,
cargo test [tags/v0.2.4]
Compiling argmin v0.2.4 (/home/rth/src/argmin)
error[E0277]: the trait bound `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>: argmin::prelude::ArgminMul<f64, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>` is not satisfied
--> examples/trustregion_nd.rs:66:15
|
66 | let res = Executor::new(cost, solver, init_param)
| ^^^^^^^^^^^^^ the trait `argmin::prelude::ArgminMul<f64, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>` is not implemented for `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>`
|
= note: required because of the requirements on the impl of `argmin::prelude::Solver<Rosenbrock>` for `argmin::solver::trustregion::TrustRegion<argmin::solver::trustregion::Dogleg>`
= note: required by `argmin::prelude::Executor::<O, S>::new`
error[E0277]: the trait bound `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>: argmin::prelude::ArgminNorm<f64>` is not satisfied
--> examples/trustregion_nd.rs:66:15
|
66 | let res = Executor::new(cost, solver, init_param)
| ^^^^^^^^^^^^^ the trait `argmin::prelude::ArgminNorm<f64>` is not implemented for `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>`
|
= note: required because of the requirements on the impl of `argmin::prelude::Solver<Rosenbrock>` for `argmin::solver::trustregion::TrustRegion<argmin::solver::trustregion::Dogleg>`
= note: required by `argmin::prelude::Executor::<O, S>::new`
error[E0277]: the trait bound `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>: argmin::prelude::ArgminDot<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>, f64>` is not satisfied
--> examples/trustregion_nd.rs:66:15
|
66 | let res = Executor::new(cost, solver, init_param)
| ^^^^^^^^^^^^^ the trait `argmin::prelude::ArgminDot<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>, f64>` is not implemented for `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>`
|
= note: required because of the requirements on the impl of `argmin::prelude::Solver<Rosenbrock>` for `argmin::solver::trustregion::TrustRegion<argmin::solver::trustregion::Dogleg>`
= note: required by `argmin::prelude::Executor::<O, S>::new`
error[E0277]: the trait bound `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>: argmin::prelude::ArgminAdd<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>` is not satisfied
--> examples/trustregion_nd.rs:66:15
|
66 | let res = Executor::new(cost, solver, init_param)
| ^^^^^^^^^^^^^ the trait `argmin::prelude::ArgminAdd<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>` is not implemented for `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>`
|
= note: required because of the requirements on the impl of `argmin::prelude::Solver<Rosenbrock>` for `argmin::solver::trustregion::TrustRegion<argmin::solver::trustregion::Dogleg>`
= note: required by `argmin::prelude::Executor::<O, S>::new`
error[E0277]: the trait bound `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>: argmin::prelude::ArgminSub<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>` is not satisfied
--> examples/trustregion_nd.rs:66:15
|
66 | let res = Executor::new(cost, solver, init_param)
| ^^^^^^^^^^^^^ the trait `argmin::prelude::ArgminSub<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>` is not implemented for `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>`
|
= note: required because of the requirements on the impl of `argmin::prelude::Solver<Rosenbrock>` for `argmin::solver::trustregion::TrustRegion<argmin::solver::trustregion::Dogleg>`
= note: required by `argmin::prelude::Executor::<O, S>::new`
error[E0277]: the trait bound `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>: argmin::prelude::ArgminZeroLike` is not satisfied
--> examples/trustregion_nd.rs:66:15
|
66 | let res = Executor::new(cost, solver, init_param)
| ^^^^^^^^^^^^^ the trait `argmin::prelude::ArgminZeroLike` is not implemented for `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>`
|
= note: required because of the requirements on the impl of `argmin::prelude::Solver<Rosenbrock>` for `argmin::solver::trustregion::TrustRegion<argmin::solver::trustregion::Dogleg>`
= note: required by `argmin::prelude::Executor::<O, S>::new`
error[E0277]: the trait bound `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>: argmin::prelude::ArgminDot<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>` is not satisfied
--> examples/trustregion_nd.rs:66:15
|
66 | let res = Executor::new(cost, solver, init_param)
| ^^^^^^^^^^^^^ the trait `argmin::prelude::ArgminDot<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>` is not implemented for `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>`
|
= note: required because of the requirements on the impl of `argmin::prelude::Solver<Rosenbrock>` for `argmin::solver::trustregion::TrustRegion<argmin::solver::trustregion::Dogleg>`
= note: required by `argmin::prelude::Executor::<O, S>::new`
error[E0277]: the trait bound `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>: argmin::prelude::ArgminInv<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>>` is not satisfied
--> examples/trustregion_nd.rs:66:15
|
66 | let res = Executor::new(cost, solver, init_param)
| ^^^^^^^^^^^^^ the trait `argmin::prelude::ArgminInv<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>>` is not implemented for `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>`
|
= note: required because of the requirements on the impl of `argmin::prelude::Solver<argmin::prelude::OpWrapper<Rosenbrock>>` for `argmin::solver::trustregion::Dogleg`
= note: required because of the requirements on the impl of `argmin::prelude::Solver<Rosenbrock>` for `argmin::solver::trustregion::TrustRegion<argmin::solver::trustregion::Dogleg>`
= note: required by `argmin::prelude::Executor::<O, S>::new`
error[E0599]: no method named `forward_diff` found for type `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>` in the current scope
--> examples/sr1_trustregion.rs:36:17
|
36 | Ok((*p).forward_diff(&|x| rosenbrock(&x.to_vec(), self.a, self.b)))
| ^^^^^^^^^^^^
error[E0599]: no method named `forward_hessian` found for type `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>` in the current scope
--> examples/sr1_trustregion.rs:45:17
|
45 | Ok((*p).forward_hessian(&|x| self.gradient(&x).unwrap()))
| ^^^^^^^^^^^^^^^
error[E0277]: the trait bound `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>: argmin_core::math::ArgminSub<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>` is not satisfied
--> examples/sr1_trustregion.rs:69:15
|
69 | let res = Executor::new(cost, solver, init_param)
| ^^^^^^^^^^^^^ the trait `argmin_core::math::ArgminSub<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>` is not implemented for `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>`
|
= note: required because of the requirements on the impl of `argmin_core::Solver<Rosenbrock>` for `argmin::solver::quasinewton::SR1TrustRegion<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>, argmin::solver::trustregion::Steihaug<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>>`
= note: required by `argmin_core::executor::Executor::<O, S>::new`
error[E0277]: the trait bound `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>: argmin_core::math::ArgminAdd<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>` is not satisfied
--> examples/sr1_trustregion.rs:69:15
|
69 | let res = Executor::new(cost, solver, init_param)
| ^^^^^^^^^^^^^ the trait `argmin_core::math::ArgminAdd<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>` is not implemented for `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>`
|
= note: required because of the requirements on the impl of `argmin_core::Solver<Rosenbrock>` for `argmin::solver::quasinewton::SR1TrustRegion<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>, argmin::solver::trustregion::Steihaug<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>>`
= note: required by `argmin_core::executor::Executor::<O, S>::new`
error[E0277]: the trait bound `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>: argmin_core::math::ArgminDot<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>, f64>` is not satisfied
--> examples/sr1_trustregion.rs:69:15
|
69 | let res = Executor::new(cost, solver, init_param)
| ^^^^^^^^^^^^^ the trait `argmin_core::math::ArgminDot<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>, f64>` is not implemented for `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>`
|
= note: required because of the requirements on the impl of `argmin_core::Solver<Rosenbrock>` for `argmin::solver::quasinewton::SR1TrustRegion<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>, argmin::solver::trustregion::Steihaug<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>>`
= note: required by `argmin_core::executor::Executor::<O, S>::new`
error[E0277]: the trait bound `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>: argmin_core::math::ArgminDot<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>>` is not satisfied
--> examples/sr1_trustregion.rs:69:15
|
69 | let res = Executor::new(cost, solver, init_param)
| ^^^^^^^^^^^^^ the trait `argmin_core::math::ArgminDot<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>>` is not implemented for `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>`
|
= note: required because of the requirements on the impl of `argmin_core::Solver<Rosenbrock>` for `argmin::solver::quasinewton::SR1TrustRegion<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>, argmin::solver::trustregion::Steihaug<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>>`
= note: required by `argmin_core::executor::Executor::<O, S>::new`
error[E0277]: the trait bound `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>: argmin_core::math::ArgminNorm<f64>` is not satisfied
--> examples/sr1_trustregion.rs:69:15
|
69 | let res = Executor::new(cost, solver, init_param)
| ^^^^^^^^^^^^^ the trait `argmin_core::math::ArgminNorm<f64>` is not implemented for `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>`
|
= note: required because of the requirements on the impl of `argmin_core::Solver<Rosenbrock>` for `argmin::solver::quasinewton::SR1TrustRegion<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>, argmin::solver::trustregion::Steihaug<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>>`
= note: required by `argmin_core::executor::Executor::<O, S>::new`
error[E0277]: the trait bound `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>: argmin_core::math::ArgminZeroLike` is not satisfied
--> examples/sr1_trustregion.rs:69:15
|
69 | let res = Executor::new(cost, solver, init_param)
| ^^^^^^^^^^^^^ the trait `argmin_core::math::ArgminZeroLike` is not implemented for `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>`
|
= note: required because of the requirements on the impl of `argmin_core::Solver<Rosenbrock>` for `argmin::solver::quasinewton::SR1TrustRegion<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>, argmin::solver::trustregion::Steihaug<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>>`
= note: required by `argmin_core::executor::Executor::<O, S>::new`
error[E0277]: the trait bound `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>: argmin_core::math::ArgminMul<f64, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>` is not satisfied
--> examples/sr1_trustregion.rs:69:15
|
69 | let res = Executor::new(cost, solver, init_param)
| ^^^^^^^^^^^^^ the trait `argmin_core::math::ArgminMul<f64, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>` is not implemented for `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>`
|
= note: required because of the requirements on the impl of `argmin_core::Solver<Rosenbrock>` for `argmin::solver::quasinewton::SR1TrustRegion<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>, argmin::solver::trustregion::Steihaug<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>>`
= note: required by `argmin_core::executor::Executor::<O, S>::new`
error[E0277]: the trait bound `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>: argmin_core::math::ArgminSub<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>>` is not satisfied
--> examples/sr1_trustregion.rs:69:15
|
69 | let res = Executor::new(cost, solver, init_param)
| ^^^^^^^^^^^^^ the trait `argmin_core::math::ArgminSub<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>>` is not implemented for `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>`
|
= note: required because of the requirements on the impl of `argmin_core::Solver<Rosenbrock>` for `argmin::solver::quasinewton::SR1TrustRegion<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>, argmin::solver::trustregion::Steihaug<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>>`
= note: required by `argmin_core::executor::Executor::<O, S>::new`
error[E0277]: the trait bound `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>: argmin_core::math::ArgminDot<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>` is not satisfied
--> examples/sr1_trustregion.rs:69:15
|
69 | let res = Executor::new(cost, solver, init_param)
| ^^^^^^^^^^^^^ the trait `argmin_core::math::ArgminDot<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>` is not implemented for `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>`
|
= note: required because of the requirements on the impl of `argmin_core::Solver<Rosenbrock>` for `argmin::solver::quasinewton::SR1TrustRegion<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>, argmin::solver::trustregion::Steihaug<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>>`
= note: required by `argmin_core::executor::Executor::<O, S>::new`
error[E0277]: the trait bound `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>: argmin_core::math::ArgminDot<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>>` is not satisfied
--> examples/sr1_trustregion.rs:69:15
|
69 | let res = Executor::new(cost, solver, init_param)
| ^^^^^^^^^^^^^ the trait `argmin_core::math::ArgminDot<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>>` is not implemented for `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>`
|
= note: required because of the requirements on the impl of `argmin_core::Solver<Rosenbrock>` for `argmin::solver::quasinewton::SR1TrustRegion<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>, argmin::solver::trustregion::Steihaug<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>>`
= note: required by `argmin_core::executor::Executor::<O, S>::new`
error[E0277]: the trait bound `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>: argmin_core::math::ArgminAdd<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>>` is not satisfied
--> examples/sr1_trustregion.rs:69:15
|
69 | let res = Executor::new(cost, solver, init_param)
| ^^^^^^^^^^^^^ the trait `argmin_core::math::ArgminAdd<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>>` is not implemented for `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>`
|
= note: required because of the requirements on the impl of `argmin_core::Solver<Rosenbrock>` for `argmin::solver::quasinewton::SR1TrustRegion<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>, argmin::solver::trustregion::Steihaug<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>>`
= note: required by `argmin_core::executor::Executor::<O, S>::new`
error[E0277]: the trait bound `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>: argmin_core::math::ArgminMul<f64, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>>` is not satisfied
--> examples/sr1_trustregion.rs:69:15
|
69 | let res = Executor::new(cost, solver, init_param)
| ^^^^^^^^^^^^^ the trait `argmin_core::math::ArgminMul<f64, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>>` is not implemented for `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>`
|
= note: required because of the requirements on the impl of `argmin_core::Solver<Rosenbrock>` for `argmin::solver::quasinewton::SR1TrustRegion<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>, argmin::solver::trustregion::Steihaug<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>>`
= note: required by `argmin_core::executor::Executor::<O, S>::new`
error[E0277]: the trait bound `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>: argmin_core::math::ArgminMul<f64, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>` is not satisfied
--> examples/sr1_trustregion.rs:61:38
|
61 | let subproblem = Steihaug::new().max_iters(20);
| ^^^^^^^^^ the trait `argmin_core::math::ArgminMul<f64, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>` is not implemented for `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>`
error[E0277]: the trait bound `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>: argmin_core::math::ArgminDot<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>, f64>` is not satisfied
--> examples/sr1_trustregion.rs:61:38
|
61 | let subproblem = Steihaug::new().max_iters(20);
| ^^^^^^^^^ the trait `argmin_core::math::ArgminDot<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>, f64>` is not implemented for `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>`
error[E0277]: the trait bound `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>: argmin_core::math::ArgminAdd<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>` is not satisfied
--> examples/sr1_trustregion.rs:61:38
|
61 | let subproblem = Steihaug::new().max_iters(20);
| ^^^^^^^^^ the trait `argmin_core::math::ArgminAdd<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>` is not implemented for `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>`
error[E0277]: the trait bound `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>: argmin_core::math::ArgminMul<f64, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>` is not satisfied
--> examples/sr1_trustregion.rs:61:22
|
61 | let subproblem = Steihaug::new().max_iters(20);
| ^^^^^^^^^^^^^ the trait `argmin_core::math::ArgminMul<f64, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>` is not implemented for `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>`
|
= note: required by `argmin::solver::trustregion::Steihaug::<P>::new`
error[E0277]: the trait bound `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>: argmin_core::math::ArgminDot<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>, f64>` is not satisfied
--> examples/sr1_trustregion.rs:61:22
|
61 | let subproblem = Steihaug::new().max_iters(20);
| ^^^^^^^^^^^^^ the trait `argmin_core::math::ArgminDot<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>, f64>` is not implemented for `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>`
|
= note: required by `argmin::solver::trustregion::Steihaug::<P>::new`
error[E0277]: the trait bound `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>: argmin_core::math::ArgminAdd<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>` is not satisfied
--> examples/sr1_trustregion.rs:61:22
|
61 | let subproblem = Steihaug::new().max_iters(20);
| ^^^^^^^^^^^^^ the trait `argmin_core::math::ArgminAdd<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>, ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>` is not implemented for `ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>`
|
= note: required by `argmin::solver::trustregion::Steihaug::<P>::new`
error[E0599]: no method named `add_observer` found for type `argmin_core::executor::Executor<Rosenbrock, argmin::solver::quasinewton::SR1TrustRegion<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>, argmin::solver::trustregion::Steihaug<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>>>` in the current scope
--> examples/sr1_trustregion.rs:70:10
|
70 | .add_observer(ArgminSlogLogger::term(), ObserverMode::Always)
| ^^^^^^^^^^^^
|
= note: the method `add_observer` exists but the following trait bounds were not satisfied:
`argmin::solver::quasinewton::SR1TrustRegion<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 2]>>, argmin::solver::trustregion::Steihaug<ndarray::ArrayBase<ndarray::OwnedRepr<f64>, ndarray::Dim<[usize; 1]>>>> : argmin_core::Solver<Rosenbrock>`
error[E0599]: no method named `add_observer` found for type `argmin::prelude::Executor<Rosenbrock, argmin::solver::trustregion::TrustRegion<argmin::solver::trustregion::Dogleg>>` in the current scope
--> examples/trustregion_nd.rs:67:10
|
67 | .add_observer(ArgminSlogLogger::term(), ObserverMode::Always)
| ^^^^^^^^^^^^
|
= note: the method `add_observer` exists but the following trait bounds were not satisfied:
`argmin::solver::trustregion::TrustRegion<argmin::solver::trustregion::Dogleg> : argmin::prelude::Solver<Rosenbrock>`
warning: unused import: `argmin_core::finitediff`
--> examples/sr1_trustregion.rs:15:5
|
15 | use argmin_core::finitediff::*;
| ^^^^^^^^^^^^^^^^^^^^^^^
|
= note: #[warn(unused_imports)] on by default
error: aborting due to 21 previous errors
Some errors occurred: E0277, E0599.
For more information about an error, try `rustc --explain E0277`.
error: aborting due to 9 previous errors
Some errors occurred: E0277, E0599.
For more information about an error, try `rustc --explain E0277`.
error: Could not compile `argmin`.
warning: build failed, waiting for other jobs to finish...
error: Could not compile `argmin`.
warning: build failed, waiting for other jobs to finish...
error: build failed
I want to use the nalgebra support for argmin
in master, but I am having some difficulties. Specifically, I want to use NelderMead
optimization, but I have found that it requires the param to be Default. I am using nalgebra::DMatrix
. DMatrix
does not implement Default
. I looked at the implementation of NelderMead
, and it seems that it does not utilize the Default
bound at all. Is it possible to remove this bound so that I can use DMatrix
with it. Thank you!
Hi there!! Thank you for the great work!!
I am starting to use the crate and run into the following problem:
Implementing ArgminOp
, the method apply
, I would like to correctly handle errors. Then, I need to return an argmin::core::Error
, but it is private. I even tried with the suitable variant of argmin::core::ArgminError
, but it is also private.
Make errors public and easy to construct. And add the corresponding indications in the documentation when implementing the ArgminOp
trait.
I would like for something like this to work.
use argmin::prelude::*;
use serde::{Deserialize, Serialize};
#[derive(Clone, Default, Serialize, Deserialize)]
pub struct MyProblem;
impl ArgminOp for MyProblem {
type Param = ();
type Output = ();
type Hessian = ();
type Jacobian = ();
type Float = ();
/// Apply the cost function to a parameter `p`
fn apply(&self, _p: &Self::Param) -> Result<Self::Output, Error> {
argmin::core::ArgminError::InvalidParameter{text: String::from("MyProblem does not take this parameters")}
// or better
// argmin::error::ArgminError::InvalidParameter::new("Error message")
}
}
After I clone the repo, I can run cargo build
and it's find, but to cargo build examples, there are lots of errors:
error[E0277]: the trait bound `ArrayBase<OwnedRepr<f64>, Dim<[usize; 1]>>: ArgminSub<ArrayBase<OwnedRepr<f64>, Dim<[usize; 1]>>, ArrayBase<OwnedRepr<f64>, Dim<[usize; 1]>>>` is not satisfied
--> examples/bfgs.rs:53:15
|
53 | let res = Executor::new(cost, solver, init_param)
| ^^^^^^^^^^^^^ the trait `ArgminSub<ArrayBase<OwnedRepr<f64>, Dim<[usize; 1]>>, ArrayBase<OwnedRepr<f64>, Dim<[usize; 1]>>>` is not implemented for `ArrayBase<OwnedRepr<f64>, Dim<[usize; 1]>>`
|
= note: required because of the requirements on the impl of `Solver<Rosenbrock>` for `BFGS<MoreThuenteLineSearch<ArrayBase<OwnedRepr<f64>, Dim<[usize; 1]>>, f64>, ArrayBase<OwnedRepr<f64>, Dim<[usize; 2]>>, f64>`
= note: required by `argmin::core::Executor::<O, S>::new`
Use the following as suggested in #136 :
cargo clippy --all-targets --all-features -- -D warnings
I am pretty experienced with Rust, but I have no idea how to use this, although I think if someone could just tell me how to set it up I would be golden. This seems like the exact thing I need for the job, but I can't get past the nomenclature.
I'm working on a tool that produces a GIF with @kornelski's most excellent Gifski tool. The problem is finding the right parameters in order to get as close to 15 MB without going over.
Variables:
I may also use butteraugli metrics as extra data points to feed in. Was surprisingly easy to compile given their recommendation to give Bazel another try (I'll pass).
Anyways, where should I start with argmin on a problem like this? I have a bunch of variables I want to tweak in order to maximize a desired outcome. Right now I've been doing alchemy by changing weights and biases for things and tweaking formulas on my own and I would definitely prefer a more general solution. I'll keep re-reading the docs and hope I have an epiphany just in case I don't hear back ;)
Also also...thanks for the hard work to all involved! Seems like a great project, just wish I knew how to wrap my head around it, haha!
When creating an instance of OpWrapper
, the passed in operator is cloned. This can be inefficient (particularly if a solver is called inside a solver, which means at least one copy of the operator per iteration of the "outer" solver) and requires the operators to implement Clone
. Removing the clone may however necessitate a change of the overall design.
I just ran a cargo install
install on https://github.com/rust-cv/cv/tree/main/vslam-sandbox. It seems that the new version of argmin
released within the last hour broke the build, despite being labeled as semantically compatible.
It seems to be related to nalgebra support. Below is a resulting error (which did not occur in the previous v0.4.6, which I was building with earlier in the day):
error[E0277]: the trait bound `Matrix<f64, Const<6_usize>, Const<1_usize>, ArrayStorage<f64, 6_usize, 1_usize>>: ArgminAdd<Matrix<f64, Const<6_usize>, Const<1_usize>, ArrayStorage<f64, 6_usize, 1_usize>>, Matrix<f64, Const<6_usize>, Const<1_usize>, ArrayStorage<f64, 6_usize, 1_usize>>>` is not satisfied
--> cv-optimize/src/single_view_optimizer.rs:22:43
|
22 | NelderMead::new().with_initial_params(variants)
| ^^^^^^^^ the trait `ArgminAdd<Matrix<f64, Const<6_usize>, Const<1_usize>, ArrayStorage<f64, 6_usize, 1_usize>>, Matrix<f64, Const<6_usize>, Const<1_usize>, ArrayStorage<f64, 6_usize, 1_usize>>>` is not implemented for `Matrix<f64, Const<6_usize>, Const<1_usize>, ArrayStorage<f64, 6_usize, 1_usize>>`
I coded a golden section search in our enterprise project and I wonder if you want it in argmin. It's a simple algorithm with a cost function, nothing complicated.
fn golden_section_search(
cost_function: &F,
min_bound: f32,
max_bound: f32,
init_estimate: f32
) -> f32 { ... }
I just need to take the time to port it to argmin design and add a test. Are you interested in this contribution?
Calling the function Executor::run
from web assembly panics with the following error:
test result: FAILED. 1 passed; 1 failed; 0 ignored
console.log div contained:
panicked at 'time not implemented on this platform', library/std/src/sys/wasm/../unsupported/time.rs:13:9
This is probably because of the use of std::time
, which is not supported in a browser.
To reproduce the error follow these steps:
wasm-pack build
wasm-pack test
You need to install wasm-pack, if you don't have it already.
First of all, I'd like to thank everyone involved in making this crate. I've been using it in multiple places in a project of mine for the past few months, and I love the ergonomics and find the crate a joy to use - Kudos!
Anyway, I was doing some performance tuning on my code recently and discovered that in my calls to Executor::run
, only about 44% of processor time was spent inside my ArgminOp::apply
. When looking at where the rest of the time was being spent, I noticed three main areas:
alloc::fmt::format
, ArgminKV::merge
, and drop ArgminKV
. This would be expected if I was using an observer, but this overhead happens even with no registered observers.Instant::elapsed
and Instant::now
. These are called even if I don't care about the elapsed time (as is often the case once I've finished tuning the optimisation params)`.ArgminCheckpoint::store_cond
. This allocates a string, concats it with another string, and deallocs the result on exit - every single time through the Executor::run
loop, even if Checkpoints are off, or this loop is not a checkpointed loop.Here's a section of my flamechart from cargo-flamechart to illustrate this.
Green boxes highlight my code inside the apply function, magenta is for point (1) above, red for point (2), and blue for point (3)
I've got a branch in my fork where I've addressed points (1) and (3) by some refactoring, which I don't think will cause any issues (but you all know better than me, I may be wrong!). I've also addressed (2) by introducing a config flag to make the timer optional, similarly to how enabling/disabling Ctrl-C works. You can see my changes here: sdd@a67f56a
With the modifications above in place, I get a 44% speedup, and I can see that the executor now spends 93% of its time running my ArgminOp::apply
, rather than 44% of the time. My cargo flamechart now looks more like this:
I realize that my workload may be atypical to many others, as it might be quite light. I'm really just doing some matrix and trig operations. But, even with intensive apply
functions, some performace gains are there to be had for others.
I'm happy to raise a PR and whip it into the kind of shape you'd expect it to be in if you think this is a worthwhile change.
Thanks!
Checkpointing currently serializes all of the Executor
, which requires the ArgminOp
to be serializable (even if no checkpointing is used). However, one actually only needs to serialize the state
and the solver
field, the other things can be supplied when starting from a loaded checkpoint.
So instead of
let res = Executor::from_checkpoint(".checkpoints/landweber_exec.arg")
.unwrap_or(Executor::new(operator, solver, init_param))
.max_iters(iters)
.checkpoint_dir(".checkpoints")
.checkpoint_name("landweber_exec")
.checkpoint_mode(CheckpointMode::Every(20))
.add_observer(ArgminSlogLogger::term(), ObserverMode::Always)
.run()?;
One could write this like this:
let res = Executor::from_checkpoint(".checkpoints/landweber_exec.arg", &operator)
.unwrap_or(Executor::new(operator, solver, init_param))
.max_iters(iters)
.checkpoint_dir(".checkpoints")
.checkpoint_name("landweber_exec")
.checkpoint_mode(CheckpointMode::Every(20))
.add_observer(ArgminSlogLogger::term(), ObserverMode::Always)
.run()?;
This would, to a large extent, solve #7 .
Caveat:
Since solvers may have internal state, they need to be part of the checkpointing.
I'm open to suggestions.
One of the method I coded in Rust is the "Conjugate gradient solver for positive-definite matrices." I see that you already coded this method. I didn't compare them yet because there's a small difference: I'm using a diagonal conditioner
for idx_iter in 1..max_iterations {
// We're using the diagonal conditioner.
let z = &r / &diag;
... standard algo, I think ...
As you know, I'm not a mathematician at all, so I can't explain you what this is and why we need it in argmin. Maybe almost nobody uses that and we shouldn't code it. I don't know. This looks like a new trait and a new template parameter, something you may be tired of :)
Is there a minimum-effort way to compare our algo? Like, using what you already coded to update the array or some magic that you know?
As I try to upgrade linfa
to ndarray 0.14
(cf. rust-ml/linfa#110), I noticed that argmin 0.4.2
has ndarray 0.14
and ndarray-rand 0.14
dependencies.
Unfortunately ndarray-rand 0.14
depends on ndarray 0.15
, it should be ndarray-rand 0.13
to depend onndarray 0.14
.
It could explain the compilation problems I get (?) Strangely even if I specify argmin 0.4.1
it pulls argmin 0.4.2
(not sure what is going on here!).
I would suggest to release argmin 0.4.3
reverting ndarray-rand
to 0.13
.
Rename master
branch to main
It seems as if failure may not be the best choice. Moving to custom error types is however probably quite a big change.
There seems to be a compilation error caused by the type resolution in the latest nightly compiler. This broke our CI system rust-ml/linfa#118 yesterday ๐ For cargo 1.52.0-nightly (1e8703890 2021-03-26)
everything compiles fine, but if I update my toolchain to cargo 1.53.0-nightly (65d57e6f3 2021-04-04)
I'm getting the following error:
Compiling argmin v0.4.1
error[E0282]: type annotations needed
--> src/solver/particleswarm/mod.rs:129:9
|
129 | impl<O, P, F> Solver<O> for ParticleSwarm<P, F>
| ^ cannot infer type for type parameter `P`
error[E0282]: type annotations needed
--> src/solver/quasinewton/bfgs.rs:62:12
|
62 | impl<O, L, H, F> Solver<O> for BFGS<L, H, F>
| ^ cannot infer type for type parameter `H`
error[E0282]: type annotations needed
--> src/solver/quasinewton/dfp.rs:52:12
|
52 | impl<O, L, H, F> Solver<O> for DFP<L, H, F>
| ^ cannot infer type for type parameter `H`
error[E0282]: type annotations needed
--> src/solver/quasinewton/sr1.rs:78:12
|
78 | impl<O, L, H, F> Solver<O> for SR1<L, H, F>
| ^ cannot infer type for type parameter `H`
error[E0282]: type annotations needed
--> src/solver/quasinewton/sr1_trustregion.rs:100:9
|
100 | impl<O, B, R, F> Solver<O> for SR1TrustRegion<B, R, F>
| ^ cannot infer type for type parameter `B`
Currently, stopping criteria are defined during the implementation of the solver. However, sometimes it might be useful if users could add their own stopping criteria. This would require a complete redesign of how termination works. One of the key aspects of termination is that it needs access to the fields of the solver which makes it difficult to implement in a generic way. However, similar to how observers work, the needed variables could be stored in a key-value store and passed to the user-defined termination function. It needs to be evaluated how efficient such an approach would be.
Note: I had to switch to building with the nightly toolchain because of issues with ndalgebra v0.26.2
due to "feature resolver
isrequired"
I attempted to compile and run the Nelder-Mead example using the following in my Cargo.toml
:
[dependencies]
argmin = { version = "0.4.4", features = ["ctrlc", "ndarrayl", "nalgebral"] }
argmin_testfunctions = "*"
ndarray = { version = "0.15", features = ["serde"] }
Adding features = ["serde"]
was added to eliminate Serialize
and Deserialize
errors. Now I am getting the following errors:
error[E0277]: the trait bound `ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>: ArgminAdd<ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>, ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>>` is not satisfied
--> src\main.rs:36:30
|
36 | .with_initial_params(vec![
| ______________________________^
37 | | // array![-2.0, 3.0],
38 | | // array![-2.0, -1.0],
39 | | // array![2.0, -1.0],
... |
42 | | array![2.0, -1.0],
43 | | ])
| |_________^ the trait `ArgminAdd<ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>, ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>>` is not implemented for `ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>`
|
= note: this error originates in a macro (in Nightly builds, run with -Z macro-backtrace for more info)
error[E0277]: the trait bound `ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>: ArgminSub<ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>, ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>>` is not satisfied
--> src\main.rs:36:30
|
36 | .with_initial_params(vec![
| ______________________________^
37 | | // array![-2.0, 3.0],
38 | | // array![-2.0, -1.0],
39 | | // array![2.0, -1.0],
... |
42 | | array![2.0, -1.0],
43 | | ])
| |_________^ the trait `ArgminSub<ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>, ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>>` is not implemented for `ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>`
|
= note: this error originates in a macro (in Nightly builds, run with -Z macro-backtrace for more info)
error[E0277]: the trait bound `ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>: ArgminMul<_, ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>>` is not satisfied
--> src\main.rs:36:30
|
36 | .with_initial_params(vec![
| ______________________________^
37 | | // array![-2.0, 3.0],
38 | | // array![-2.0, -1.0],
39 | | // array![2.0, -1.0],
... |
42 | | array![2.0, -1.0],
43 | | ])
| |_________^ the trait `ArgminMul<_, ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>>` is not implemented for `ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>`
|
= note: this error originates in a macro (in Nightly builds, run with -Z macro-backtrace for more info)
error[E0277]: the trait bound `ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>: ArgminAdd<ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>, ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>>` is not satisfied
--> src\main.rs:35:18
|
35 | let solver = NelderMead::new()
| ^^^^^^^^^^^^^^^ the trait `ArgminAdd<ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>, ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>>` is not implemented for `ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>`
|
= note: required by `NelderMead::<P, F>::new`
error[E0277]: the trait bound `ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>: ArgminSub<ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>, ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>>` is not satisfied
--> src\main.rs:35:18
|
35 | let solver = NelderMead::new()
| ^^^^^^^^^^^^^^^ the trait `ArgminSub<ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>, ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>>` is not implemented for `ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>`
|
= note: required by `NelderMead::<P, F>::new`
error[E0277]: the trait bound `ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>: ArgminMul<_, ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>>` is not satisfied
--> src\main.rs:35:18
|
35 | let solver = NelderMead::new()
| ^^^^^^^^^^^^^^^ the trait `ArgminMul<_, ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>>` is not implemented for `ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>`
|
= note: required by `NelderMead::<P, F>::new`
error[E0599]: the method `sd_tolerance` exists for struct `NelderMead<ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>, _>`, but its trait bounds were not satisfied
--> src\main.rs:44:10
|
44 | .sd_tolerance(0.0001);
| ^^^^^^^^^^^^ private field, not a method
|
::: C:\Users\Nobody\.cargo\registry\src\github.com-1ecc6299db9ec823\ndarray-0.15.1\src\lib.rs:1276:1
|
1276 | / pub struct ArrayBase<S, D>
1277 | | where
1278 | | S: RawData,
1279 | | {
... |
1289 | | strides: D,
1290 | | }
| | -
| | |
| | doesn't satisfy `_: ArgminAdd<ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>, ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>>`
| |_doesn't satisfy `_: ArgminMul<_, ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>>`
| doesn't satisfy `_: ArgminSub<ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>, ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>>`
|
= note: the following trait bounds were not satisfied:
`ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>: ArgminAdd<ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>, ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>>`
`ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>: ArgminSub<ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>, ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>>`
`ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>: ArgminMul<_, ArrayBase<OwnedRepr<{float}>, Dim<[usize; 1]>>>`
Let me know if there is any further information I can provide.
This example on the master branch seems a little outdated. Only by reading through other issues in this repo I was able to find this new example.
I'd like to suggest that at least the new example should get referenced in the newest documentation. https://docs.rs/argmin/0.4.7/argmin/solver/simulatedannealing/struct.SimulatedAnnealing.html Currently, it links to the old, outdated one.
Some optimization algorithms require several independent cost function evaluations per iteration. Sometimes (actually most of the times in my applications), evaluating the cost function can be costly, and its evaluations dominate the optimization time. When the cost function itself cannot be parallelized, an obvious improvement is to evaluate it at these points in parallel. It could also be desirable to evaluate the cost function and its gradient or hessian in parallel when applicable.
rayon
is the perfect library to allow this kind of things. Would it be okay to add it as a dependency? As a feature? Should we create an option to enable parallel evaluation on demand? We could probably even decide automatically whether it is worth to enable it or not, depending on the duration of the first few evaluations.
It would be useful to add tests that would run the optimization algorithm and check that the expected convergence is reached.
For instance, maybe translating scipy/optimize/test/test_optimize.py
could be a start.
I will look into it.
Edit: actually it looks like there is a number of classical functions implemented in https://github.com/argmin-rs/argmin-testfunctions already..
This is a long-term issue in order to keep these things in mind.
as_
, to_
, into_
conventions (C-CONV)iter
, iter_mut
, into_iter
(C-ITER)Copy
, Clone
, Eq
, PartialEq
, Ord
, PartialOrd
, Hash
, Debug
,Display
, Default
From
, AsRef
, AsMut
(C-CONV-TRAITS)FromIterator
and Extend
(C-COLLECT)Serialize
, Deserialize
(C-SERDE)Send
and Sync
where possible (C-SEND-SYNC)Hex
, Octal
, Binary
formatting (C-NUM-FMT)R: Read
and W: Write
by value (C-RW-VALUE)?
, not try!
, not unwrap
(C-QUESTION-MARK)Deref
and DerefMut
(C-DEREF)bool
or Option
(C-CUSTOM-TYPE)bitflags
, not enums (C-BITFLAG)Debug
(C-DEBUG)Debug
representation is never empty (C-DEBUG-NONEMPTY)I'm sorry the example is quite big, but isolating a smaller section of the codebase for this is nontrivial.
git clone https://github.com/yukarichess/yukari -b logistic-regression
cd yukari
xz -d quiescent_positions_with_results.xz
cargo run --release --example tune
e.g. if you uncomment the ArmijoCondition/BacktrackingLineSearch in yukari/src/tune.rs
:
Loading FENs...
Found 1336010 positions
0.41249916
Jul 28 00:41:34.650 INFO L-BFGS, max_iters: 100
0.41249916
0.41249916
0.41249916
0.41249916
0.41249916
0.41249916
0.41249916
0.41249916
0.41249916
0.41249916
0.41249916
0.41249916
0.41249916
0.41249916
0.41249916
0.41249916
Jul 28 00:41:47.101 INFO iter: 0, cost: 0.4124991579283057, best_cost: 0.4124991579283057, cost_func_count: 1, grad_func_count: 2, jacobian_func_count: 0, hessian_func_count: 0, modify_func_count: 0, gamma: 1.0, time: 12.4510831
Jul 28 00:41:48.829 INFO iter: 1, cost: 0.4124991579283057, best_cost: 0.4124991579283057, cost_func_count: 1, grad_func_count: 3, jacobian_func_count: 0, hessian_func_count: 0, modify_func_count: 0, gamma: -16734916.191364305, time: 1.7277031
ArgminResult:
param (best): [snipped]
cost (best): 0.4124991579283057
iters (best): 1
iters (total): 2
termination: No change in cost function value
time: Some(16.8272069s)
Loading FENs...
Found 1336010 positions
0.41249916
Jul 28 00:23:52.412 INFO L-BFGS, max_iters: 100
0.41249916
0.41249916
0.41249916
0.41249915
0.41249914
0.41249908
0.41249884
0.41249788
0.41249421
0.41248170
0.41246661
0.41296339
0.41265837
0.41254610
0.41250183
0.41248326
0.41247498
0.41247106
0.41246910
0.41246806
0.41246748
0.41246714
0.41246694
0.41246682
0.41246674
0.41246669
0.41246666
0.41246665
0.41246663
0.41246663
0.41246662
0.41246662
0.41246662
0.41246661
0.41246661
0.41246661
0.41246661
0.41246661
0.41246661
0.41246661
0.41246661
0.41246661
0.41246661
0.41246661
0.41246661
0.41246661
0.41246661
0.41246661
0.41246661
0.41246661
0.41246661
0.41246661
0.41246661
0.41246661
0.41246661
Jul 28 00:26:26.145 INFO iter: 0, cost: 0.41246661320768635, best_cost: 0.41246661320768635, cost_func_count: 1, grad_func_count: 2, jacobian_func_count: 0, hessian_func_count: 0, modify_func_count: 0, gamma: 1.0, time: 153.7329519
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Condition violated: "MoreThuenteLineSearch: Search direction must be a descent direction."', yukari/examples/tune.rs:39:17
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Aborted
I think this means that L-BFGS is going in the wrong direction? I admit to being very new to logistic regression, but this definitely seems like a bug to me.
Automatic Differentiation (AD) generates the derivative of arbitrary order n
for an input function and is heavily used in Machine Learning. Perhaps we can implement one behind a feature flag. Some previous attempts are:
I imagine a rust macro which takes a normal pure function and automatically implements the ArgminOp
trait. An overview and starting point can be found here: https://arxiv.org/abs/1502.05767
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.