benfred / fmin

Unconstrained function minimization in Javascript

fmin

Unconstrained function minimization in javascript.

This package implements some basic numerical optimization algorithms: Nelder-Mead, Gradient Descent, Wolf Line Search and Non-Linear Conjugate Gradient methods are all provided.

Interactive visualizations with D3 explaining how these algorithms work are also included in this package. Descriptions of the algorithms as well as most of the visualizations are available on my blog post An Interactive Tutorial on Numerical Optimization.

Installing

If you use NPM, npm install fmin. Otherwise, download the latest release.

API Reference

# nelderMead(f, initial)

Uses the Nelder-Mead method to minimize a function f starting at location initial.

Example usage minimizing the function f(x, y) = x² + y² + x sin y + y sin x is:

function loss(X) {
    var x = X[0], y = X[1];
    return Math.sin(y) * x  + Math.sin(x) * y  +  x * x +  y *y;
}

var solution = fmin.nelderMead(loss, [-3.5, 3.5]);
console.log("solution is at " + solution.x);

# conjugateGradient(f, initial)

Minimizes a function using the Polak–Ribière non-linear conjugate gradient method . The function f should compute both the loss and the gradient.

An example minimizing Rosenbrock's Banana function is:

function banana(X, fxprime) {
    fxprime = fxprime || [0, 0];
    var x = X[0], y = X[1];
    fxprime[0] = 400 * x * x * x - 400 * y * x + 2 * x - 2;
    fxprime[1] = 200 * y - 200 * x * x;
    return (1 - x) * (1 - x) + 100 * (y - x * x) * (y - x * x);
}

var solution = fmin.conjugateGradient(banana, [-1, 1]);
console.log("solution is at " + solution.x);

fmin's Issues

Nelder-Mead minTolerance not parsed from parameters passed

This looks to be a bug in the N-M implementation:

minErrorDelta = parameters.minErrorDelta || 1e-6,
minTolerance = parameters.minErrorDelta || 1e-5,

If Wolfe conditions not met, conjugateGradient runs through all iterations silently...

I think if the Wolfe conditions are not met and "a" in the loop below is zero, the code simply runs through maxIterations without reporting it or making in progress. If the user is unaware of this, it effectively creates an infinite loop where no progress is made.

I'm not sure how to solve this, but I would recommend simply "failing fast" by through an error if "a" is returned as 0 from wolfeLineSearch. Through an error back and let the user deal with it.

for (var i = 0; i < maxIterations; ++i) {
a = wolfeLineSearch(f, pk, current, next, a);

        // todo: history in wrong spot?
        if (params.history) {
            params.history.push({x: current.x.slice(),
                                 fx: current.fx,
                                 fxprime: current.fxprime.slice(),
                                 alpha: a});
        }

// console.log("XXX",i,current.x,current.fx,a);

    // NOTE: I think there is a bug here.  If a == 0, then we run throguh
    // all iterations without making any change at all; we are not using i.
    // If the Wolfe conditions fail, we should abort immediately or do something else.
        if (!a) {
            // faiiled to find point that satifies wolfe conditions.
            // reset direction for next iteration
            scale(pk, current.fxprime, -1);

        } else {
            // update direction using Polak–Ribiere CG method
            weightedSum(yk, 1, next.fxprime, -1, current.fxprime);

            var delta_k = dot(current.fxprime, current.fxprime),
                beta_k = Math.max(0, dot(yk, next.fxprime) / delta_k);

            weightedSum(pk, beta_k, pk, -1, next.fxprime);

            temp = current;
            current = next;
            next = temp;
        }

        if (norm2(current.fxprime) <= 1e-5) {
            break;
        }
    }

    if (params.history) {
        params.history.push({x: current.x.slice(),
                             fx: current.fx,
                             fxprime: current.fxprime.slice(),
                             alpha: a});
    }

    return current;
}

How to change contour plot color?

Hello,

I saw details on:
https://libraries.io/npm/contour_plot
But how it's binded on fmin?

Thanks

Can you release the latest version of fmin?

I found that fmin had been modified in 2018 without a new release. Can you release the latest version of fmin?

can't npm install (using node v8.7.0)

When I npm install fmin, I get

npm ERR! code MODULE_NOT_FOUND
npm ERR! Cannot find module 'internal/errors'

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/roberto/.npm/_logs/2018-02-09T19_26_55_019Z-debug.log

Security: Update dependencies

"fmin": "0.0.2" contains dependencies:

"tape": "4",
"uglify-js": "^2.6.0",

which both have vulnerabilities with fixes

fmin_vis.js ERR_FILE_NOT_FOUND

Hello,

On the release build the fmin_vis.js is not provided.
So any sample are testable.

Thanks

No graphs on Safari

Hello --

The current version of Safari on OSX 10.10.5 doesn't like something about your Javascript. It complains about an "unexpected token '>'" at fmin_vis.js:320 and doesn't show the animations.

Thanks!

vite打包报错

1.vue2项目
2.webpack打包的时候没有问题，换成vite打包出现
node_modules/venn.js/src/layout.js:1:76: error: Could not resolve "../node_modules/fmin/index.js"
1 │ import { nelderMead, bisect, conjugateGradient, zeros, zerosM, norm2 } from '../node_modules/fmin/index.js'
╵ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

node_modules/venn.js/src/diagram.js:6:27: error: Could not resolve "../node_modules/fmin/index.js"
6 │ import { nelderMead } from '../node_modules/fmin/index.js'
╵ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

error when starting dev server:
Error: Build failed with 2 errors:
node_modules/venn.js/src/diagram.js:6:27: error: Could not resolve "../node_modules/fmin/index.js"
node_modules/venn.js/src/layout.js:1:76: error: Could not resolve "../node_modules/fmin/index.js"
at failureErrorWithLog (E:\cms_working\jifen-cms\jifen-cms\node_modules\esbuild\lib\main.js:1493:15)
at E:\cms_working\jifen-cms\jifen-cms\node_modules\esbuild\lib\main.js:1151:28
at runOnEndCallbacks (E:\cms_working\jifen-cms\jifen-cms\node_modules\esbuild\lib\main.js:941:63)
at buildResponseToResult (E:\cms_working\jifen-cms\jifen-cms\node_modules\esbuild\lib\main.js:1149:7)
at E:\cms_working\jifen-cms\jifen-cms\node_modules\esbuild\lib\main.js:1258:14
at E:\cms_working\jifen-cms\jifen-cms\node_modules\esbuild\lib\main.js:629:9
at handleIncomingPacket (E:\cms_working\jifen-cms\jifen-cms\node_modules\esbuild\lib\main.js:726:9)
at Socket.readFromStdout (E:\cms_working\jifen-cms\jifen-cms\node_modules\esbuild\lib\main.js:596:7)
at Socket.Readable.push (node:internal/streams/readable:226:10)
at Pipe.onStreamRead (node:internal/stream_base_commons:190:23)

Can you publish new version to include package.json dependencies changes

I found you move some development dependencies to devDependencies. That's great, I want to use it.

But no new version published. Can you publish a version to include this change?

Nealder-Mead fails for Goldman-Price

Any idea why nealder-mead fails for the Goldman-Price function?

Goldman-Price is well behavied and it's usually used as test function.
https://en.wikipedia.org/wiki/Test_functions_for_optimization

var fmin = require("fmin");

// https://en.wikipedia.org/wiki/File:Goldstein_Price_function.pdf
function goldsteinPrice(x, y) {
    return (
        (1 + Math.pow(x + y + 1, 2) * (19 - 14 * x + 3 * x * x - 14 * y + 6 * x * x + 3 * y * y)) * (30 + Math.pow(2 * x - 3 * y, 2) * (18 - 32 * x + 12 * x * x + 48 * y - 36 * x * y + 27 * y * y))
    );
}

// var res = fmin.nelderMead((x) => goldsteinPrice(x[0], x[1]), [0.2, -0.18]); // Works
var res = fmin.nelderMead((x) => goldsteinPrice(x[0], x[1]), [0.2, -0.17]); // Fails
console.log(JSON.stringify(res)); // Should be x=0, y=-1