carmenpopoviciu / neural-net

Basic implementation of a feedforward neural network

Neural Network

Basic implementation of a feedforward neural network.

How to use it in a nutshell

WARNING! Needs updating. Please refer to the rest of the readme for updated APIs

![neural-net-rec] (https://cloud.githubusercontent.com/assets/4638332/18604559/7c637cde-7c7e-11e6-889b-0ab166e675ba.gif)

Running an execution context for the net

Run npm start to start a node repl.REPLServer instance. The NeuralNet constructor function is already exposed to the REPL, so you can start creating your own neural net instances and train them straight away

Much <3 to Sebastian Gräßl for implementing this

Cleaning the snapshots folder

Run npm run clean-snapshots to remove all snapshot files in the snapshots folder

Create a net instance

let myNet = new NeuralNet();

or

let customConfig = {
  learningRate: 0.5,
  numEpochs: 2000
};
let myNet = new NeuralNet(customConfig);

if you are using a custom configuration.

Initialise the net based on a layer configuration

myNet.initialise({layers: [2,3,3,1]});

Initialising a net instance using a layer configuration implies defining the number of layers in the net, and the number of nodes in each layer.

The code above initialises that net's structure to a 4-layer structure. The input layer will consist of 2 neurons, the two hidden layers will have 3 neurons each, and the output layer just one neuron.

Initialise the net based on an existing snapshot

An alternative to the above method is to initialise the net with data from an existing snapshot.

let myNet = new NeuralNet();
myNet.initalise('path/to/snapshot.json');
myNet.predict([0,1]);

A snapshot is a .json file which contains the configuration details of a pre-trained net. The file contains a weights & biases configuration for each node on each layer, which will be used to initialise the nodes of the net. For a sample of the expected json object format, please check the sample snapshot.

Snapshots will prove to be extremely useful for use cases when one wants to train a net from an already pre-trained state. Snapshots also open up the possibilities as to where the actual net training needs to happen. One such example would be having a net instance trained in the cloud and then porting its resulting weights and biases to a local net.

Train the net

myNet.train([ [[0,0],[0]], [[0,1],[1]]]);

Once the net is initialised it can be trained with a given training data set. The training set can contain one or multiple items, like in the example above. The number of times the network will train on the given data set is based either on the default number of epochs or the one defined in your configuration. This means that for a number of epochs of 1000 for example, the net will process each entry in the data set 1000 times, before it is finished training.

Predict an output

myNet.predict([[0,1]]);  // hopefully outputs 1 ;)

neural-net's Issues

Error: Sigmoid: Can't compute the sigmoidFn for non numerical values, Got NaN

I always get this error when following Readme instructions:
"Error: Sigmoid: Can't compute the sigmoidFn for non numerical values, Got NaN".

neuralNet > let myNet = new NeuralNet();
undefined
neuralNet > myNet.initialise({layers: [2,3,3,1]});
undefined
neuralNet > myNet.train([ [[0,0],[0]], [[0,1],[1]]]);
undefined
neuralNet > myNet.predict([[0,1]]);
Error: Sigmoid: Can't compute the sigmoidFn for non numerical values, Got NaN
    at Object.sigmoid [as fn] (/Users/ahmadbarakat/Documents/GitHub/neural-net/utils/sigmoid.js:14:13)
    at Neuron.updateOutput (/Users/ahmadbarakat/Documents/GitHub/neural-net/neuron.js:110:35)
    at /Users/ahmadbarakat/Documents/GitHub/neural-net/neuralNet.js:372:16
    at Array.forEach (native)
    at /Users/ahmadbarakat/Documents/GitHub/neural-net/neuralNet.js:367:22
    at Array.forEach (native)
    at feedforward (/Users/ahmadbarakat/Documents/GitHub/neural-net/neuralNet.js:366:12)
    at NeuralNet.predict (/Users/ahmadbarakat/Documents/GitHub/neural-net/neuralNet.js:210:12)
    at repl:1:7
    at ContextifyScript.Script.runInContext (vm.js:37:29)

(refactor)net: refactor for multiple activation fns future support

The current implementation does not support custom activation functions other than the sigmoid Fn. Adding support for custom fns is important because some of these functions have huge benefits for the net's overall performance and prediction accuracy.

This task is meant to refactor the hardcoded sigmoid function in order to allow support for multiple activation functions (implementations of such functions will be added as part of a different issue)

(feat)net: add support for snapshots

A snapshot is a json file that contains a net's configuration at a given point in time: number of layers, number of nodes per layer, weights and biases for each node in the net.

A snapshot is extremely useful when we want to save the state of our net, after we trained it on a given training data set and want to resume training, at a point later in time, from where we left of. Not only that, but it also makes the state of our net portable to other net instances and gives us the option to initialise our net from an already pre-trained state, thus saving us the trouble of having to retrain the net over the same data set.

Adding snapshots support means 2 things:

1. Having the ability to save our net's state once it's been trained
2. Having the option to initialise our net based on a pre-existing weights/biases configuration