ralampay / ann

Artificial Neural Networks implemented in C++

Installation

$ cmake .
$ make

Usage

Training

$ ./train [path/to/config.json]

Autoencoder

$ ./autoencode_classify [path/to/config.json]

ann's Issues

Refactor Matrix Multiplication

Refactor Constructor and Network Config

Autoencoder Mode

• checks if we have an odd numbered topology (true case: 4, 2, 4. false case: 4, 3, 3, 4)
• checks if input size is equal to output size

json namespace not recognized

Within main.cpp, auto configJson = json::parse(str); shows error:
E0304: no instance of overloaded function "nlohmann::basic_json<ObjectType, ArrayType, BooleanType....
matches the argument list"

Edge Detector Config: specify target width and height of possible objects

Autoencoder one class classifier module

Given set of weights and datapoint, classify entity based on error rate

Save hidden nodes as feature vector

Backpropagation function

Non maximum suppression for edge detector

Save and load neural network configuration

cannot train

Hi, I follow your instruction building with CMake. However, I cannot train. I ended up with
/home/leanne/Dev/projects/ann/cmake-build-debug/train
Syntax:
train [configFile]

Process finished with exit code 255
I use CLion. I tried again over teminal, the result is same and I got
Syntax:
train [configFile]
as the final output.

What did I miss? Everything built fine.

wy aren't you calling delete? (ever)

Yep noticed that too. Might roll out a new version with different architecture. Got the concepts down first. Thanks. (I think the latest push has the delete stuff but it's largely messy). Thanks

Make all units as matrices

Load weight matrices from a file

Save detected objects after exiting program

Save weight matrices to a file

Can't train with more than 1 hidden layer

Autoencoder 1 class classifier from input file

Get training error

Use difference of consecutive errors as means to end training

Bug: Not training if hidden layer is greater than input (topology config)

[625, 50, 625] works but [625, 1000, 50, 625] doesn't

Problem when creating layers with more neurons than the previous layer.

Vector range problem.

Despite the fix you implemented(watched your video too!), the bug still persists.
The problem was that the matrix deltaWeights in Backpropagation() did not have the right dimensions to be looped over in:

for(int r = 0; r < tempNewWeights->getNumRows(); r++) {
for(int c = 0; c < tempNewWeights->getNumCols(); c++) {
double originalValue = this->weightMatrices.at(i - 1)->getValue(r, c);
double deltaValue = deltaWeights->getValue(r, c);

            originalValue = this->momentum * originalValue;
            deltaValue    = this->learningRate * deltaValue;
            
            tempNewWeights->setValue(r, c, (originalValue - deltaValue));
        }

}

(I couldn't get 'insert code' to work, sorry about the plain text)

The bug arises whenever deltaWeights matrix was of smaller dimensions than the tempNewWeights matrix.

The solution was to resize the deltaWeight Matrix to the same size of tempNewWeights before entering the nested loops above.

I've added a function to the Matrix source files that returns a resized array and hence the fixed code loop looks like:

Matrix* resizedDeltaWeights = deltaWeights->resize(
tempNewWeights->getNumRows(),
tempNewWeights->getNumCols()
);

for(int r = 0; r < tempNewWeights->getNumRows(); r++) {
for(int c = 0; c < tempNewWeights->getNumCols(); c++) {
double originalValue = this->weightMatrices.at(i - 1)->getValue(r, c);
double deltaValue = resizedDeltaWeights->getValue(r, c);

            originalValue = this->momentum * originalValue;
            deltaValue    = this->learningRate * deltaValue;
            
            tempNewWeights->setValue(r, c, (originalValue - deltaValue));
        }

}

And the implementation of resize is:

Matrix *Matrix::resize(unsigned long newRows, unsigned long newCols) {
Matrix *m = new Matrix(newRows, newCols, false);

unsigned long j = 0;

for(unsigned long i = 0; i < newRows; i++) {
    
    //if we've reached the boundry rows of the original mat, no left to copy.
    if (i > this->getNumRows() - 1) {
        m->setValue(i, j, 0);
    }else{
        for(unsigned long j = 0;j < newCols - 1; j++) {
            //if we've reached the boundry cols of the original mat, Set values beyond to zero.
            if (j >= this->getNumCols()) {
                m->setValue(i, j, 0);
            }else{
                m->setValue(i, j, this->getValue(i, j));
            }

        }
       
    }
    
}

return m;

}
Of course, delete resizedDeltaWeights later on.
Currently testing this implementation and it works well so far.

Confusing activationType and activationVal

In Neuron.cpp line 12 I don't understand why it's activatedVal == RELU and not activationType. Same question on line 18